Expand +



5 Steps to Measuring COE Value

by Ken Grady

August 11, 2009

How to create metrics for an SAP NetWeaver environment has puzzled IT managers since the platform’s initial release. You know you’re doing a good job, but how do you prove it? Learn a five-step approach you can use to demonstrate the value of your SAP center of excellence.

The dirty little secret in IT has always been that it is notoriously bad about transparency around costs. Few IT people can answer the questions, “What is the current support burden? What does it cost to do what you do?” IT is always the first to complain that it spends too much time and effort on the support burden, but it can’t convince executive management to fund the initiatives that IT knows would contribute directly to the company’s strategic objectives.

You know whether you’re doing a good job, but how do you move from anecdotal evidence to a yardstick used to demonstrate what you add to the mix? IT departments fear being considered unallocated corporate overhead, instead of the value-creating organizations they know themselves to be. The reality is that IT will never be able to link itself completely to revenue or cost-saving programs, so it needs to continually demonstrate that it provides the greatest benefit for the least possible cost.

Developing Measurements of Success

Creating metrics for the SAP NetWeaver environment is a task that IT departments have wrestled with since its inception. Here is a five-step approach you can implement to demonstrate the value of your SAP Center of Excellence (COE) to your executive team, users, and external customers. I didn’t invent any of this, but I have applied the best practices that I have learned from integrators, consultants, and others using a deliberate approach.

First, you need to be aware of one important prerequisite without which no metrics effort can succeed. You need to develop a common language — including a common set of expectations — with the business; this is one of the most overlooked elements in a measurement scheme. Throughout this process you will communicate with your customers and your management, so it’s essential that you speak the same language.

For example, one common IT metric is uptime. In today’s federated landscapes, a single user might touch a dozen different systems over the course of his or her workday. However, if system one goes down at one time, system two at another time, and system three at yet another time, the user has no idea that on average, the uptime was 98%. All the users know is that they were only able to use the system approximately 90% of the time.

Step 1: Define Where You Are Today

The old saying, “You can’t manage what you don’t measure,” establishes the ground rules for the first step. You must begin by measuring your total support burden using definable operational metrics. From there you can develop baselines, from which you can extrapolate service levels and quantifiable metrics. Yes, it’s easier said than done.

Many IT managers believe that defining the metrics by which you will measure your organization is one of the hardest things to do, but it’s really not all that difficult. From your conversations with the business during the service-level agreement (SLA) process, you should have some idea of what types of performance metrics are important to them.

The most common metrics have already been defined in the form of KPIs and by the Information Technology Infrastructure Library (ITIL), a set of best practices for IT service delivery. Be sure to make use of ITIL practices to define incident and change management processes, as necessary. Once you have selected the metrics and established the benchmarks, create an operational dashboard to track your progress.

One of the global support metrics my organization tracks is incident resolution. “Sample Dashboard Graphics,” below, illustrates two ways to measure the support burden: incident by business unit and by time-to-resolution. These examples aren’t particularly granular. Typically, I drill down to get answers; my business partners only want to see results and progress. Any businessperson can understand these charts, the trends they represent, and what they mean.

An operations dashboard can help you monitor the support burden. (This is hypothetical data.)


One thing to understand is that the business finds these charts — and others like them — extremely interesting for about two months. Then, they aren’t very attentive anymore. Don’t confuse the business’s lack of attention with a lack of interest. This data is of great use to IT internally, whether executive management still reads the charts or not. Once you establish credibility through these reports, however, you don’t need to have this same conversation every month. As “Identifying the Problem Spots,” below, shows, you can use this data to identify trends and deploy resources.


Using an operations dashboard can help you determine your problem spots by applying the data to identify various trends or issues.



Recently, my CFO mentioned that support costs were too high. I used a report similar to this one to show him that his department had called for support approximately 400 times in the previous month. Then, I asked him how many of those requests he didn’t want my team to answer and how much he would save per call. This turned the conversation from an emotional argument about budgets to a less-charged discussion about quantifiable data. This approach provided him with a level of comfort as to how IT was spending time and money. I can find many ways to “shift spend,” and I can describe the impact of each. Being able to quantify my support burden makes these conversations a great deal easier.

Step 2: Understand Your Customers’ Viewpoint

Even if you think you are performing well, it doesn’t necessarily mean that your customers think you are, and perception often governs reality. My organization takes great pains to find out what our users are thinking. You can use customer satisfaction surveys and other measures to understand the pain points of both the average user and management. From those measurement tools, you can identify which activities users perceive as having high value, and through this, find quick wins that will earn you high marks from customers without negatively affecting your support burden.

My organization surveys its users four times a year, reaching a different 25% of the workforce each time. Each survey should be concise and consist of three parts: Quantifiable questions, open-ended questions, and questions about demographics. The following are examples of quantifiable questions; base your questions on what’s important to your company.

  • How important is SAP to your daily business function?

  • How satisfied are you with the SAP functionality in supporting your daily business function?

  • How satisfied are you with the SAP support you receive from IT?

You can also capture the responses to open-ended questions in your customer satisfaction survey and analyze them for trends. “Response Trend Analysis,” below, shows you a series of sample categories identifying some of the trends you might find, including training, user metrics, performance, and so on.


The open-ended responses have been separated into a series of buckets to identify various trends and concerns.



Looking at the results, you can see that the user interface (UI) is a significant concern among users. But how much work should your team spend trying to redesign the standard SAP interface? Not a lot. The second greatest concern is training; you can do something about that. By increasing training, you can make better use of your SAP system as well as improve what you currently use.

Equally important is the demographic data. You need to know what all the users think about the IT services they receive — from the CEO to the rank and file — but you also want to know how each individual group feels. This will tell you whether you’re reaching and addressing the key opinion leaders and the key job roles. By inserting two or three questions into your survey regarding demographics, your results can typically show you exactly where you have problems and why.

Step 3: Map Your Delivery Capability and Utilization

Before you change for the better, you have to understand your capabilities. You should perform an ongoing inventory of your internal and external utilization, list your skill sets, and identify the critical gaps. Now that you know what your users think your priorities should be, you want to look at your resource utilization to see if your resources are deployed in those areas. If not, how should they change?

All members of my team must track monthly what percentage of time they spend on which activities. This is not a precise exercise, like a consultant figuring billable hours, but it has to be accurate enough to determine trends.

Once a month the members of my team fill out a tracking sheet, which takes less than a half-hour of their time. From that, I can get an analysis of full-time equivalents (FTEs) per activity and generate a report that compares resource utilization against the value-add. “Resource Utilization vs. Value-Added Service Delivery,” below, shows that the resources are divided into four categories: projects, support user training, legacy support user training, and administrative/paid time off activities.


Measuring resource utilization enables you to measure the shift from “keep the lights on” to “value creation” of
internal resources and provides the hard data needed to plan necessary operating expenses.



My objective is to grow the projects bar so that my team spends more time adding value and less time on the care and feeding of SAP. This chart is not exact, but it’s good enough for a conversation with upper management. From here, you can determine whether you are moving in the right direction.

Step 4: Align Your Functional Map with Key Business Objectives

Make sure your application functional map aligns with the key business objectives and strategy. From the intelligence derived in step 3, you can plan future investment and identify upcoming needs according to areas of strategic value. Then, construct a roadmap that you can take to the business for ratification.

Based on benchmarks, user feedback, and resource utilization trends, your roadmap should show current and future SAP deployments graphically by department and by function, SAP and non-SAP. Align it with your business objectives and strategy to plan key investments. This represents a great number of conversations between IT and business, alignment with corporate priorities, budget analysis, and possible future investments.

From this process, you can determine what is important and where you can get the most value. The reality is, you can’t do everything, but you can come to a consensus with the business and plan how to incrementally handle your technology investments from that time forward.

Step 5: Create Benchmarks for Improvement

You know where you’ve been. You know how well you have done what you said you would do. Now it’s time to define where you’re heading. With benchmarks in place that the business agrees to, you can shift all future budget conversations from cost to value because you know the demonstrated transparency of all your delivered services.

Benchmarking the support burden takes several months of continuous data collection. Defining those benchmarks can also take a long time making certain you are measuring the service levels that your business cares about. However, by taking the time to create the measurements that the IT team, business leadership, and end users can all easily understand, you can establish clear and quantifiable expectations for delivering service.

One of the ways to establish those mutually agreeable metrics is to create SLAs. Although it is outside the scope of this article, creating an SLA accomplishes the following objectives:

  • You come away with an objective, demonstrable, and agreed-upon measurement for success with the business

  • You develop a meaningful common language between the SAP team and the end users (e.g., “What is a high priority?”)

  • You create an environment where the SAP team can feel the goal is achievable versus always feeling pressured by undefined (but high) expectations

  • Resource and prioritization planning become more objective equations with measurable data (e.g., if you apply N more of Y type resources, service levels will increase by X%)

From an IT support perspective, SLAs make support an objective equation, not a constant effort to placate the squeaky wheels. They are also starting points for conversations with the business. I constantly review SLAs with my customers and craft them in business terms, not tech-speak, so they are meaningful to both parties.

I also go over the SLAs with my COE team because I expect them to populate a lot of data. Although I try to make populating the data as easy as possible, it’s still a chore. It’s critical for the entire team to understand why you have to make it part of your day-to-day activity.

The metrics created in step 1 define how well you are delivering IT services to your customers. The metrics you define in this step delineate how you intend to improve over time. Some of the metrics you can use to measure delivered value are:

  • The cost per user

  • The “keep the lights on” vs. the “value creation” investment

  • The total spend amount as a percentage of the operating expenses

By tracking these metrics, you can focus on continual improvement. “Total Spend Per User, ” below, represents an example. The nice thing about this metric is that it’s relatively insulated from typical business changes, such as a merger or an acquisition. The objective here is to grow the spend per user in the value-creation area and to demonstrate improvements in efficiency in the maintenance spend.


This example separates spend by standard support burden (i.e., “keep the lights on”) from value-creation activities such as projects and enhancements.



With a narrow selection of metrics of this type, you can shift your resources to meet the organization’s objectives and demonstrate the reasons and value in the kind of terms that any businessperson can understand.

Developing Measurements of Success

Every IT organization faces the constant challenges of budget restrictions, the need for organizational flexibility, and how to balance service capabilities. It’s critical to be able to provide the transparency and measurements that can help you and the business’s management to make timely and well-informed decisions for strategic investments and to ensure that existing investments deliver the expected results. You can then use this information to define clear and understandable allocation/consumption-based costs, SLAs, and so on.

Every business and organization is somewhat different, and you need to define the metrics that measure your company’s key and relevant outputs. However, the categories reflected here fit within the ITIL framework and apply across industries (and across COEs for other applications as well).

This is no quick fix. This process took several months, hundreds of hours of work, and a tremendous amount of determination. It’s not glamorous, but it is worthwhile. In my current position, I now have 12 to 18 months of data, and these metrics have become an extremely valuable tool.

Implementing a benchmarking program like the one above will shorten a lot of your conversations with the business and with the executive team. You will spend less time defending and more time planning your growing IT capabilities.



Ken Grady is CIO and head of ERP for Novartis Vaccines and Diagnostics in the United Kingdom.


An email has been sent to:

More from SAPinsider


Please log in to post a comment.

No comments have been submitted on this article. Be the first to comment!