Business Intelligence & Business Analytics Blog

Dashboard Best Practices to Improve Outsourcing Deals

Dashboard de melhores práticas que melhoram a terceirização - do Gartner Group

Business unit managers look to sourcing managers for effective business-centric support measures, so dashboard metric views must be relevant. We give examples of effective dashboard processes, inputs, outputs and formats with which to report business metrics.

Key Findings

  • Sourcing managers, business unit managers, service provider managers and contract managers can use this report to review and implement performance dashboards focused on outsourcing service provider management. Such dashboards are crucial for maintaining multisourced environments.
  • Gartner's IT Key Metrics indicate that 27% of deals engage the business to tailor metrics to business requirements, 41% have business metrics defined by IT, 23% are based on service providers' standard metrics, and 9% have no business metrics.1
  • Performance dashboards for IT outsourcing generally focus on IT technical and IT process performance metrics that do not adequately indicate the business effectiveness of IT outsourcing.2
  • Primary outsourcing service levels typically address IT technical and IT process metrics, while ignoring service provider management metrics and business performance metrics.
  • Business unit managers are not interested in current IT metrics and are increasingly frustrated with a measured performance status of "green" when the impact on the business is "red" — something known as the "watermelon effect."3

Recommendations to IT Sourcing Managers

  • Divide service-level metrics into two categories, to include IT technical and IT process metrics on the one hand, and business unit key performance indicators on the other.
  • Align the goals of business unit managers with IT deliverables by jointly developing effective metrics that support the business.
  • Develop and manage effective metrics to encourage appropriate service provider behavior in order to achieve your desired IT and business outcomes.
  • Include motivational metrics that encourage service providers to deliver innovation throughout the life of your outsourcing contract.
  • Implement dashboard reporting of metrics using a "fewer is better" approach, and always strive to deliver appropriate dashboard reports.

Analysis

Sourcing managers lead the transformation of IT, manage ROI and establish the size and skills of the retained organization to ensure effective financial results and resourcing. They need to be able to develop and implement the right performance measures and tools. The approach currently taken typically centers on the performance of IT technical and process outcomes, and the IT metrics often reflect performance measures developed, measured and reported by the IT organization in an attempt to define its value to the business. This often leads to a monthly reporting cycle in which metrics are meticulously maintained, but rarely of value beyond the IT organization. This approach ignores the value of IT performance to the business and thus creates several problems:

  • A need for effective metrics and dashboards to measure performance and value.
  • IT technical and process metrics that are used to measure the results of deals but that do not reflect business outcomes and are therefore not valuable beyond the IT organization.
  • Business unit managers frustrated by IT technical and IT process metrics that indicate effective IT performance but are ineffective for judging business success.
  • Service providers that, while meeting defined IT technical and process requirements, are not necessarily motivated by the metrics to do the right thing for the client's business.
  • Service providers not "owning" the complete process, a situation that thwarts the end-to-end measures that should be captured in an end-to-end dashboard.

IT Technical and IT Process Metrics

IT technical and IT process metrics have two key purposes: (1) to determine whether basic IT requirements are being met; (2) to measure the extent of delivery success or failure, and in the latter case focus root-cause analysis and restore IT operations and processes to an effective level. Without IT technical and IT process metrics, an IT organization can quickly become ineffective and perform unreliably.

A case in point was faced by one of Gartner's CIO clients who inherited a dysfunctional IT operation that had become complacent about system uptime. The CIO quickly implemented an effective service-level assessment with the company's service provider to focus on IT technical measures that identified problems and set performance improvement goals. He quickly had the two organizations delivering effective service. He also introduced a formal monthly meeting to address service-level performance and a weekly meeting to evaluate the root causes of any recurring problems and solve them. Soon all the IT employees were focused on maintaining service at a level that met the metrics.

From cases like this we know that the use of IT technical and IT process metrics are important for the delivery of consistent services for a business. Effective management of the service provider is also important to ensure that service levels are met and that delivery is driven by continuous improvement of its processes. The results are IT organizations that consistently deliver effective services.

Examples of IT technical and IT process metrics include:

  • Server uptime — 99.98% of the time, not counting maintenance windows (IT technical)
  • Network uptime — 99.999% of the time (IT technical)
  • Help desk call resolution — 80% first-call resolution (IT process)
  • Help desk time to answer — up to 45 seconds, 90% of the time (IT process)
  • Help desk user satisfaction — 85% satisfied or better with tickets and biannual survey (IT process)
  • Application development — less than 5% defect leakage to production (IT process)
  • Desk-side support problem resolution severity levels (1, 2 and 3) — Level 1 problems solved in two hours, 90% of the time (IT process)

For more information, see "Toolkit: Outsourcing Contract Statement of Work, Service-Level Agreement and Price Attachment" is one of 34 elements that make up an IT Outsourcing contract. These are just examples of service-level metrics — a complete SLA would also include detailed descriptions, penalties, earn-back criteria, examples of failure and formulas for measured performance. The IT technical and IT process metrics cited above are from SLAs that Gartner has collected from over 500 outsourcing deals and tracked on an annual basis for the past 15 years. The levels of service they indicate should not be considered as standard or a best practices; rather they denote common, understandable levels that are generally considered a reasonable basis for meeting the requirements of business units. Client organizations, together with their service providers, can use operational metrics to monitor performance, communicate the results, and provide input to a continuous improvement process.

Reporting and Motivation

The standard reporting process for IT technical and IT process metrics includes a monthly review to determine whether the metrics have been met and assess the penalties for any failure to meet them. (Penalties might include the loss of a bonus for an internal team or a reduced payment to the service provider, though the latter would need to be supported by full earn-back opportunities in order to encourage appropriate behavior from the service provider in future.) In addition, there should be a requirement to deliver a root-cause analysis of performance failure within a reasonable period from the time of reported failure — typically three to five business days. In cases where there is a history of failure, the IT technical and IT process metrics should be of interest to the business unit leaders.

A dashboard that captures the elements of IT operations metrics should include the day-to-day performance of the IT team. The example shown in Figure 1 is of an effective desktop support dashboard covering responsiveness, resolution, image management and customer satisfaction. It represents a compilation of a month's worth of data, which should be reported in the monthly service-level review meetings. The dashboard should be representative of metrics that are tied to penalties for poor performance (indicated by red circles).

Data for IT technical and IT process metrics are typically readily available and, if defined properly, both this data and the associated outcomes are unarguable. This data should be supplied to business units and other appropriate stakeholders in a standard monthly report. To minimize reporting requirements, many reporting strategies call for exception reporting only.

Figure 1. Example of an Effective Dashboard for Desktop Support

Figure 1.Example of an Effective Dashboard for Desktop Support
Green: high level of service
Yellow: meeting minimum requirements of the service level
Red: below the minimum requirements of the service level

Source: Gartner (June 2012)

Service Provider Management Metrics

Management processes for service providers should encourage them to behave in a way that delivers performance in the best interests of the client's organization. Among other things, there should be metrics to foster innovation (discussed further below), including initiatives to lower cost, and the implementation of processes that deliver simple, seamless and repeatable outcomes. At present, such metrics are often absent or neglected because they are generally less quantifiable from a cause-and-effect perspective, but they nevertheless represent important means for improving an service provider's behavior. These metrics may evolve over time as the measurement process matures and the client organization develops better processes. Service provider relationship performance metrics may also be used. For more information, see "Assessing Your Outsourcing Relationship's Health" and "Toolkit: Outsourcing Contract Innovation Plan Attachment" (note: the latter document has been archived; some of its content may not reflect current conditions).

When service providers propose innovative ideas, there is a need to measure their progress toward the anticipated new state, such as a defined innovative environment. The innovation process should answer the following questions: "What can we do?" "What should we do?" and "What will we do?" Answers to these questions are found by conducting a series of innovation workshops to define, prioritize and fund innovative improvements to technical, process and business outcome activities.

Service providers commonly aggregate agreed service metrics. Since the service metrics commonly used are technical and process level, there is a need to include metrics that drive continuous improvement throughout the life of a contract.

Examples of metrics relating to continuous improvement, service provider relationship and service provider innovation include:

  • Help desk incidents per user per month declining from 1.0 to 0.6 over two years
  • Help desk cost per user per month decreasing by 10% a year to reflect the unique-incident reduction expected from the deal
  • Effective relationship rating of 3.5 or better in areas of technical, financial, commercial and service provider feedback
  • Effective account management with successful escalation of issues for resolution, measured by time to resolution and resolution satisfaction.
  • Flexibility rating — a measure of how the service provider addresses new business needs or changes that may not be addressed in the contract
  • A formal innovation process with biannual meetings that deliver business impact savings totaling 20% of the annual outsourcing fees
  • Server processor utilization increasing to 68% over two years, with server virtualization increasing to 80% over two years (as applications allow)
  • Storage utilization to reach 75% of the available storage capacity in two years as storage is consolidated and storage processes are improved

The aim with such metrics is to motivate the service provider to deliver services and prices in an innovative way throughout the life of the engagement. This is accomplished by linking them to behavioral drivers, including financial rewards. Enterprises will also have to invest money from time to time to implement innovative ideas, as well as the technical and process improvements identified above (see "Improve Your Outsourcing Deal With the Proper Gainsharing Structure").

Reporting and Motivation

The standard reporting process for service provider continuous improvement and innovation metrics includes a monthly or quarterly review of metrics, depending on the "delivery gates" to be met. This process addresses whether metrics have been met and whether the expected business impact can be documented and approved by both the service provider and the client organization.

Reporting may be required on an exception basis. For example, root-cause analysis of missed performance should typically be required within three to five business days of the miss. Such reporting focuses attention on adjusting processes to improve performance.

When metrics are achieved, a shared-savings bonus may be generated for the service provider, while the client organization benefits from cost savings. The performance impact of meeting these metrics may result in a step-down price for the services delivered for the rest of the engagement. The impact of metrics may be unique to each engagement, so both parties need to define the level of the service metrics.

Since billing can be impacted, to be effective, continuous improvement, innovation and gainsharing arrangements must form a contract-based statement of performance measurement that rewards service providers for specific performance. These metrics must be specified in the contract and be designed to influence the service provider's behavior and enable both client organization and service provider to manage their relationship effectively. Used properly, continuous improvement, innovation and gainsharing metrics can foster a positive relationship between client organization and service provider, and encourage the service provider to "go the extra mile." Used improperly, however, they can cause significant relationship problems, especially within the client organization, as it learns that it is paying additional fees not commensurate with the business value delivered. This can lead to the relationship's demise.

A good dashboard captures elements from useful survey data addressing the areas of technical and process expertise, data about cost and financial capabilities, data about commercial validity, as well as the service provider's feedback. In the example shown in Figure 2 the score is weighted for an overall rating to answer the question "Do we have a good relationship?"

Figure 2. Example of an Effective Dashboard for Relationship Management
Figure 2.Example of an Effective Dashboard for Relationship Management
Source: Gartner (June 2012)
 

Business Metrics

Business metrics measure the impact of IT on the client organization's business, as well as the impact of the service provider's performance on the client organization's revenue, the cost of producing the end product or service, and customer retention or satisfaction. The business metrics should be formulated with the help of business unit managers, the IT organization and the service provider.

Service providers can play an important role as they deliver services that may draw on innovation or findings from similar industries or accounts. The expertise required for this may extend beyond the normal areas of IT, and thereby impact on the processes found in business organizations. To be effective, business metrics should link to business transactions or capabilities that are enhanced by more effective IT processes. The metrics should encourage all involved parties to deliver in the best interests of the business unit. The metrics themselves may evolve over time as the measurement process matures and the business unit calculates the revenue and expense impact. Their purpose is to have a direct impact on the business in terms of revenue, cost or customer satisfaction.

Examples of business metrics for various industries include:

  • Radiology equipment utilization
  • Automotive assembly line uptime
  • Product production defect rate
  • Retail website transaction capacity
  • Website sales to visits percentage increase
  • Reduced time in weeks for product development in the IT life cycle
  • Claims process improvement of 10% per agent over a given time frame
  • Reduced settlement times for insurance claims
  • Reduced customer time in queue for higher satisfaction
  • Develop a mobile appliance app to encourage purchase of product
  • Technical enhancement with store kiosk to improve customer experience
  • Reduced supply chain error rate and improved fulfillment time

A recent success in this regard was the presentation by the CIO of a fast-growing retail chain of an IT strategy for the next three years. It was full of business metrics and plans to use technology to improve the customer experience in the company's stores, in anticipation of increased revenue. There was a surprising lack of discussion about infrastructure and software projects, as the CIO had outsourced provision of the company's basic IT requirements and transformed the IT team to deliver results for the business.

Reporting and Motivation

The standard reporting process for business metrics should include a monthly or quarterly meeting of the client's business and IT managers and the service provider to review the metrics. These meetings and associated reports should address whether the metrics have been met, potential future problems and the expected business impact, which can be documented and monitored by all parties. These reviews should be of particular interest to business unit managers when they include forecasts of the potential impact of the client's service provider(s) in terms of progress toward meeting its business goals. All parties should be measured as they work together to meet delivery gates over the stipulated time period.

An effective dashboard that captures business metrics should include business data captured from key areas identified by business unit leaders. In Figure 3 the dashboard is reporting on the business metrics for radiology examinations. The ability of IT to maintain effective connectivity to these vital business machines is reflected in the dashboard metrics as indicated.

Figure 3. Example of an Effective Dashboard for Business Metrics
Figure 3.Example of an Effective Dashboard for Business Metrics
Source: ASG
 

Conclusions

Sourcing managers, business managers and IT managers have different demands, perspectives on IT services, and perceptions of what constitutes quality IT services. Each area has a unique perspective that dictates the measures that should be present in a dashboard.

Sourcing managers need to ensure that the metrics and dashboards selected do a good job of representing the strategic objectives of outsourcing deals. If the objective is simply to deliver IT services more efficiently year over year, then technical and process metrics will suffice. But if the focus is business operational efficiency, they should combine IT technical and IT process views with specific service provider management metrics to measure the services provided. If the objectives are business-oriented and the services need to be represented by their impact on business metrics, they should ensure the dashboard reflects these business-driven metrics as well.

Sourcing managers should use metrics and dashboards that will enable effective management of a service provider's performance. They should not waste time defining business metrics to manage a service provider if it is only expected to deliver, and only measured on, more efficient delivery of existing services. They should also find out whether their company's business managers have an interest in the IT services being delivered, so that they can meet their needs via an effective dashboard.

Gartner Group

Comments powered by CComment