Benchmarking Outcomes That Matter Most to Patients: The Globe Programme

INTRODUCTION TO VALUE-BASED HEALTHCARE

In 2012 the International Consortium for Health Outcomes Measurement (ICHOM) was founded as a non-profit organisation to develop Standard Sets of outcome measures for different medical conditions and populations, as well as drive their adoption by healthcare institutions. It is our belief that the systematic measurement of Standard Sets of outcomes by institutions around the world will enable, for the first time, global outcome comparisons. We think this will catalyse a new wave of innovative learning for healthcare professionals, as institutions will be able to see where the greatest outcomes are being achieved and then learn from the processes that they have resulted from, as well as really inform patient choice. ICHOM has now created Standard Sets of outcomes that matter most to patients for 21 medical conditions, including prostate cancer, cataracts (CAT), hip and knee osteoarthritis (HKO), and stroke amongst many others.

The concept of value-based healthcare^1,2 has been defined by Prof Michael E. Porter, Bishop William Lawrence University Professor, Harvard Business School, Boston, USA, and Prof Elizabeth Teisberg, Full Professor, Dell Medical School, University of Texas, Austin, USA, as the patient outcomes achieved per dollar expended.^3,4 Outcomes are the results people care about most when seeking treatment, including functional improvement and the ability to live ‘normal’, productive lives. Current metrics for care across many conditions tend to capture processes and costs and do not measure whether they achieve the outcomes which matter most to patients. The process of developing a Standard Set has been previously described.^5-7 In developing globally-agreed-upon Standard Sets of outcomes for different medical conditions and patient populations, we aim to enable providers to measure the most important outcomes for their patients and then compare themselves, in a consistent manner, with other countries around the world.⁸

The application of performance benchmarking, transparency in reporting, improved performance, and collaborative learning, gained from the processes that the best performers share, ultimately informs patient choice towards value. The ICHOM Global Outcomes Benchmarking (GLOBE) programme was launched in May 2016 and aims to globally compare standardised outcomes between international partners to enable the identification of treatment paradigms that are more effective, leading to improvements in healthcare outcomes. The launch of two prospective, non-interventional, observational pilot studies using the ICHOM HKO,^9,10 and CAT^11,12 Standard Sets which include patient-reported outcome measurement (PROM) instruments, assess the feasibility of collecting outcome data from multiple international institutions, and provide risk-adjusted benchmarks. The GLOBE pilot programme will bring together highly motivated participants including institutions, hospitals, and registries that are keen to compare outcomes and learn from each other.

THE GLOBAL OUTCOMES BENCHMARKING PROGRAMME

The idea of performance benchmarking as a mechanism to drive improvement is by no means a new one. A myriad of examples in the literature indicate that, when thoughtfully organised and well managed, the systematic collection of standardised data sets accelerates improvement upon measurement of process, structure, and outcome indicators which inherently reduces total cost (see Table 1).

At its most basic level, benchmarking sets the bar with regards to what is achievable, providing motivation for clinicians, care teams, and institutions to adopt best practices and/or evidence-based guidelines, or to innovate on their own. When combined with efforts to identify the underlying factors that lead to high performance, and co-ordinated quality improvement activities focussed on disseminating these lessons, the benefits of benchmarking can be amplified.

ICHOM’s GLOBE programme builds on available examples in two important ways. Firstly, it will be the primary truly global initiative, unlike numerous existing national or regional benchmarking programmes. By including institutions from around the world, we expect to see wider variation with regard to both processes and outcomes, thereby expanding the possible research questions that can be investigated and increasing the likelihood of discovering new best practices in pockets of innovation and excellence. Secondly, this will be the first initiative to emphasise PROM such as mobility, pain, and quality of life. This is critically important if we are to guide improvement efforts in a way that is compatible with patients’ preferences and values.

Global Outcomes Benchmarking Pilot Programme Objectives

The GLOBE pilot programme is a proof-of-concept initiative that aims to test the feasibility of setting up an international benchmarking programme based on the ICHOM Standard Sets. The pilot has five primary objectives:

a. Identify and overcome legal and technical hurdles to aggregating data from an international set of providers

b. Assess the appropriateness of the ICHOM CAT, and HKO sets for benchmarking and define the required changes

c. Design risk adjustment methodologies to adequately adjust participant outcomes between sites

d. Deliver risk-adjusted outcomes, reporting to participating institutions/hospitals so that they can understand their relative performance

e. Determine how pilot sites can leverage outcomes data to learn from one another

Pilot Overview

International Consortium for Health Outcomes Measurement community

ICHOM supports institutions, hospitals, and organisations in implementing Standard Sets of outcomes and develops case studies of examples of outcome measurement around the world. These have been described in further detail in case studies,^19-21 and an example of implementation has been described in Box 1. The importance of supporting implementation of the Standard Sets is two-fold: scaling the effective use of standard PROM instruments for outcomes; and for decision-making in performance benchmarking, transparency, and narrowing variation toward improved outcomes. The programme design and success of the GLOBE pilot programme is largely based on the ICHOM community of implementers of ICHOM Standard Sets.

Conditions

The two ICHOM Standard sets, HKO⁹ and CAT,¹⁰ were chosen for initial examination in the GLOBE pilot. According to a 2010 assessment, 51% of world blindness is caused by CAT,²² and HKO has been ranked as the 11^th highest contributor to global disability in the 2010 Global Burden of Disease.²³ It is therefore globally appropriate to start with these two conditions. Using the lessons derived from this pilot, ICHOM intends to launch benchmarking programmes across additional Standard Sets going forward.

Participants

The GLOBE pilot participants were recruited from ICHOM’s international group of strategic partners, members of Standard Set implemention communities, the working groups of the ICHOM HKO, and CAT Standard Sets, and from the ICHOM global community. Participants determined which hospitals within their institutions would be participating. All self-identified as being interested in the ICHOM GLOBE pilot given their engagement with the ICHOM community. Participating institutions were required to sign a memorandum of understanding (MOU) governing data privacy and security, namely requiring the de-identification of data gathered by participating institutions, prior to transmission of data. As the need and requirements for ethics boards and patient consent varies by country, the responsibility of acquiring regulatory approvals and/or ethics committee review for participation rested with the institution. Each institution sought approval for i) the use of data for benchmarking analysis and ii) the transfer of de-identified patient data outside the operating jurisdiction. The project is best classified as a service evalution or clinical audit that does not require informed consent as no patient care is impacted. The GLOBE programme provides a resource guide, study plan, data dictionary, and MOU to aid approval processes.

As of December 2016, the institutions identified in Figure 1 had agreed to contribute data to the pilot. Due to the popularity of the programme, recruitment is ongoing for both pilots at the time of this publication.

Outcomes collected

The endpoints collected are based on the ICHOM HKO and CAT Standard Sets that have been previously detailed and published.^10,12 Standard Sets include baseline conditions and risk factors to enable meaningful case-mix adjustment globally, ensuring that comparisons of outcomes account for differences in patient populations across not just providers but also countries and regions. High-level treatment variables are included to allow stratification of outcomes by major treatment types. Additional measurement time points have been added, where appropriate, to decrease the timeline needed to observe outcome differences between sites. Additionally, a pilot objective is to test the Standard Sets for the purposes of detailed benchmarking and identify required modifications to ensure the ICHOM HKO, and CAT sets can be used for benchmarking going forward.

Methodology

Both the CAT and HKO programmes are prospective, non-interventional observational studies that are expected to run for 15 and 18 months respectively. Each pilot is divided into three main phases: Programme Design, Data Collection, and Analysis and Reporting, each with a specific goal (Figure 2).

Programme design

The aim of the Programme Design phase was to recruit potential pilot participants and provide a framework to work together to prepare sites to sign up for the pilot. There are four major engagement steps that were explained to onboard pilot participants:

a. Onboarding session: For each interested institution, a call was set up that included representation from the hospital leadership, clinical and programme management teams to discuss the pilot methodology and objectives. The aim of this session was to assess alignment with pilot objectives and ensure adequate resources were available at the hospital site to execute the pilot objectives.

b. Gap analysis assessment: All interested sites completed a gap analysis which assessed their sites’ overall alignment with the ICHOM Standard Sets. Sites were vetted to ensure that they collected the correct outcomes, which were based on the ICHOM defined outcome definitions, at acceptable time points. Sites that diverged from the ICHOM Standard Set recommendations or collected an incomplete set were coached on what gaps needed addressing to participate in the pilot.

c. Legal session: An exploratory discussion was held with interested sites to understand the legal and regulatory requirements for their participation in the pilot programme. These discussions were used to inform the legal contract ICHOM developed for this pilot discussed in Box 2.

d. Technical: A technical discussion was held with a member of the Technical/Informatics department to provide an overview of the data transfer process and assess if sites were technically able to extract and transfer the complete ICHOM Standard Set data.

Data collection

During the data collection phase participants submit data through a secure ftp protocol to ICHOM’s data partner’s (ICON) server on a monthly basis and receive validation or data quality reports post-data submission. This process is followed so that institutions and their hospital sites can receive timely feedback on their collection and make any required adjustments as early as possible in the pilot. An example of the data collection process for the CAT pilot is shown in Figure 3.

Analysis and reporting

The final phase of the pilot will focus on i) testing and implementing risk adjustment techniques to allow for adjusted comparisons between institutions and their hospital sites, ii) assessing potential visualisation techniques to allow for comparisons between providers, and iii) developing reporting options to showcase relative performance and facilitate collaborative sharing of best practices.

CURRENT STATUS AND INITIAL LESSONS

Currently the HKO and CAT pilots are in the data collection phase with both expected to deliver results by the end of 2017, after which each pilot is expected to transition into a fully defined benchmarking programme. To date, the pilots have overcome a number of potential hurdles including, but not limited to, defining a legal framework (Box 2) to allow for data sharing, developing a data transfer process that accommodates institutions with different collection systems and technical capabilities, and identifying new variable definitions/requirements to help capture process differences between sites.

EXPECTED OUTPUTS AND NEXT STEPS

The HKO and CAT pilots will provide ICHOM with important lessons to allow it to develop a benchmarking programme that draws outcomes data from an international set of contributors and both analyses and reports back to providers on their relative performance. Lessons learned from evaluating risk adjustment within these two pilot conditions will also inform future Standard Sets and improve upon an ongoing benchmarking programme.

The risk-adjusted benchmarking performance data will provide an immense opportunity to improve patient care by identifying differences in hospital performance that can be used to facilitate learning programmes between institutions. Over time, the programme will likely incorporate additional data sources and expand its scope of analyses. Potential options include but are not limited to the identification of determinants of hospital performance, an understanding of individualised treatment response at the patient level, or an assessment of the most efficacious set of medical interventions for a given patient subpopulation. Outcomes data sharing programmes like this will provide the largest repository of structured data that has been collected to date, linking both underlying patient characteristics and treatment options to patient outcomes. In this exciting future, we expect the ICHOM GLOBE programme to serve a pivotal role in facilitating, sharing, and utilisation of data at an international level.