Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations
Jakub P. Hlávka, PhD; Pei-Jung Lin, PhD; and Peter J. Neumann, ScD
Several alternative payment models (APMs) are being piloted to address affordability, equity, and quality-of-care challenges in oncology care. Of these arrangements, the Oncology Care Model (OCM), developed by the Center for Medicare and Medicaid Innovation (CMMI), is among the most extensive, covering about 200,000 chemotherapy episodes annually. The OCM relies on multiple quality measures to determine the level of payment for each provider, with the goal of incentivizing higher-quality care in a cost-effective manner. The OCM’s payment design is described in eAppendix A
(available at ajmc.com
Despite innovations in the payment landscape, limited consensus exists about what constitutes indispensable quality measures in oncology. The absence of such consensus may not only limit the development of better payment models, which increasingly link payment to quality of care, but also result in a lack of agreement on how value should be defined (and demonstrated) in an era of innovative, ever more expensive cancer therapies. In 2016, a roundtable of national experts in cancer care and oncology measurement recommended that policy makers “prioritize and develop effective cross-cutting measures that assess clinical and patient-reported outcomes, including shared decision making, care planning, and symptom control” and highlighted an “overreliance on condition-specific process measures.”1
Disease-specific quality measures have been developed, but there is a lack of consensus on what quality measures ought to be utilized across multiple cancers, especially when measuring clinical outcomes.2
As oncology APMs evolve, there are practical considerations in the design and implementation of outcome-based measures.
The development of new quality measures for oncology has been underway for many years. For example, CMS together with America’s Health Insurance Plans (AHIP) and others developed Core Quality Measures in 8 therapeutic areas, including medical oncology, to assess provider performance.3
This initiative included quality indicators focusing on breast cancer, colorectal cancer, prostate cancer, and, more generally, end-of-life care.4
It also identified areas for future measure development in oncology (eg, pain control, hospital admission, 5-year cure rates) and highlighted challenges related to data access and measurement as the standard of cancer care progresses, requiring frequent reassessments.4
In parallel, the American Society for Radiation Oncology has been working with the American Society of Clinical Oncology (ASCO) to develop measures “for utilization by both organizations in various quality programs and reporting environments.”5
However, no comprehensive set of core outcome-based quality measures in oncology has been published.
Research on healthcare quality measures typically differentiates between 2 key categories: process-based measures, which focus on proper reporting and procedure execution, and outcome-based measures, which involve clinical outcomes and patient-reported experience of care. This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are underutilized given their perceived benefits. We also present a set of “core outcome measures” that may be considered in future oncology APMs. Our study does not aim to provide a definitive list but, rather, to present a diverse set of outcome measures most commonly included in quality initiatives and payment models in oncology.
Our research consists of a focused literature review, content analysis, and measure categorization, similar to Macefield et al.6
First, our focused literature review summarizes key evidence related to quality measures in oncology, with an emphasis on classification, their unique advantages and disadvantages, and the challenges related to implementation in clinical practice. Second, our descriptive analysis of the most commonly used quality measures in oncology draws on a convenience sample of existing payment models and other quality assessment frameworks. Our sample includes 7 oncology quality assessment programs, frameworks, and payment models (also referenced as “oncology quality assessment frameworks”), which encompass 142 quality measures: the OCM by CMMI, the Quality Oncology Practice Initiative by ASCO, the Prospective Payment System–Exempt Cancer Hospital Quality Reporting Program by CMS, the Core Quality Measures Collaborative Core Sets by CMS and AHIP, the Oncology Medical Home program by the Community Oncology Alliance, the Osteoporosis Quality Improvement Registry by the National Osteoporosis Foundation and National Bone Health Alliance, and the Oncology Qualified Clinical Data Registry by the Oncology Nursing Society.
In this paper, we categorize these measures into process- versus outcome-based and analyze their frequency. Given our primary focus on outcome-based quality measures, we decided to expand our literature review to include the evidence base for each of the key outcome measure categories identified. These sources were identified by snowballing from quality measure summaries by CMS and reviewing other relevant literature. In addition, we review and summarize published reports on the impact of emerging oncology APMs on clinical outcomes and spending. We conclude with a synthesis of existing evidence on key outcome-based measures and their appropriateness in future oncology APMs. Finally, we discuss directions for customization and further validation of oncology core outcome measures.
Advantages of Process- and Outcome-Based Measures
Previous scholarship finds that both process- and outcome-based quality measures have advantages and disadvantages (for one such classification, see Table 17
For example, it is generally easier to generate actionable feedback based on process-based measures and there is mostly no or limited need for risk adjustment (unlike the case for quality measures such as mortality, for which complex case mix, indication, and disease stage adjustments are often required).7
In addition, data collection for process measures is generally faster, can draw on smaller sample sizes, and does not require advanced statistical analysis to yield practical results.7
On the other hand, outcome measures are generally based on clinical end points with proven significance in the quality of care. They are more understandable by patients and nonclinicians and are easier to define comprehensively (eg, hospice admissions for at least 3 days prior to death).7
Relatedly, an improvement in process measures may be a useful step in care coordination but may not always have an observable effect on improvement in clinical outcomes, especially when included for billing purposes only.11
Given these realities, the Agency for Healthcare Research and Quality regards outcome-based measures as the “gold standard” in quality measurement.12
Expert groups such as the Healthcare Association of New York State suggest that “regulators and payers should focus on overall performance (outcome measures), and defer the operations and use of process measures for internal quality improvement by healthcare providers.”13
Process-based measures dominate the OCM and other oncology quality assessment frameworks, yet outcome-based measures have an important role to play. Outcome-based measures are directly connected to real-world outcomes, ranging from hospital admissions to mortality and patient-reported outcomes (PROs), reflecting what patients and providers care about most.
Outcome Measures in Existing Oncology Quality Frameworks
Of the 142 quality measures from 7 oncology quality assessment frameworks that we reviewed, 80.3% (n = 114) were process-based measures and 19.7% (n = 28) were outcome-based measures. An earlier analysis of the National Quality Measures Clearinghouse found an even lower proportion of outcome-based measures (7.1%) based on a total of 1958 quality indicators from a wide range of therapeutic areas.11
Of those nearly 2000 indicators, only 1.6% were patient-reported outcome measures (PROMs).11
We condensed the 28 outcome-based measures into 23 unique
outcome measures by merging identical or near-identical measures and grouping them into 5 categories: (1) admissions and hospital visits (including emergency department [ED] visits), (2) hospice care, (3) mortality, (4) PROs, and (5) adverse events (AEs) (Table 2
Admissions and hospital visits.
Admissions and hospital visits, after risk adjustment, are important indicators of the appropriateness and timeliness of care. Up to 50% of ED visits are related to complications from chemotherapy, which can potentially indicate suboptimal management of the disease and care coordination (ranging from information sharing among providers to education about end-of-life care).14
The significant variation observed in admission rates and hospital/ED visits between different providers, even when controlling for other factors, has spurred research related to avoidable hospitalizations and appropriateness of care, especially in late-stage cancer care.15
In 2016, for example, CMS announced the inclusion of inpatient admissions and ED visits for patients receiving outpatient chemotherapy in its Hospital Outpatient Quality Reporting Program.16
Including hospital visits in payment models aims to “encourage reporting facilities to take steps to prevent and improve management of side effects and complications from treatment.”17
Although it offers patients, caregivers, and the healthcare system advantages relative to hospital settings, hospice care is generally underutilized. Quality measures related to hospice care may improve the quality of life of patients with late-stage cancer, reduce spending, and reduce burden among providers and caretakers.
Allowing patients with cancer to receive palliative care in a hospice setting is traditionally associated with improvements in quality of life, as well as system efficiencies. Yet, up to 66% of patients with cancer are not enrolled in hospice in the last 30 days of life, and less than 29% are enrolled for at least 2 months (considered appropriate care), based on an international review of 78 studies published between 1998 and 2011.18
One factor explaining the suboptimal transition to palliative care may be doctors’ tendency to overestimate survival prospects of a patient.19
Hospice-related measures aim to reduce wasteful spending on care that is unlikely to improve clinical outcomes and to provide patients with a higher quality of life. Specific provisions may be needed for palliative chemotherapy and other treatments that can extend the length of life while in palliative care.
Although outcome-based measures in this domain tend to focus on hospice settings alone, recent discussions suggest that palliative care may improve patient quality of life if initiated earlier in the cancer treatment course.20,21
In 2012, the National Quality Forum endorsed 14 measures related to end-of-life care, of which several are outcome-based, including “comfortable dying” and bereaved family survey measures.22
Finally, patient- and caregiver-reported outcomes may have a more prominent role to play, as end-of-life care should reflect patient and caregiver preferences.23
Mortality is a common outcome indicator used in both clinical practice and clinical trials, and it may be reported in different ways (eg, patient mortality over a specific period, overall survival, progression-free survival, by the setting of a patient’s death). Additionally, the setting of death (in a hospital vs at home or in a hospice) may play an important role in patients’ quality of life18,24
and is sometimes used as part of mortality-related quality measures.
Ranging from pain to social function evaluation, PROs are increasingly used to evaluate appropriateness of care given their ability to reflect patient needs and preferences, which may vary significantly. The FDA issued a PRO-specific guidance in 2009,25
defining PROs as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else,” and stating that in general, “findings measured by a well-defined and reliable PRO instrument in appropriately designed investigations can be used to support a claim in medical product labeling if the claim is consistent with the instrument’s documented measurement capability.”25
PROs are relevant in both early and late disease stages, complementing other measures that address outcomes in a hospital setting. In 2016, a multistakeholder roundtable on improving oncology measurement recommended that PROs be collected “before, during, and after treatment.”26
Although more evidence is needed, PROMs have been studied for use during an initial consultation and during shared decision making regarding patient care, aside from tracking treatment progress and patient satisfaction.27
A 2013 report on PROs by the National Quality Forum indicates that several guiding principles for selecting PROMs should be followed: psychometric soundness, person-centricity, meaningfulness, amenability to change, and implementability.28
Progress in drawing on electronic PROs may catalyze the ability of plans to systematically and reliably collect patient- and caregiver-reported outcomes, especially if they impose minimal burden on staff and patients.29
Nonetheless, the use of PROs has been associated with multiple challenges, including representativeness, inclusion of PROs in medication labels, necessity for both standardized and customizable PROs, and operational and organizational barriers to collecting and analyzing them.30
In addition, PRO collection is often resource-intensive, the validity of disease-specific PROs may be limited, and many PROs lack predictive value.30
Despite these challenges, PROs can be useful tools to obtain insight into patient needs and preferences in order to make better patient-level, as well as policy, decisions and to support further research and development.
Quality measures based on reporting of AEs aim to lower the number of avoidable incidents, potentially shortening hospitalization length and reducing costs as well as mortality. Initial guidance on the reporting of AEs in oncology trials was published by the Consolidated Standards of Reporting Trials in 2003, and AEs related to oncology care are now understood better.31
However, AEs in clinical settings are thought to be significantly underreported, partly driven by voluntary reporting and the use of instruments that may be prone to lower sensitivity.32
Improvements in documentation and electronic reporting are expected to improve the reliability of data about AEs observed in clinical practice (most hospitals do not use electronic health records [EHRs] to “directly measure [or record] patient harm”).33
If implemented broadly, the magnitude of potential improvements may be significant: Studies have shown that AEs can extend the length of hospitalizations, increase costs of care, and increase mortality up to 2-fold.34
Evidence From Early Quality Initiatives in Oncology
Although no comprehensive evaluation of OCM has taken place, limited evidence suggests that previous quality initiatives with outcome-based components have improved care while reducing costs. For instance, during a 2-year pilot in Texas involving 221 oncology patients (Innovent Oncology program by McKesson Specialty Health, Texas Oncology, and Aetna), savings of more than $500,000
The program has also been shown to improve adherence to clinical pathways and clinical outcomes: Pathway adherence has increased from 63% to 76%; reductions in ED visits, hospital admissions, and hospital days of 48%, 34%, and 44%, respectively, were observed; and average inpatient days decreased from 2.1 to 1.2 days.32,33
Innovent Oncology based its value-based reimbursement on 3 pillars: (1) Level I Pathways Program (aiming to increase the use of evidence-based treatment guidelines), (2) clinical benchmarking (based on a number of quality indicators), and (3) contract negotiation services.37
Among the quality measures included have been gastrointestinal toxicities, infection, thromboembolic events, pain, and depression.23
Similarly, an oncology pilot by UnitedHealthcare that drew on episode payments for more than 800 patients with breast, colon, and lung cancer in 5 oncology practices achieved net savings of more than $33 million (a 34% reduction of the predicted total medical cost).38
Some of the key quality measures used by this pilot included ED and hospitalization rates, admissions for cancer symptoms, febrile neutropenia occurrence rate, admissions for treatment-related symptoms, days from last chemotherapy to death, and hospice days for patients who died.38
However, a lack of a counterfactual (via a matched control group, for example) undermines a direct causal link between quality measurement and observed outcomes in these pilots, and more comprehensive evaluations are still needed.
Recommendations for Outcome-Based Measures in Oncology
A synthesis and recommendations for future core outcome sets in oncology are available in Table 3
. Measures that are generally seen as being closely tied to the quality of care received by oncology patients were classified into 5 quality domains identified by a CMMI APM toolkit39
: clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention. When possible, this set of outcome measures should be tailored to unique patient populations, diseases, providers, or other factors in individual payment models. In addition, some measures, such as hospice care—albeit appropriate for patients with more advanced disease—may not be relevant for patients with curable, early-stage cancer. Future oncology APMs should implement outcome measures relevant to the disease type and stage(s). For a detailed justification and discussion of individual categories and measures, please see eAppendix B
Collecting outcomes data in all 5 domains of cancer care is fraught with challenges that have been documented in multiple studies. For example, to measure and track outcomes properly, programs often require big data that involve multiple sources, such as EHRs, health insurance claims, and patient/caregiver surveys; however, whether data are complete and accessible and can be translated into clinical practice remains a challenging issue.40
Many outcome-based measures rely on administrative claims data, which tend to have a long report lag. Some outcomes data, such as hospice care, may be challenging to access, especially when the patient is transferred from one payer to another. Chung and Basch41
discuss specific challenges related to collecting and using patient-generated health data (including PROs), ranging from “provider concerns, workflow issues, standardization of patient-generated health data and interoperability of devices/sensors, security and privacy issues” to a “lack of the necessary EHR functionalities and software innovations.” Additionally, statistical challenges related to missing values, highly dimensional data sets, and confounding (bias) require robust statistical approaches that are not yet available in broad clinical practice.42
Nonetheless, new approaches are being tested as outcome measures gain support from clinicians, patients, and payers, including a collaborative pilot on establishing a framework to evaluate real-world end points in advanced non–small cell lung cancer led by the Friends of Cancer Research and supported by both public and private stakeholders.43
As highlighted in this paper, both OCM and other quality initiatives in oncology rely on process- or outcome-based quality measures to determine the quality of care and—in some cases—the level of payment. Given evidence from the literature and an analysis of 7 oncology quality assessment frameworks, we presented a set of outcome-based measures for consideration in future payment models in oncology. Although some measures may be omitted in specific cases, we believe the inclusion of measures related to all 5 domains—clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention—is highly desirable in future oncology APMs. Selective measurement of 1 outcome domain may create perverse incentives for providers to improve performance by underutilizing appropriate care and jeopardize optimal patient outcomes. Where appropriate, indication-specific quality measures should be included to account for quality-of-care complexities associated with individual cancer types and disease stages.
Overcoming hurdles to broader utilization of outcome-based measures in oncology will require a consensus between both payers and providers. These efforts should highlight the benefits of implementing outcome-based measures in oncology APMs (especially relative to the cost of implementation) and solutions to data and evaluation challenges (including risk adjustment and bias control). Future research is also needed to develop best practices for the inclusion and implementation of outcome measures in oncology clinical pathways.44
Additional considerations include developing strategies for quality control, dispute resolution, and administrative burden on providers and payers.
Given the steadily increasing costs of oncology care and, in some cases, the availability of multiple high-cost treatment options for patients with cancer, oncology care is in need of a more rigorous approach to quality assessment. The success of emerging oncology APMs will depend on a robust set of quality indicators that are relevant to patients, providers, and payers alike.
The authors thank Jacqueline Vanderpuye-Orgle, Jeffrey Lemay, Zachary Wessler, and Harshali Patel for helpful comments on an earlier draft of this paper. The authors retained full control over research design, analysis, and findings presented herein.