Peter J. Neumann, ScD

Even unprecedented crises can have silver linings. The coronavirus disease 2019 (COVID-19) pandemic is responsible for a dramatic reduction in clinician visits and medical procedures. This decline encompasses clinically indicated care but also includes services that deliver little or no clinical benefit. As we establish a “new normal” level of care post COVID-19, the medical community should leverage a rare opportunity to enhance the efficiency of the delivery system. Going forward, the key will be to invest in services that improve individual and population health while deterring a resurgence of low-value care (

A given clinical service is never by itself high or low value. Instead, its value depends on who receives it, when in the progression of a patient’s disease it is provided, and where the service is delivered. Consider cancer screening: Differentiating high- versus low-value care depends on various factors, including a patient’s age and risk profile, as well as screening frequency and site of care delivery. Among individuals at low risk or with a short life expectancy, cancer screening may often be deemed low value or even harmful because the risk of the procedure and follow-up care outweighs a small clinical benefit. Specific examples include prostate-specific antigen (PSA) screening for prostate cancer among men 70 years and older, cervical cancer screening among average-risk women younger than 21 years and older than 65 years, and colorectal cancer screening among nearly all adults younger than 40 years and older than 85 years.

Despite guideline recommendations, low-value care remains prevalent. For example, our analysis of commercial insurance claims from more than 750,000 Medicare Advantage enrollees shows that 4 in 10 men older than 70 years received at least 1 PSA screening test in 2019. Overall, investigators have estimated that the country spends $100 billion annually on low-value care.

How the health care system will respond to the ongoing pandemic and its aftermath remains unclear. During March 2020, the frequency of all routine cancer screenings (and virtually all nonemergent care unrelated to COVID-19) nose-dived, regardless of clinical benefit.

 What is certain is that a large and rapid infusion of funds is necessary to support substantial unmet clinical demand and to restore the financial footing of clinicians and hospitals. A multipronged strategy that aligns provider-facing incentives (eg, payment) and consumer-facing incentives (eg, benefit design) is warranted to ensure that a greater proportion of health care spending is dedicated to high-value care that improves health and a lower proportion to low-value care that produces no or little clinical benefit.

First, we need to build on alternative payment models that base reimbursement on patient-centered outcomes. The most impactful solution would increase reimbursement for high-value clinical services and reduce or cease payment for known low-value care. For example, a premium should be applied to payments that target preventive services used in clinical scenarios with a US Preventive Services Task Force A or B grade (eg, colorectal cancer screening for those aged 50-75 years; 

 risk assessment and genetic counseling/testing for those with a family history). Likewise, minimal/no reimbursement should be provided for services with a D grade (eg, colorectal cancer screening for those aged >85 years; 

 risk assessment and genetic counseling/testing for those without a family history). Section 4105 of the Affordable Care Act provides the secretary of HHS authority to “provide no payment for a preventive service that has not received a Grade of A, B, C, or I by such Task Force.” Given the pent-up demand for high-value care, such clinically driven incentives would simultaneously motivate providers to optimize patient needs and address financial apprehensions.

Second, we should leverage the widespread adoption of electronic health records (EHRs) to make it easier to order high-value care with simplified processes and discourage the use of low-value care with alerts. A recent analysis of multiple EHR systems highlighted the lack of association between ease of ordering (ie, number of clicks per service) and the evidence-based value of clinical services.

Third, health plans need to align patient cost sharing with the value of the underlying services. Current “blunt” instruments, such as plan deductibles, do not distinguish between high- and low-value care. A robust evidence base demonstrates that patient cost sharing indiscriminately decreases the use of both clinically indicated and unnecessary services. As alternative approaches, high-value medications that have demonstrated clinical benefit can be moved to “predeductible” status, in which patients face little or no cost sharing in a high-deductible health plan. Examples include insulin and other glucose-lowering medications for diabetes, inhaled corticosteroids for asthma, and tumor necrosis factor inhibitor therapy for autoimmune diseases. Similarly, for commonly overused low-value services (eg, the 3 low-value cancer screening scenarios mentioned previously), health plans can selectively increase the cost-sharing level as a lever to direct patient behavior. Also, a novel benefit design template, referred to as VBID-X, increases access to high-value care and targets low-value care without raising premiums or deductibles. In May 2020, the HHS 2021 Notice of Benefit and Payment Parameters final rule strongly recommended that federally qualified health plans adopt this approach.

All stakeholders agree that there is more than enough money in the US health system, but misaligned incentives have driven spending for the wrong services in the wrong places on the wrong patients. The COVID-19 pandemic can become a catalyst to align incentives to invest more resources in high-value care and less in those that do not improve clinical outcomes. Without these changes, decreased revenues and furloughed employees will push providers for a rapid return to archaic volume-driven patterns to restore financial solvency. Some may aggressively increase the utilization of services, regardless of their value, to compensate for their financial loss.

Albert Einstein famously said, “In the midst of every crisis, lies great opportunity.” As we establish the new normal after COVID-19, the medical community can propel a more efficient, patient-centered health system without increasing total expenditures. The resurgence should be clinically driven, but we also need to make providers “whole” for doing what is right from a patient-centered perspective. Specifically, incentives to encourage the delivery of evidence-based care, not services that produce the most revenue, must be implemented. As we reopen practices, ambulatory centers, and hospitals, providers should prioritize high-value services, and public and private payers must reimburse generously for these services while creating benefit designs that make them accessible and affordable. The added costs incurred from the increased use of high-value care can be paid for by reductions in the billions spent annually on the care that does not improve the health of Americans.

The authors are grateful for financial support from Arnold Ventures (formerly the Laura and John Arnold Foundation) and statistical analysis from Benjamin Koethe (Tufts Medical Center). Dr Kim is an OptumLabs Visiting Fellow.

 Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center (DDK, DAO, PJN), Boston, MA; Department of Medicine, Tufts University School of Medicine (DDK, DAO, PJN), Boston, MA; Department of Internal Medicine, University of Michigan (AMF), Ann Arbor, MI.

 Arnold Ventures (formerly the Laura and John Arnold Foundation).

 Drs Kim, Ollendorf, and Neumann have received a research grant from Arnold Ventures. Dr Fendrick has been a consultant for AbbVie, Amgen, Centivo, Community Oncology Alliance, Covered California, EmblemHealth, Exact Sciences, Freedman Health, GRAIL, Harvard University, Health & Wellness Innovations, Health at Scale Technologies, MedZed, Merck, Montana Health Cooperative, Penguin Pay, Risalto, Sempre Health, State of Minnesota, US Department of Defense, Virginia Center for Health Innovation, Wellth, Yale—New Haven Health System, and Zansors; has performed research for the Agency for Healthcare Research and Quality, Arnold Ventures, Boehringer Ingelheim, Gary and Mary West Health Policy Center, National Pharmaceutical Council, Patient-Centered Outcomes Research Institute, PhRMA, Robert Wood Johnson Foundation, and State of Michigan/CMS; and holds outside positions as co-editor-in-chief of 

, member of the Medicare Evidence Development & Coverage Advisory Committee, and partner in V-BID Health, LLC.

 Concept and design (DDK, DAO, AMF); acquisition of data (DDK); analysis and interpretation of data (DDK, DAO); drafting of the manuscript (DDK, PJN, AMF); critical revision of the manuscript for important intellectual content (DDK, DAO, AMF); statistical analysis (DDK); obtaining funding (DDK); administrative, technical, or logistic support (DDK, PJN); and supervision (DAO, PJN).

 David D. Kim, PhD, Tufts Medical Center, 800 Washington St, Box 063, Boston, MA 02111. Email: DKim3@tuftsmedicalcenter.org.

1. Shrank WH, Rogstad TL, Parekh N. Waste in the US health care system: estimated costs and potential for savings. 

. 2019;322(15):1501-1509. doi:10.1001/jama.2019.13978

2. Robbins R. Routine cancer screenings have plummeted during the pandemic, medical records data show. STAT. May 4, 2020. Accessed June 3, 2020. https://www.statnews.com/2020/05/04/cancer-screenings-drop-coronavirus-pandemic-epic

3. Mehrotra A, Chernew M, Linetsky D, Hatch H, Cutler D. The impact of the COVID-19 pandemic on outpatient visits: a rebound emerges. The Commonwealth Fund. May 19, 2020. Accessed June 3, 2020. https://www.commonwealthfund.org/publications/2020/apr/impact-covid-19-outpatient-visits

4. Colla CH. Swimming against the current—what might work to reduce low-value care? 

. 2014;371(14):1280-1283. doi:10.1056/NEJMp1404503

5. Budros M, Fendrick AM. Levers to reduce use of unnecessary services: creating needed headroom to enhance spending on evidence-based care. 

6. Schwartz E, Ruff A, Kinning M, Fendrick AM. Ease of ordering high- and low-value services in various electronic health records. 

7. CMS announces final payment notice for 2021 coverage year. News release. CMS. May 7, 2020. Accessed June 3, 2020. https://www.cms.gov/newsroom/press-releases/cms-announces-final-payment-notice-2021-coverage-year

Articles

With a “new normal” level of care going forward post coronavirus disease 2019 (COVID-19), the key will be to invest in high-value services while deterring a resurgence of low-value care.

Crisis Into Opportunity: Can COVID-19 Help Set a Path to Improved Health Care Efficiency?

 This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are often underutilized. We also present a set of “core outcome measures” that may be considered in future oncology alternative payment models (APMs).

 Our research consists of a focused literature review, content analysis, and quality measure synthesis and categorization.

 We conducted a focused literature review to generate key evidence on quality measures in oncology. We studied 7 oncology quality assessment frameworks, encompassing 142 quality metrics, and synthesized recommendations using the Center for Medicare and Medicaid Innovation APM toolkit, focusing on outcome measures.

 We present 34 outcome-based oncology quality measures for consideration, which are classified into 5 domains: clinical care (eg, hospital and emergency department visits, treatment effectiveness, mortality), safety (eg, infections, hospital adverse events), care coordination (for hospital and hospice care), patient and caregiver experience, and population health and prevention. Both general and indication-specific outcome measures should be considered in oncology APMs, as appropriate. Utilizing outcome-based measures will require addressing multiple challenges, ranging from risk adjustment to data quality assurance.

 Oncology care will benefit from a more rigorous approach to quality assessment. The success of oncology APMs will require a robust set of quality measures that are relevant to patients, providers, and payers.

Evidence is scarce on what outcome measures are most suitable and feasible for future oncology payment models. This paper reviews 7 oncology quality assessment frameworks and existing literature on quality measurement in oncology and recommends that:

Outcome quality measures in future oncology payment models should include domains for clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention.

Future oncology payment models should draw on the core outcome set but should be tailored to unique care delivery needs and context.

The inclusion of diverse outcome measures in oncology may help improve care delivery to all patients with cancer.

Several alternative payment models (APMs) are being piloted to address affordability, equity, and quality-of-care challenges in oncology care. Of these arrangements, the Oncology Care Model (OCM), developed by the Center for Medicare and Medicaid Innovation (CMMI), is among the most extensive, covering about 200,000 chemotherapy episodes annually. The OCM relies on multiple quality measures to determine the level of payment for each provider, with the goal of incentivizing higher-quality care in a cost-effective manner. The OCM’s payment design is described in 

Despite innovations in the payment landscape, limited consensus exists about what constitutes indispensable quality measures in oncology. The absence of such consensus may not only limit the development of better payment models, which increasingly link payment to quality of care, but also result in a lack of agreement on how value should be defined (and demonstrated) in an era of innovative, ever more expensive cancer therapies. In 2016, a roundtable of national experts in cancer care and oncology measurement recommended that policy makers “prioritize and develop effective cross-cutting measures that assess clinical and patient-reported outcomes, including shared decision making, care planning, and symptom control” and highlighted an “overreliance on condition-specific process measures.”

 Disease-specific quality measures have been developed, but there is a lack of consensus on what quality measures ought to be utilized across multiple cancers, especially when measuring clinical outcomes.

 As oncology APMs evolve, there are practical considerations in the design and implementation of outcome-based measures.

The development of new quality measures for oncology has been underway for many years. For example, CMS together with America’s Health Insurance Plans (AHIP) and others developed Core Quality Measures in 8 therapeutic areas, including medical oncology, to assess provider performance.

 This initiative included quality indicators focusing on breast cancer, colorectal cancer, prostate cancer, and, more generally, end-of-life care.

 It also identified areas for future measure development in oncology (eg, pain control, hospital admission, 5-year cure rates) and highlighted challenges related to data access and measurement as the standard of cancer care progresses, requiring frequent reassessments.

 In parallel, the American Society for Radiation Oncology has been working with the American Society of Clinical Oncology (ASCO) to develop measures “for utilization by both organizations in various quality programs and reporting environments.”

 However, no comprehensive set of core outcome-based quality measures in oncology has been published.

Research on healthcare quality measures typically differentiates between 2 key categories: process-based measures, which focus on proper reporting and procedure execution, and outcome-based measures, which involve clinical outcomes and patient-reported experience of care. This paper aims to synthesize existing scholarship on quality measures in oncology, with a specific focus on outcome-based quality measures, which are underutilized given their perceived benefits. We also present a set of “core outcome measures” that may be considered in future oncology APMs. Our study does not aim to provide a definitive list but, rather, to present a diverse set of outcome measures most commonly included in quality initiatives and payment models in oncology.

Our research consists of a focused literature review, content analysis, and measure categorization, similar to Macefield et al.

 First, our focused literature review summarizes key evidence related to quality measures in oncology, with an emphasis on classification, their unique advantages and disadvantages, and the challenges related to implementation in clinical practice. Second, our descriptive analysis of the most commonly used quality measures in oncology draws on a convenience sample of existing payment models and other quality assessment frameworks. Our sample includes 7 oncology quality assessment programs, frameworks, and payment models (also referenced as “oncology quality assessment frameworks”), which encompass 142 quality measures: the OCM by CMMI, the Quality Oncology Practice Initiative by ASCO, the Prospective Payment System—Exempt Cancer Hospital Quality Reporting Program by CMS, the Core Quality Measures Collaborative Core Sets by CMS and AHIP, the Oncology Medical Home program by the Community Oncology Alliance, the Osteoporosis Quality Improvement Registry by the National Osteoporosis Foundation and National Bone Health Alliance, and the Oncology Qualified Clinical Data Registry by the Oncology Nursing Society.

In this paper, we categorize these measures into process- versus outcome-based and analyze their frequency. Given our primary focus on outcome-based quality measures, we decided to expand our literature review to include the evidence base for each of the key outcome measure categories identified. These sources were identified by snowballing from quality measure summaries by CMS and reviewing other relevant literature. In addition, we review and summarize published reports on the impact of emerging oncology APMs on clinical outcomes and spending. We conclude with a synthesis of existing evidence on key outcome-based measures and their appropriateness in future oncology APMs. Finally, we discuss directions for customization and further validation of oncology core outcome measures.

Advantages of Process- and Outcome-Based Measures

Previous scholarship finds that both process- and outcome-based quality measures have advantages and disadvantages (for one such classification, see 

 For example, it is generally easier to generate actionable feedback based on process-based measures and there is mostly no or limited need for risk adjustment (unlike the case for quality measures such as mortality, for which complex case mix, indication, and disease stage adjustments are often required).

 In addition, data collection for process measures is generally faster, can draw on smaller sample sizes, and does not require advanced statistical analysis to yield practical results.

On the other hand, outcome measures are generally based on clinical end points with proven significance in the quality of care. They are more understandable by patients and nonclinicians and are easier to define comprehensively (eg, hospice admissions for at least 3 days prior to death).

 Relatedly, an improvement in process measures may be a useful step in care coordination but may not always have an observable effect on improvement in clinical outcomes, especially when included for billing purposes only.

 Given these realities, the Agency for Healthcare Research and Quality regards outcome-based measures as the “gold standard” in quality measurement.

 Expert groups such as the Healthcare Association of New York State suggest that “regulators and payers should focus on overall performance (outcome measures), and defer the operations and use of process measures for internal quality improvement by healthcare providers.”

Process-based measures dominate the OCM and other oncology quality assessment frameworks, yet outcome-based measures have an important role to play. Outcome-based measures are directly connected to real-world outcomes, ranging from hospital admissions to mortality and patient-reported outcomes (PROs), reflecting what patients and providers care about most.

Outcome Measures in Existing Oncology Quality Frameworks

Of the 142 quality measures from 7 oncology quality assessment frameworks that we reviewed, 80.3% (n = 114) were process-based measures and 19.7% (n = 28) were outcome-based measures. An earlier analysis of the National Quality Measures Clearinghouse found an even lower proportion of outcome-based measures (7.1%) based on a total of 1958 quality indicators from a wide range of therapeutic areas.

 Of those nearly 2000 indicators, only 1.6% were patient-reported outcome measures (PROMs).

We condensed the 28 outcome-based measures into 23 unique

outcome measures by merging identical or near-identical measures and grouping them into 5 categories: (1) admissions and hospital visits (including emergency department [ED] visits), (2) hospice care, (3) mortality, (4) PROs, and (5) adverse events (AEs) (

 Admissions and hospital visits, after risk adjustment, are important indicators of the appropriateness and timeliness of care. Up to 50% of ED visits are related to complications from chemotherapy, which can potentially indicate suboptimal management of the disease and care coordination (ranging from information sharing among providers to education about end-of-life care).

The significant variation observed in admission rates and hospital/ED visits between different providers, even when controlling for other factors, has spurred research related to avoidable hospitalizations and appropriateness of care, especially in late-stage cancer care.

 In 2016, for example, CMS announced the inclusion of inpatient admissions and ED visits for patients receiving outpatient chemotherapy in its Hospital Outpatient Quality Reporting Program.

 Including hospital visits in payment models aims to “encourage reporting facilities to take steps to prevent and improve management of side effects and complications from treatment.”

 Although it offers patients, caregivers, and the healthcare system advantages relative to hospital settings, hospice care is generally underutilized. Quality measures related to hospice care may improve the quality of life of patients with late-stage cancer, reduce spending, and reduce burden among providers and caretakers.

Allowing patients with cancer to receive palliative care in a hospice setting is traditionally associated with improvements in quality of life, as well as system efficiencies. Yet, up to 66% of patients with cancer are not enrolled in hospice in the last 30 days of life, and less than 29% are enrolled for at least 2 months (considered appropriate care), based on an international review of 78 studies published between 1998 and 2011.

 One factor explaining the suboptimal transition to palliative care may be doctors’ tendency to overestimate survival prospects of a patient.

 Hospice-related measures aim to reduce wasteful spending on care that is unlikely to improve clinical outcomes and to provide patients with a higher quality of life. Specific provisions may be needed for palliative chemotherapy and other treatments that can extend the length of life while in palliative care.

Although outcome-based measures in this domain tend to focus on hospice settings alone, recent discussions suggest that palliative care may improve patient quality of life if initiated earlier in the cancer treatment course.

 In 2012, the National Quality Forum endorsed 14 measures related to end-of-life care, of which several are outcome-based, including “comfortable dying” and bereaved family survey measures.

 Finally, patient- and caregiver-reported outcomes may have a more prominent role to play, as end-of-life care should reflect patient and caregiver preferences.

 Mortality is a common outcome indicator used in both clinical practice and clinical trials, and it may be reported in different ways (eg, patient mortality over a specific period, overall survival, progression-free survival, by the setting of a patient’s death). Additionally, the setting of death (in a hospital vs at home or in a hospice) may play an important role in patients’ quality of life

 and is sometimes used as part of mortality-related quality measures.

 Ranging from pain to social function evaluation, PROs are increasingly used to evaluate appropriateness of care given their ability to reflect patient needs and preferences, which may vary significantly. The FDA issued a PRO-specific guidance in 2009,

 defining PROs as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else,” and stating that in general, “findings measured by a well-defined and reliable PRO instrument in appropriately designed investigations can be used to support a claim in medical product labeling if the claim is consistent with the instrument’s documented measurement capability.”

PROs are relevant in both early and late disease stages, complementing other measures that address outcomes in a hospital setting. In 2016, a multistakeholder roundtable on improving oncology measurement recommended that PROs be collected “before, during, and after treatment.”

 Although more evidence is needed, PROMs have been studied for use during an initial consultation and during shared decision making regarding patient care, aside from tracking treatment progress and patient satisfaction.

 A 2013 report on PROs by the National Quality Forum indicates that several guiding principles for selecting PROMs should be followed: psychometric soundness, person-centricity, meaningfulness, amenability to change, and implementability.

 Progress in drawing on electronic PROs may catalyze the ability of plans to systematically and reliably collect patient- and caregiver-reported outcomes, especially if they impose minimal burden on staff and patients.

Nonetheless, the use of PROs has been associated with multiple challenges, including representativeness, inclusion of PROs in medication labels, necessity for both standardized and customizable PROs, and operational and organizational barriers to collecting and analyzing them.

 In addition, PRO collection is often resource-intensive, the validity of disease-specific PROs may be limited, and many PROs lack predictive value.

 Despite these challenges, PROs can be useful tools to obtain insight into patient needs and preferences in order to make better patient-level, as well as policy, decisions and to support further research and development.

 Quality measures based on reporting of AEs aim to lower the number of avoidable incidents, potentially shortening hospitalization length and reducing costs as well as mortality. Initial guidance on the reporting of AEs in oncology trials was published by the Consolidated Standards of Reporting Trials in 2003, and AEs related to oncology care are now understood better.

 However, AEs in clinical settings are thought to be significantly underreported, partly driven by voluntary reporting and the use of instruments that may be prone to lower sensitivity.

 Improvements in documentation and electronic reporting are expected to improve the reliability of data about AEs observed in clinical practice (most hospitals do not use electronic health records [EHRs] to “directly measure [or record] patient harm”).

If implemented broadly, the magnitude of potential improvements may be significant: Studies have shown that AEs can extend the length of hospitalizations, increase costs of care, and increase mortality up to 2-fold.

Evidence From Early Quality Initiatives in Oncology

Although no comprehensive evaluation of OCM has taken place, limited evidence suggests that previous quality initiatives with outcome-based components have improved care while reducing costs. For instance, during a 2-year pilot in Texas involving 221 oncology patients (Innovent Oncology program by McKesson Specialty Health, Texas Oncology, and Aetna), savings of more than $500,000

 The program has also been shown to improve adherence to clinical pathways and clinical outcomes: Pathway adherence has increased from 63% to 76%; reductions in ED visits, hospital admissions, and hospital days of 48%, 34%, and 44%, respectively, were observed; and average inpatient days decreased from 2.1 to 1.2 days.

 Innovent Oncology based its value-based reimbursement on 3 pillars: (1) Level I Pathways Program (aiming to increase the use of evidence-based treatment guidelines), (2) clinical benchmarking (based on a number of quality indicators), and (3) contract negotiation services.

 Among the quality measures included have been gastrointestinal toxicities, infection, thromboembolic events, pain, and depression.

Similarly, an oncology pilot by UnitedHealthcare that drew on episode payments for more than 800 patients with breast, colon, and lung cancer in 5 oncology practices achieved net savings of more than $33 million (a 34% reduction of the predicted total medical cost).

 Some of the key quality measures used by this pilot included ED and hospitalization rates, admissions for cancer symptoms, febrile neutropenia occurrence rate, admissions for treatment-related symptoms, days from last chemotherapy to death, and hospice days for patients who died.

However, a lack of a counterfactual (via a matched control group, for example) undermines a direct causal link between quality measurement and observed outcomes in these pilots, and more comprehensive evaluations are still needed.

Recommendations for Outcome-Based Measures in Oncology

A synthesis and recommendations for future core outcome sets in oncology are available in 

. Measures that are generally seen as being closely tied to the quality of care received by oncology patients were classified into 5 quality domains identified by a CMMI APM toolkit

: clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention. When possible, this set of outcome measures should be tailored to unique patient populations, diseases, providers, or other factors in individual payment models. In addition, some measures, such as hospice care—albeit appropriate for patients with more advanced disease&mdash;may not be relevant for patients with curable, early-stage cancer. Future oncology APMs should implement outcome measures relevant to the disease type and stage(s). For a detailed justification and discussion of individual categories and measures, please see 

Collecting outcomes data in all 5 domains of cancer care is fraught with challenges that have been documented in multiple studies. For example, to measure and track outcomes properly, programs often require big data that involve multiple sources, such as EHRs, health insurance claims, and patient/caregiver surveys; however, whether data are complete and accessible and can be translated into clinical practice remains a challenging issue.

 Many outcome-based measures rely on administrative claims data, which tend to have a long report lag. Some outcomes data, such as hospice care, may be challenging to access, especially when the patient is transferred from one payer to another. Chung and Basch

 discuss specific challenges related to collecting and using patient-generated health data (including PROs), ranging from “provider concerns, workflow issues, standardization of patient-generated health data and interoperability of devices/sensors, security and privacy issues” to a “lack of the necessary EHR functionalities and software innovations.” Additionally, statistical challenges related to missing values, highly dimensional data sets, and confounding (bias) require robust statistical approaches that are not yet available in broad clinical practice.

 Nonetheless, new approaches are being tested as outcome measures gain support from clinicians, patients, and payers, including a collaborative pilot on establishing a framework to evaluate real-world end points in advanced non—small cell lung cancer led by the Friends of Cancer Research and supported by both public and private stakeholders.

As highlighted in this paper, both OCM and other quality initiatives in oncology rely on process- or outcome-based quality measures to determine the quality of care and—in some cases&mdash;the level of payment. Given evidence from the literature and an analysis of 7 oncology quality assessment frameworks, we presented a set of outcome-based measures for consideration in future payment models in oncology. Although some measures may be omitted in specific cases, we believe the inclusion of measures related to all 5 domains&mdash;clinical care, safety, care coordination, patient and caregiver experience, and population health and prevention&mdash;is highly desirable in future oncology APMs. Selective measurement of 1 outcome domain may create perverse incentives for providers to improve performance by underutilizing appropriate care and jeopardize optimal patient outcomes. Where appropriate, indication-specific quality measures should be included to account for quality-of-care complexities associated with individual cancer types and disease stages.

Overcoming hurdles to broader utilization of outcome-based measures in oncology will require a consensus between both payers and providers. These efforts should highlight the benefits of implementing outcome-based measures in oncology APMs (especially relative to the cost of implementation) and solutions to data and evaluation challenges (including risk adjustment and bias control). Future research is also needed to develop best practices for the inclusion and implementation of outcome measures in oncology clinical pathways.

 Additional considerations include developing strategies for quality control, dispute resolution, and administrative burden on providers and payers.

Given the steadily increasing costs of oncology care and, in some cases, the availability of multiple high-cost treatment options for patients with cancer, oncology care is in need of a more rigorous approach to quality assessment. The success of emerging oncology APMs will depend on a robust set of quality indicators that are relevant to patients, providers, and payers alike.&ensp;

The authors thank Jacqueline Vanderpuye-Orgle, Jeffrey Lemay, Zachary Wessler, and Harshali Patel for helpful comments on an earlier draft of this paper. The authors retained full control over research design, analysis, and findings presented herein.

 University of Southern California Schaeffer Center for Health Policy and Economics (JPH), Los Angeles, CA; RAND Corporation (JPH), Los Angeles, CA; Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center (PJL, PJN), Boston, MA.

 This research was funded by a grant from Amgen Inc via a contract with Tufts Medical Center. Additional research support for Dr Hlávka was provided by the National Institute on Aging of the National Institutes of Health under Award Number R01AG062277. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

This review presents a set of evidence-based outcome measures for oncology alternative payment models, drawing on evidence from existing and proposed quality measures.

Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations

 To compare coverage of orphan and nonorphan drugs, to examine variation in orphan drug coverage across the largest US private plans, and to evaluate factors influencing coverage decisions.

 We analyzed a data set of private health plan coverage decisions for specialty drugs (N = 5000) in 3 ways. First, we compared the frequency with which plans applied restrictions in their decisions for orphan and nonorphan drugs. Second, we examined variation in the frequency with which 17 of the largest 20 private plans applied coverage restrictions for orphan drugs. Third, we used multivariate regression to examine factors associated with restricted orphan drug coverage.

 Health plans are less likely to restrict orphan drugs compared with nonorphan drugs. Of orphan drug decisions (n = 2168), plans did not apply coverage restrictions in 70% of cases, applied restrictions in 29%, and did not cover in 1%. In contrast, of nonorphan drug decisions (n = 2832), plans did not apply coverage restrictions in 53% of cases, applied restrictions in 41%, and did not cover in 6%. The frequency of restrictions for orphan drugs varied from 11% to 65% across plans. The attributes of orphan drugs that were more likely to be associated with restrictions than others included drugs for noncancer diseases, drugs with alternatives, self-administered drugs, drugs indicated for diseases with a higher prevalence, and drugs with higher annual costs (all 

 Health plans restrict access to orphan drugs approximately one-third of the time, and restrictions vary considerably across plans. Plans more often add restrictions for orphan drugs that are indicated for diseases with a higher prevalence and that have higher annual costs.

Private US health plans less often apply restrictions in their coverage decisions for orphan drugs than in their coverage decisions for nonorphan drugs (30% vs 47% of the time, respectively).

Health plans vary in the frequency with which they apply restrictions in their orphan drug coverage decisions.

Private US health plans more often add orphan drug coverage restrictions (eg, patient subgroup restrictions, step therapy protocols) for drugs for diseases other than cancer, drugs with alternatives, self-administered drugs, drugs indicated for diseases with a higher prevalence, and drugs with higher annual costs.

Orphan drugs (ie, drugs indicated for rare diseases) pose a singular challenge for health plans. Although they target small patient populations, their high costs, rising numbers, and strong backing by patient groups force plans to confront ever-more-delicate decisions about how to provide access, even in the face of sometimes highly uncertain evidence.

 The challenge is intensifying: From 1997 to 2017, spending on orphan drugs grew from 4% to 10% of total prescription drug spending, with spending on orphan drugs totaling $43 billion in 2017.

 An estimated 25 million to 30 million Americans live with an orphan disease.

Eighty-seven percent of orphan drug spending is made up of specialty drugs, which are typically complex large-molecule drugs that require special administration, monitoring, and handling, although high-priced small-molecule drugs are also often considered specialty drugs.

 Specialty drugs are also used to treat a range of nonorphan conditions, including rheumatoid arthritis, hepatitis C, Crohn disease, and multiple sclerosis.

Research has demonstrated that health plans sometimes restrict patient access to orphan drugs. Results of one study found that private plans apply restrictions in roughly a quarter of orphan drug coverage decisions.

 Another study reported that Medicare and private plans often place orphan drugs on the highest co-payment tier of their formularies.

However, the circumstances under which plans restrict access to orphan drugs have not been reported. It is unclear, for example, whether plans are more likely to restrict orphan drugs with larger budget impacts. Knowledge of these circumstances would help patients, providers, and policy makers better understand the coverage decisions affecting access. We analyzed a database of private health plan coverage decisions to (1) compare coverage of orphan and nonorphan drugs, (2) examine variation in orphan drug coverage across the largest US private plans, and (3) evaluate factors influencing orphan drug coverage decision making.

We identified orphan drug coverage decisions using the Tufts Medical Center Specialty Drug Evidence and Coverage (SPEC) database. SPEC includes publicly available specialty drug coverage decisions issued by 17 of the 20 largest US private health plans. SPEC includes coverage decisions relevant to the included plans’ commercial lines of business. Of the 3 largest plans excluded from SPEC, 1 does not make its coverage decisions publicly available and 2 focus exclusively on Medicare or Medicaid populations.

SPEC includes information on health plans’ specialty drug coverage decisions and the supporting evidence that they cite. SPEC details drug-level information, including the number of years since a drug’s FDA approval and whether a drug is indicated for a pediatric population. Roughly half (51%) of drugs in SPEC are physician-administered and covered through the plans’ medical benefit. The remaining drugs are self-administered drugs, which plans cover through their pharmacy benefit.

We define a coverage restriction as a health plan decision to reduce the size of the eligible patient population compared with the population described in the FDA label. We further classify restrictions into categories: (1) patient subgroup (requirement for patients to meet certain clinical criteria, such as disease severity), (2) step therapy protocol (requirement for patients to first fail an alternative treatment before gaining access to a drug), (3) prescriber restriction (requirement for a certain type of physician—for example, a rheumatologist&mdash;to prescribe the drug), (4) combination therapy (requirement for a drug to be used concomitantly with another medication), and (5) other.

 SPEC includes 204 drugs. Because a drug can be indicated for multiple diseases, the database includes 409 unique drug—indication pairs (186 orphan drug–indication pairs; 223 nonorphan drug–indication pairs). For example, because ruxolitinib is indicated for myelofibrosis and polycythemia vera (both orphan diseases), it appears twice in the database. SPEC includes 5000 coverage decisions (2168 for orphan drugs; 2832 for nonorphan drugs) across the 17 health plans. Included decisions are current as of March 2018.

The dependent variable is dichotomous (yes/no) and describes whether the health plan’s coverage decision restricted patients’ access to a drug beyond the drug’s FDA label. “No restrictions” means a plan covered the drug consistent with its label. Coverage restrictions included decisions in which the plan did not cover the drug or the plan applied some restriction (for example, by step therapy protocol).

 We include 10 independent variables that we selected either because they have been previously shown to be associated with restricted drug coverage

 or because we consider them likely to be associated with coverage restrictions. 

 is a dichotomous variable capturing whether a drug is indicated to treat cancer. 

 is a dichotomous variable capturing whether a drug has any therapeutic alternatives (ie, whether the FDA has approved another drug for the same indication). 

 is a dichotomous variable accounting for whether a drug is indicated for pediatric or adolescent patients. Examples include nusinersen for spinal muscular atrophy and velaglucerase alfa for Gaucher disease type 1. 

 is a continuous variable accounting for the number of years since a drug’s FDA approval. When a drug is approved for multiple orphan indications, we consider the approval date for each indication separately. For example, tocilizumab was approved to treat polyarticular juvenile idiopathic arthritis in 2013 and approved for cytokine release syndrome in 2017; thus, the time since FDA approval differs for these indications. 

 is a dichotomous variable describing whether the drug has been flagged by the FDA for a safety concern (specifically, a black box safety warning or implementation of a Risk Evaluation and Mitigation Strategy program).

 is a dichotomous variable accounting for whether the FDA included the drug in one of its expedited approval programs, including priority review, fast track, accelerated approval, or breakthrough therapy.

 is a dichotomous variable capturing whether the drug is self- or physician-administered. We categorize tablets, capsules, and subcutaneously self-injected drugs as being self-administered, and drugs injected or infused in inpatient or outpatient settings as being physician-administered. 

 is a dichotomous variable accounting for situations in which an orphan drug is also indicated for a nonorphan indication. For instance, adalimumab is approved for orphan indications (eg, pediatric Crohn disease) and nonorphan indications (eg, rheumatoid arthritis). 

 is a categorical variable accounting for the prevalence of the indicated disorder. We obtained estimates of US disease prevalence from a review of the medical literature and the websites of national clinical organizations. To construct the variable, we divided observations into quartiles based on prevalence magnitudes. 

 is a categorical variable capturing the yearly per-patient drug cost (and dividing observations into quartiles). We estimated these costs using dosing information from the drug’s label and the Federal Supply Schedule (FSS) price (federal purchaser price based on a drug’s average sales price inclusive of rebates).

 If the FSS drug price was unavailable, we relied on alternative sources, including the CMS Average Sales Price (a product’s average sales price as submitted to CMS) and the IBM Micromedex RED BOOK (a proprietary source of drug pricing).

We analyzed the data in 3 ways. First, we compared the frequency with which plans applied coverage restrictions for nonorphan and orphan drugs. Second, we evaluated variation in orphan drug coverage across health plans. Third, we developed a multilevel, random-effect, logistic regression model to estimate the relationship between the independent variables and the likelihood of restrictive orphan drug coverage. We used a multilevel model to estimate the average effect of an independent variable on coverage across health plans while accounting for differences among them. We report associations between independent variables and coverage restrictiveness using odds ratios, which describe how much larger or smaller the odds of restricted coverage are for an independent variable (eg, drugs with alternatives compared with drugs with no alternatives). We considered 

 values <.05 to be statistically significant. We performed analyses using Stata SE version 13 (StataCorp LP; College Station, Texas).

Comparing Orphan and Nonorphan Drug Coverage

The vast majority of orphan drug coverage decisions (2146 of 2168 decisions; 99%) provide at least some coverage: 70% of decisions provide coverage without restrictions, 29% add restrictions, and 1% provide no coverage. By comparison, among nonorphan drug coverage decisions in SPEC (n = 2832), 53% provide coverage without restrictions, 41% add restrictions, and 6% provide no coverage (

In orphan drug restrictions, plans most commonly limit coverage by patient subgroup restrictions (47% of restricted decisions). In contrast, for nonorphan drug restrictions, plans most commonly apply step therapy protocols (77% of restricted decisions) (

The degree to which plans restricted orphan drug coverage decisions varied across health plans, ranging from 11% to 65% (

Factors Associated With Restricted Orphan Drug Coverage

Health plans are more likely to restrict coverage of orphan drugs indicated for noncancer diseases, for drugs having available alternatives, for self-administered drugs, and for more recently approved products. Plans are also more likely to restrict orphan drugs with higher annual costs and drugs indicated for higher-prevalence diseases (

Health plans cover orphan drugs more generously than nonorphan drugs: 70% versus 53% of coverage decisions do not include coverage restrictions, respectively. Still, the roughly one-third of orphan drug coverage decisions with restrictions are notable and have not been previously reported.

Plans most often restrict coverage of orphan drugs by applying patient subgroup restrictions. For example, one plan restricts coverage of omalizumab for chronic idiopathic urticaria by requiring that patients be symptomatic for at least 6 weeks before they are granted access to the treatment. In another example, a plan restricts coverage of rilonacept for cryopyrin-associated periodic syndromes by requiring that patients’ disease causes functional impairment that results in limitations of activities of daily living.

We found considerable variation in orphan drug coverage across plans. This finding has important implications and suggests that a patient’s insurance company can impact their access to orphan drugs. Reasons for this variation are unclear but may reflect differences in contracting arrangements or in the rebates that plans negotiate, as well as the fact that plans tailor decisions according to their particular patient populations, local practice patterns, and budgetary realities.

 Various factors are associated with the restrictiveness of orphan drug coverage. Health plans less often apply restrictions in their coverage decisions for orphan drugs indicated for cancer than for orphan drugs indicated for noncancer diseases, a finding that may indicate that health plans award cancer treatments a special status.

 Health plans more often restrict coverage for more recently approved orphan drugs, possibly because such drugs have a less mature evidence base and because plans have less experience managing them. Health plans tend to restrict coverage of self-administered more than physician-administered orphan drugs, suggesting that plans manage pharmacy benefit drugs more aggressively than medical benefit drugs. Plans more often restrict coverage of drugs with a therapeutic alternative, likely due to the fact that plans provide more generous access to drugs representing patients’ sole therapeutic option.

Although by definition orphan drugs are indicated for small populations (the FDA assigns orphan status to drugs indicated for diseases affecting fewer than 200,000 individuals

), drugs indicated for the most prevalent orphan diseases are more than twice as likely to be restricted as drugs indicated for the least prevalent orphan diseases. Health plans’ sensitivity to disease rarity suggests their attentiveness to drugs’ budget impact. This sensitivity may also explain the fact that drugs with the highest estimated annual costs are roughly twice as likely to be restricted as drugs with the lowest estimated annual costs. These findings are consistent with previous research that found that health plans were more likely to restrict specialty drugs (orphan and nonorphan specialty drugs) with higher annual costs and greater budget impacts.

 Concerns over the overall costs of orphan drugs seem likely to intensify (58% of all drugs approved by the FDA in 2018 were orphan drugs, for example).

This research provides a window into the struggles that plans face in light of an ever-increasing number of orphan drugs. Historically, plans have been able to absorb the costs of the few available orphan drugs. However, the growing number of orphan drugs and their larger budget impact likely mean that health plans will more often apply coverage restrictions to orphan drugs in the future. It is important for the medical community to understand that orphan drugs are not exempt from drug utilization management strategies.

Our study has a number of limitations. First, because the drug prices that plans negotiate with product manufacturers are not publicly disclosed, we rely on alternative sources of drug prices, not the actual prices paid by health plans. Second, our findings may not generalize to decisions made by other private health plans or by Medicare and Medicaid. Also, because we focus exclusively on orphan drugs for which plans have issued a coverage decision, our findings may not generalize to drugs that plans cover through formularies. Third, we do not account for patient cost sharing or other possible benefit design features. Fourth, there may be unobserved confounders influencing plan decision making. For instance, we do not control for the quality of the evidence that plans cite in their orphan drug coverage decisions. Because the FDA often bases orphan drug approvals on clinical studies with less rigorous study designs (eg, lack of control arms and small sample sizes), health plans are faced with uncertain evidence upon which to base their decisions.

The largest US private health plans restrict access to orphan drugs in approximately one-third of coverage decisions. Plans most often restrict coverage by requiring patients to meet certain clinical criteria, such as experiencing symptoms that result in limitation of activities of daily living. Plans vary in the frequency with which they apply coverage restrictions to their orphan drug coverage decisions. Plans more often add restrictions to drugs for diseases other than cancer, drugs with alternatives, self-administered drugs, drugs indicated for diseases with a higher prevalence, and drugs with higher annual costs.

 Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center (JDC, ADP, DDK, NMM, PJN), Boston, MA.

 This research study was supported by a consortium of funders: Amgen, Genentech, Janssen Pharmaceuticals, Novartis Pharmaceuticals, and Takeda.

 Dr Neumann is a consultant or paid advisory board member for Precision Health Economics, Avexis, AbbVie, Novartis Pharmaceuticals, Research Triangle Institute, Merck, Genentech, Bluebird Bio, GlaxoSmithKline, DePuy, Otsuka, ICON, Indivior, Acorda, and Biogen; has received grants from Amgen, Lundbeck, Gates Foundation, National Pharmaceutical Council, Alzheimer’s Association, National Institutes of Health, and PhRMA Foundation; and has received honoraria from AbbVie, Pfizer, Celgene, Sanofi, and GlaxoSmithKline. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (JDC, DDK); acquisition of data (JDC, ADP, NMM); analysis and interpretation of data (JDC, ADP, DDK, NMM); drafting of the manuscript (JDC, DDK, PJN); critical revision of the manuscript for important intellectual content (JDC, NMM, PJN); statistical analysis (ADP, DDK); obtaining funding (JDC, PJN); administrative, technical, or logistic support (PJN); and supervision (JDC).

 James D. Chambers, PhD, Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, 800 Washington St #63, Boston, MA 02111. Email: jchambers@tuftsmedicalcenter.org.

1. Handfield R, Feldstein J. Insurance companies’ perspectives on the orphan drug pipeline. 

2. Hyde R, Dobrovolny D. Orphan drug pricing and payer management in the United States: are we approaching the tipping point? 

3. Orphan drugs in the United States: growth trends in rare disease treatments. IQVIA website. iqvia.com/-/media/iqvia/pdfs/institute-reports/orphan-drugs-in-the-united-states-growth-trends-in-rare-disease-treatments.pdf?_=1544648528329. Published October 2018. Accessed March 12, 2019.

4. FAQs about rare diseases. National Institutes of Health National Center for Advancing Translational Sciences website. rarediseases.info.nih.gov/diseases/pages/31/faqs-about-rare-diseases. Updated November 30, 2017. Accessed March 12, 2019.

5. Chambers JD, Kim DD, Pope EF, Graff JS, Wilkinson CL, Neumann PJ. Specialty drug coverage varies across US commercial health plans. 

. 2018;37(7):1041-1047. doi: 10.1377/hlthaff.2017.1553.

6. Cohen JP, Awatin JG. Patient access to orphan drugs. 

. 2017;5(12):923-932. doi: 10.1080/21678707.2017.1402676.

Health plans restrict orphan drug coverage less often than nonorphan drug coverage. However, the frequency of restrictions varies considerably across plans.

Variation in US Private Health Plans’ Coverage of Orphan Drugs

 To provide recommendations that will improve approaches to measuring the value of new medical technologies to patients.

 Informed discussion by experts after literature review.

 A working group was formed, and participants discussed how value frameworks should incorporate key features important to patients in evaluating new medical technologies, particularly for chronic diseases.

 The working group suggests that new value frameworks should integrate real-world evidence to complement randomized controlled trials, incorporate the ways in which real-world behavior mediates outcomes, and explicitly discuss how therapies affect real-world equity and disparities in care.

 Collective stakeholders that include key decision makers within our healthcare system need to recognize the importance of implementing real-world evidence and devote resources to further research into the chronic disease areas in which the impact of human behavior is amplified by the duration of disease and treatment.

Steps have been taken to develop critical evaluations of drugs with the creation of value frameworks from organizations such as the Institute for Clinical and Economic Review, American Society of Clinical Oncology, and Memorial Sloan Kettering, but more work can and should be done to further refine these approaches. The Value FORWARD (Frameworks, Outcomes, and Real-World Adherence in Chronic Disease) Committee puts forth the following recommendations:

Value frameworks should incorporate real-world effectiveness to complement randomized controlled trial efficacy.

Value assessments must explicitly incorporate the ways in which real-world behavior mediates outcomes.

Value assessments should explicitly discuss how therapies affect real-world equity and disparities in care.

Policy makers, healthcare payers, patients, employers, and other stakeholders continue to fret about the rising cost of new medical technologies. Many scientific organizations have responded by developing new tools to assess the value of these technologies, commonly called “value frameworks.” For example, the American Society of Clinical Oncology (ASCO),

 and the Institute for Clinical and Economic Review (ICER) have offered alternative approaches to assessing the value of technologies. In principle, better information about value can help moderate growth in the cost of technologies that do not provide sufficient benefit to patients.

Yet there remains a gap between the goal of measuring value conferred to patients and the available evidence for new medical technologies. Randomized controlled trial (RCT) evidence generated for regulatory approval often incorrectly measures or fails to measure aspects of value that are key to patients. Some of these gaps and limitations are widely understood,

 but others have received too little attention, such as the role of patient behavior in mediating clinical outcomes.

 For example, evidence generated by RCTs often reflects outcomes achieved with motivated patients treated in environments in which individual patient behaviors that are critical to driving real-world outcomes, including adherence to medications, are controlled. As a result, interventions that improve adherence may not demonstrate large efficacy benefits in RCTs but may demonstrate larger benefits in real-world settings in which adherence is typically lower.

A working group was formed in partnership with Precision Health Economics and Intarcia Therapeutics in 2017 to make recommendations on how value frameworks should close the gap between evidence and patient behavior when evaluating new medical technologies, particularly for chronic diseases. Our working group recommended that these value frameworks enhance attention to (1) real-world evidence (RWE), (2) the role of adherence, and (3) nonclinical components of the patient experience.

New value frameworks should incorporate real-world evidence that reflects patient treatment behavior, adherence to medication, and equity concerns arising from disparities in care.

Peter J. Neumann, ScD

Articles

Crisis Into Opportunity: Can COVID-19 Help Set a Path to Improved Health Care Efficiency?

Outcome Measures for Oncology Alternative Payment Models: Practical Considerations and Recommendations

Variation in US Private Health Plans’ Coverage of Orphan Drugs

Understanding and Improving Value Frameworks With Real-World Patient Outcomes