Currently Viewing:
The American Journal of Managed Care January 2017
Alignment of Breast Cancer Screening Guidelines, Accountability Metrics, and Practice Patterns
Tracy Onega, PhD; Jennifer S. Haas, MD; Asaf Bitton, MD; Charles Brackett, MD; Julie Weiss, MS; Martha Goodrich, MS; Kimberly Harris, MPH; Steve Pyle, BS; and Anna N. A. Tosteson, ScD
The Challenge of Paying for Cost-Effective Cures
Patricia J. Zettler, JD, and Erin C. Fuse Brown, JD, MPH
Currently Reading
An Expanded Portfolio of Survival Metrics for Assessing Anticancer Agents
Jennifer Karweit, MS; Srividya Kotapati, PharmD; Samuel Wagner, PhD; James W. Shaw, PhD, PharmD, MPH; Steffan W. Wolfe, BA; and Amy P. Abernethy, MD, PhD
Value-Based Payment in Implementing Evidence-Based Care: The Mental Health Integration Program in Washington State
Yuhua Bao, PhD; Thomas G. McGuire, PhD; Ya-Fen Chan, PhD; Ashley A. Eggman, MS; Andrew M. Ryan, PhD; Martha L. Bruce, PhD, MPH; Harold Alan Pincus, MD; Erin Hafer, MPH; and Jürgen Unützer, MD, MPH,
Patient-Centered Care: Turning the Rhetoric Into Reality
Joel S. Weissman, PhD; Michael L. Millenson, BA; and R. Sterling Haring, DO, MPH
The Effect of Massachusetts Health Reform on Access to Care for Medicaid Beneficiaries
Laura G. Burke, MD, MPH; Thomas C. Tsai, MD, MPH; Jie Zheng, PhD; E. John Orav, PhD; and Ashish K. Jha, MD, MPH
The Value of Survival Gains in Myelodysplastic Syndromes
Joanna P. MacEwan, PhD; Wes Yin, PhD; Satyin Kaura, MSci, MBA; and Zeba M. Khan, PhD
Electronic Health Records and the Frequency of Diagnostic Test Orders
Ibrahim Hakim, BBA; Sejal Hathi, BS; Archana Nair, MS; Trishna Narula, MPH; and Jay Bhattacharya, MD, PhD
An Assessment of the CHIP/Medicaid Quality Measure for ADHD
Justin Blackburn, PhD; David J. Becker, PhD; Michael A. Morrisey, PhD; Meredith L. Kilgore, PhD; Bisakha Sen, PhD; Cathy Caldwell, MPH; and Nir Menachemi, PhD, MPH

An Expanded Portfolio of Survival Metrics for Assessing Anticancer Agents

Jennifer Karweit, MS; Srividya Kotapati, PharmD; Samuel Wagner, PhD; James W. Shaw, PhD, PharmD, MPH; Steffan W. Wolfe, BA; and Amy P. Abernethy, MD, PhD
A novel, simplified cost-value analysis tool was created to better differentiate the value of anticancer agents and further characterize the expected survival benefit of all patients.
The results of the cost-value analysis varied depending on the applied metric, as some agents achieved a higher than average cost value (appearing above the fitted regression line) with some metrics, but not with others (Figure). The greatest cost value based on median OS, mean OS, and NNT was provided by ado-trastuzumab emtansine (second-line BC; Figure [a]), ipilimumab (first- and second-line melanoma; Figure [b]), and ipilimumab (first- and second-line melanoma; Figure [c]), respectively. Higher clinical value based on the individual survival metrics (Table 1) did not necessarily translate to a higher cost-value benefit (Figure).


The cancer therapeutic landscape is changing, with novel agents being introduced with differing response durability and disease outcomes. When assessing treatment, it is important to look at survival metrics by disease category because baseline outcomes differ so remarkably (eg, in the last decade, median OS was 25 months in metastatic BC30 vs 8 months in metastatic melanoma31). Our analysis suggests a pattern of evolving disease outcomes (eg, increasing 1-year OS rates that outpace increasing median and mean OS) aligned by disease and its relative emerging therapeutic landscape. Although metastatic melanoma, NSCLC, and RCC have been historically recognized as diseases with very poor survival rates, the advent of new agents is shifting the placement and contour of the survival curve.16-18

Several metrics are currently used in cancer trials, each with positive and negative characteristics (Table 2); among them, median OS is the most commonly used metric.22 However, median OS requires a long data collection period and a large sample size,19 and it may underestimate the full survival benefit because it overlooks patients who are alive at the end of the study follow-up.10,16,32 This scenario may be encountered with new treatments that prolong survival in a nontrivial proportion of patients (eg, immuno-oncology agents).16-18 For example, ipilimumab (a monoclonal antibody that blocks the immune checkpoint inhibitor molecule cytotoxic T-lymphocyte antigen-4) demonstrated a plateau in the survival curve that began approximately 3 years after initiating treatment in 21% of patients and continued for up to 10 years.17,33

Mean OS, which represents the area under the full survival curve, is more sensitive than median OS to the shape of the final portion of the curve, because it takes into account all patients, not just the surviving 50% (Table 2).10,22 Mean OS is also considered the standard metric for determining cost-effectiveness of cancer treatments,22 and, when estimated alone or as part of a cost-effectiveness analysis, it is required by many healthcare payers and health technology assessment agencies (eg, the United Kingdom’s National Institute for Health and Care Excellence).22 However, unlike median OS, mean OS is not as good an indicator of the central tendency of survival times.10,22

PFS was not included in our assessment tool, as it is frequently considered a surrogate outcome and a proxy for median OS (Table 2). Its role in estimating OS frequently comes into question, however, as PFS is subject to measurement error. It is possible that some agents—especially those targeting cell signaling and angiogenesis—may, with chronic administration, delay progression for a time, but lead to evolutionary changes in tumors, thereby producing a more aggressive phenotype, and subsequently offsetting the earlier delay in progression.34

Survival rate can be highly effective in comparing therapies within a tumor type,35 and NNT is useful for assessing the effectiveness of agents across populations (Table 2).27,28 However, survival rate does not specify whether survivors are still undergoing treatment or have achieved remission, and NNT is limited by difficulties in interpreting results when the treatment or follow-up period is not stated,36,37 as well as because it is not an absolute value and depends on the comparison of 2 treatment groups.38

For this cost-value analysis, we assigned a cost to the therapy based on a reported cost per month multiplied by the duration of therapy. The duration of therapy was based largely on data availability and varied between median duration of administration, median PFS, and median time to progression. Using medians for assessing duration of therapies would theoretically lead to underestimations of costs compared with analysis based on means. However, if costs are estimated similarly for all comparators, the bias should be fairly consistent across treatments.

Our preliminary quantitative and qualitative assessment shows that there is no unique or preferred relative clinical value metric for anticancer agents, but rather that an expanded metrics portfolio may be required. As shown here, an individual agent can have a high relative clinical value using 1 survival outcome and low value using another. In particular, there is variation between median OS and the other metrics. To assess relative clinical value, we developed a new cost-value analysis tool that graphically plots total drug cost versus the survival outcome. Using this tool, a higher relative clinical value based on a particular survival outcome did not necessarily translate into a greater economic value, further underscoring that no single metric is optimal. Future research should seek to provide guidance on how to determine optimal sets of metrics to assess value, particularly within the context of a specific indication.

The relative value assessment (RVA) tool we describe may fit into the spectrum of other value assessment tools/metrics, such as the American Society of Clinical Oncology (ASCO) Value Framework,39 the European Society for Medical Oncology’s Magnitude of Clinical Benefit Scale (ESMO-MCBS),40 the National Comprehensive Cancer Network Value Pathways,41 the Institute for Clinical and Economic Review Value Assessment Project,42 and Memorial Sloan Kettering Cancer Center’s DrugAbacus. These value assessment tools/metrics differ with regard to methodology and parameters assessed. For example, the ASCO Value Framework compares a new agent with the current standard of care for an indication using efficacy and safety data derived from a prospective randomized trial,39 whereas the ESMO-MCBS assesses an agent’s value through survival, QOL, and safety data from comparative outcome studies (randomized/comparative cohort studies or meta-analyses).40 Although these value frameworks have their respective limitations (eg, lack of assessment of potential cost to patients and complex user methodology), each offers important insights into the value of cancer treatments that may aid treatment decision making. Our approach using a portfolio of outcomes molded to a specific cancer indication can supplement these solutions.


Although our results provide insights into the assessment of the relative value of anticancer agents, several factors limit the data interpretation. First, determination of clinical value is subjective and can be approached in a number of ways, not just the ones used here. Additionally, the data sets in this analysis may not be fully mature because they were derived from phase 3 studies with predefined cut-off periods. As patient-level data were not available for all agents, mean OS, 1-year survival rate, and NNT were also estimated from digitized Kaplan-Meier curves rather than actual data points; this may result in underestimation of mean survival due to truncation of the data. Furthermore, the analysis incorporated indirect (naïve) comparisons among trials that failed to account for a tumor-specific prognosis and differences in patient characteristics (eg, number of lines of prior therapy). This analysis may also suffer from selection bias because evaluated agents were required to meet the predefined inclusion criteria and may not have been representative of the entire treatment landscape. Moreover, the cost value assigned did not take into account symptom burden, drug toxicity, or QOL, which is particularly important with newer therapies that are frequently less toxic than traditional chemotherapies. In real life, patients with cancer are treated with a variety of drugs simultaneously, making it difficult to attribute value to an individual drug. Hence, results of this model pertain to value expectations at the beginning of therapy, and should not be used to guide continued treatment. Lastly, our cost-value analysis did not take into account insurance discounting of drug costs.


In a healthcare environment hampered by intensive budgetary constraints, stakeholders struggle to contain costs while providing the best care possible. As novel and more effective oncology products—many of which have high price tags—are introduced, new methods for estimating relative clinical value are sought. Our preliminary qualitative and quantitative analysis, which used more and different metrics than may be the standard, suggests that a broad array of survival outcomes are required to fully assess and benchmark the relative clinical value of anticancer agents. This approach becomes progressively more important as drugs transition from clinical development to regulatory approval and widespread application. The portfolio of measures assessing impact needs to be more broadly meaningful in general populations; our concept of a measure portfolio starts to move in that direction. Including more therapeutic areas, basing the models on actual data points, and incorporating QOL measurements and other patient-focused concerns could further enhance this tool. Further research should concentrate on aligning best-value metrics and creating guidelines for prioritizing metrics when results differ. A mature RVA would enable more informed decisions by payers and providers in managed care, while guidelines for prioritizing metrics may decrease disagreement between stakeholders.


Professional medical writing assistance was provided by Mark Palangio and professional editing assistance was provided by Matthew Dougherty at StemScientific; these were funded by Bristol-Myers Squibb. This study was presented in part at the International Society for Pharmacoeconomics and Outcomes Research 19th Annual International Meeting, May 31 to June 4, 2014, Palais des Congrès de Montréal, Montréal, QC, Canada.

Author Affiliations: QuintilesIMS (JK, SWW), New York, NY; Bristol-Myers Squibb Company (SK, SW, JWS), Princeton, NJ; Duke Clinical Research Institute and Duke Cancer Institute (APA), Durham, NC; Flatiron Health, Inc (APA), New York, NY.
Source of Funding: This study was funded by Bristol-Myers Squibb.
Author Disclosures: Ms Karweit is a consultant to Bristol-Myers Squibb (BMS) for evaluating survival metrics. Drs Kotapati, Wagner, and Shaw are employees of BMS. Drs Wagner and Shaw own stock in BMS. Mr Wolfe is a former employee of QuintilesIMS. Dr Abernethy is a paid consultant for BMS and Helsinn Therapeutics and is chief medical officer of Flatiron Health, Inc.
Authorship Information: Concept and design (JK, SK, SW, JWS, SWW, APA); acquisition of data (JK, JWS, SWW, APA); analysis and interpretation of data (JK, SK, SW, JWS, SWW, APA); drafting of the manuscript (JK, SK, SW, JWS, SWW, APA); critical revision of the manuscript for important intellectual content (JK, SK, SW, JWS, SWW, APA); provision of patients or study materials (SK, JWS); obtaining funding (SK, JWS); administrative, technical, or logistic support (SK, JWS); and supervision (SK, SW, JWS). 
Address Correspondence to: Jennifer Karweit, MS, QuintilesIMS, 485 Lexington Ave, 26th Fl, New York, NY 10017. E-mail:

1. Abernethy A, Abrahams E, Barker A, et al. Turning the tide against cancer through sustained medical innovation: the pathway to progress. Clin Cancer Res. 2014;20(5):1081-1086. doi: 10.1158/1078-0432.CCR-13-3430.
2. International Agency for Research on Cancer. All cancers (excluding non-melanoma skin cancer) estimated incidence, mortality and prevalence worldwide in 2012. GLOBOCAN website. Accessed November 29, 2016.
3. Sleijfer S. “It’s the economy, stupid”: strategies for improved cost containment in cancer treatment. Clin Pharmacol Ther. 2014;95(4):365-367. doi: 10.1038/clpt.2014.9.
4. American Association for Cancer Research. AACR Cancer Progress Report 2013. Clin Cancer Res. 2013;19(suppl 1):S1-S88.
5. Jagsi R, Sulmasy DP, Moy B. Value of cancer care: ethical considerations for the practicing oncologist. Am Soc Clin Oncol Educ Book. 2014:e146-e149. doi: 10.14694/EdBook_AM.2014.34.e146.
6. Mariotto AB, Yabroff KR, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst. 2011;103(2):117-128. doi: 10.1093/jnci/djq495.
7. Erikson C, Salsberg E, Forte G, Bruinooge S, Goldstein M. Future supply and demand for oncologists: challenges to assuring access to oncology services. J Oncol Pract. 2007;3(2):79-86. doi: 10.1200/JOP.0723601.
8. ASCO in action brief: value in cancer care. American Society of Clinical Oncology website. Published January 21, 2014. Accessed November 29, 2016.
9. Committee on Improving the Quality of Cancer Care: Addressing the Challenges of an Aging Population; Board on Health Care Services; Institute of Medicine; Levit L, Balogh E, Nass S, Ganz PA (eds). Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis. Washington, DC: National Academies Press; 2013.
10. Davies A, Briggs A, Schneider J, et al. The ends justify the mean: outcome measures for estimating the value of new cancer therapies. Health Outcomes Res Med. 2012;3(1):e25-e36. doi: 10.1016/j.ehrm.2012.01.001.
11. Ramsey S, Schickedanz A. How should we define value in cancer care? Oncologist. 2010;15(suppl 1):1-4. doi: 10.1634/theoncologist.2010-S1-1.
12. Miller JD, Foley KA, Russell MW. Current challenges in health economic modeling of cancer therapies: a research inquiry. Am Health Drug Benefits. 2014;7(3):153-162.
13. Howard DH, Bach PB, Berndt ER, Conti RM. Pricing in the market for anticancer drugs. J Econ Perspect. 2015;29(1):139-162.
14. Mailankody S, Prasad V. Five years of cancer drug approvals: innovation, efficacy, and costs. JAMA Oncol. 2015;1(4):539-540. doi: 10.1001/jamaoncol.2015.0373.
15. Abernethy AP, Kotapati S, Gilloteau I, et al. The relative clinical and economic value of ipilimumab as first-line treatment of metastatic melanoma versus other anti-cancer agents for metastatic diseases. Presented at: International Society for Pharmacoeconomics and Outcomes Research 19th Annual International Meeting; May 31-June 4, 2014; Montreal, QC, Canada.
16. Hodi FS, O’Day SJ, McDermott DF, et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010;363(8):711-723. doi: 10.1056/NEJMoa1003466.
17. Eggermont A, Robert C, Soria JC, Zitvogel L. Harnessing the immune system to provide long-term survival in patients with melanoma and other solid tumors. Oncoimmunology. 2014;3(1):e27560.
18. McDermott D, Lebbé C, Hodi FS, et al. Durable benefit and the potential for long-term survival with immunotherapy in advanced melanoma. Cancer Treat Rev. 2014;40(9):1056-1064. doi: 10.1016/j.ctrv.2014.06.012.
19. Pazdur R. Endpoints for assessing drug activity in clinical trials. Oncologist. 2008;13(suppl 2):19-21. doi: 10.1634/theoncologist.13-S2-19.
20. Driscoll JJ, Rixe O. Overall survival: still the gold standard: why overall survival remains the definitive end point in cancer clinical trials. Cancer J. 2009;15(5):401-405. doi: 10.1097/PPO.0b013e3181bdc2e0.
21. Cheema PK, Burkes RL. Overall survival should be the primary endpoint in clinical trials for advanced non-small-cell lung cancer. Curr Oncol. 2013;20(2):e150-e160. doi: 10.3747/co.20.1226.
22. Joulain F, Proskorovsky I, Allegra C, et al. Mean overall survival gain with aflibercept plus FOLFIRI vs placebo plus FOLFIRI in patients with previously treated metastatic colorectal cancer. Br J Cancer. 2013;109:1735-1743. doi:10.1038/bjc.2013.523.
23. Hind D, Tappenden P, Tumur I, Eggington S, Sutcliffe P, Ryan A. The use of irinotecan, oxaliplatin and raltitrexed for the treatment of advanced colorectal cancer: systematic review and economic evaluation. Health Technol Assess. 2008;12(15):iii-ix, xi-162.
24. Tappenden P, Jones R, Paisley S, Carroll C. Systematic review and economic evaluation of bevacizumab and cetuximab for the treatment of metastatic colorectal cancer. Health Technol Assess. 2007;11(12):1-128, iii-iv.
25. Malinchoc M, Kamath PS, Gordon FD, Peine CJ, Rank J, ter Borg PC. A model to predict poor survival in patients undergoing transjugular intrahepatic portosystemic shunts. Hepatology. 2000;31(4):864-871.
26. Altman DG, Andersen PK. Calculating the number needed to treat for trials where the outcome is time to an event. BMJ. 1999;319(7223):1492-1495.
27. Cook RJ, Sackett DL: The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995;310(6977):452-454.
28. McQuay HJ, Moore RA. Using numerical results from systematic reviews in clinical practice. Ann Intern Med. 1997;126(9):712-720.
29. Spano D, Heck C, De Antonellis P, Christofori G, Zollo M. Molecular networks that regulate cancer metastasis. Semin Cancer Biol. 2012;22(3):234-249. doi: 10.1016/j.semcancer.2012.03.006.
30. Dawood S, Broglio K, Gonzalez-Angulo AM, Buzdar AU, Hortobagyi GN, Giordano SH. Trends in survival over the past two decades among white and black patients with newly diagnosed stage IV breast cancer. J Clin Oncol. 2008;26(30):4891-4898. doi: 10.1200/JCO.2007.14.1168.
31. Balch CM, Gershenwald JE, Soong SJ, et al. Final version of 2009 AJCC melanoma staging and classification. J Clin Oncol. 2009;27(36):6199-6206. doi: 10.1200/JCO.2009.23.4799.
32. Messori A, Becagli P, Trippoli S. Median versus mean lifetime survival in the analysis of survival data. Haematologica. 1997;82(6):730.
33. Schadendorf D, Hodi FS, Robert C, et al. Pooled analysis of long-term survival data from phase II and phase III trials of ipilimumab in unresectable or metastatic melanoma. J Clin Oncol. 2015;33(17);1889-1994. doi: 10.1200/JCO.2014.56.2736.
34. Booth CM, Eisenhauer EA. Progression-free survival: meaningful or simply measurable? J Clin Oncol. 2012;30(10):1030-1033. doi: 10.1200/JCO.2011.38.7571.
35. Mandrekar SJ, Qi Y, Hillman SL, et al. Endpoints in phase II trials for advanced non-small cell lung cancer. J Thorac Oncol. 2010;5(1):3-9. doi: 10.1097/JTO.0b013e3181c0a313.
36. Stang A, Poole C, Bender R. Common problems related to the use of number needed to treat. J Clin Epidemiol. 2010;63(8):820-825. doi: 10.1016/j.jclinepi.2009.08.006.
37. Suissa D, Brassard P, Smiechowski B, Suissa S. Number needed to treat is incorrect without proper time-related considerations. J Clin Epidemiol. 2012;65(1):42-46. doi: 10.1016/j.jclinepi.2011.04.009.
38. Curtin F, Schulz P. Assessing the benefit: risk ratio of a drug—randomized and naturalistic evidence. Dialogues Clin Neurosci. 2011;13(2):183-190.
39. Schnipper LE, Davidson NE, Wollins DS, et al. American Society of Clinical Oncology Statement: a conceptual framework to assess the value of cancer treatment options. J Clin Oncol. 2015;33(23):2563-2577. doi: 10.1200/JCO.2015.61.6706.
40. Cherny NI, Sullivan R, Dafni U, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol. 2015;26(8):1547-1573. doi: 10.1093/annonc/mdv249.
41. The National Comprehensive Cancer Network, The US Oncology Network and McKesson Specialty Health collaborate to offer new Value Pathways for cancer care [news release]. Fort Washington, PA: National Comprehensive Cancer Network. Accessed November 29, 2016.
42. ICER Value Assessment Framework. Institute for Clinical and Economic Review website. Accessed November 29, 2016.
Copyright AJMC 2006-2017 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!