Currently Viewing:
Evidence-Based Oncology May/June
I-SPY Trials Seek Effective New Drug Combinations Using Adaptive Trial Design
Ben Leach
Currently Reading
Third-Party Validation of Observed Savings From an Oncology Pathways Program
Bruce A. Feinberg, DO; Scott Milligan, PhD; Joseph Cooper, BS; Winston Wong, PharmD; Daniel Winn, MD; Neil Schneider, ASA, MAAA; Sheamus Parkes, FSA, MAAA; and Jeffrey Scott, MD
Health Plan Innovations in Delivery System Reforms
Karen Ignagni
Framing the Debate: The Trends in Mastectomy Rates
Jennifer Klem, PhD
Looking at Specialty Pharmaceutical Rejections
Dea Belazi, PharmD, MPH
The Impact of Health Reform's State Exchanges
The Changing Landscape of Oncology: Impact for Payers and Providers
Dawn Holcombe, MBA
Sledge Expects 'Big Data' to Generate Big Changes
Anita T. Shaffer
Use of Personalized Medicine to Select Adjuvant Breast Cancer Therapy
Anna Azvolinsky, PhD
NCCN Guidelines Update: NCCN 18th Annual Conference, Advancing the Standards of Cancer Care

Third-Party Validation of Observed Savings From an Oncology Pathways Program

Bruce A. Feinberg, DO; Scott Milligan, PhD; Joseph Cooper, BS; Winston Wong, PharmD; Daniel Winn, MD; Neil Schneider, ASA, MAAA; Sheamus Parkes, FSA, MAAA; and Jeffrey Scott, MD
Cancer death rates in the United States have declined 20% from their peak in 1991 (215.1 per 100,000 population) to 2009 (173.1 per 100,000 population).1 Patients are living longer with cancer, and the improvement in survival rates reflects progress in diagnosing certain cancers at an earlier stage and improvements in treatment.2 The tremendous scientific progress that has led to new therapeutic agents and tools for diagnosis and follow-up has resulted in: (1) marked increase in cancer survivorship; (2) longer periods of active treatment; and (3) marked increased variability in treatment options, all at great economic cost.3-6 In the United States, direct medical costs associated with cancer are projected to increase exponentially from $104 billion in 2006 to over $173 billion in 2020.7 Oncology cancer treatment guidelines, or clinical pathways, have been suggested as a way to curb the rising cost of cancer care by preventing unnecessary and costly treatment variations while improving quality of care.7-10

Cardinal Health, based in Dublin Ohio, partners with payers to establish evidence-based oncology treatment pathways with the intent of eliminating unnecessary medical interventions and promoting the most cost-effective treatments to realize enhanced care and significant savings. In August 2008, Cardinal Health Specialty Solutions partnered with a large nonprofit healthcare insurer for the Mid-Atlantic region of the United States to launch the first oncology clinical pathways program by a major regional insurer across its entire network of disparate providers, which included hospital- and community based practices as well as practices affiliated with academic institutions. The program included pathways for breast, lung, and colon cancer. Participation was voluntary; however, providers were incentivized to join, with increased fee schedule payments for compliance. Using claims–based analysis, results of this program were reported in a poster presentation at the annual meeting of the American Society of Clinical Oncology in 2010.14 Decreases in drug and nondrug expenses for patients treated on pathways net of program management fees and provider participation incentives resulted in more than $8 million in savings from this program.11,12

Athough cost savings from pathways in the integrated practice setting have been reported, more studies are needed to validate the real-world cost savings of pathways programs among disparate provider networks. The US Oncology Network reported significantly lower costs for patients with non-small cell lung cancer and colon cancer treated on pathway programs versus patients treated off pathways, with similar overall survival rates.13,14 Similarly, Via Oncology pathways reported preliminary cost data from a pathways pilot project with Highmark Blue Cross Blue Shield, which showed that the total cost of care for patients under pathways practices grew at a slower rate than the same costs for patients in nonpathways programs. 15 These results are limited to the integrated practice setting, where uniform technology platforms and centralized management can drive compliance. Akin to single-institution clinical research, we believe that assumptions of similar cost savings are not applicable to disparate providers across a regional payer network or defined geography without validation. Such validation has been thwarted by the absence of unified technology platforms to drive the pathway, consensus among pathway participants, and financial sponsorship of such an initiative.

The purpose of this study was to have a third-party actuary/analytical team validate the observed savings and financial impact of this pathways program.


Patient Identification

This was an observational study with a retrospectively identified control group. Milliman Inc, an international provider of actuarial and related products and services, performed the analyses. Claims data from the payer database from January 2007 to December 2010 was used to identify the pathways cohort. Patients with a primary or secondary diagnosis of cancer (International Statistical Classification of Diseases-9 diagnosis codes 140-239) who initiated chemotherapy between April 1, 2007, and April 5, 2010, at the pathways-participating practices were included. Patients older than 63 years, patients treated by practices that exited pathways or joined pathways after 2008, or patients with no allowed costs (due to billing oddities) were excluded. The control group was identified using Truven Health Analytics MarketScan databases. MarketScan is a large commercial claims database that has included more than 170 million unique patients since 1995. All patients with a primary or secondary diagnosis of breast, colon, or lung cancer who initiated chemotherapy between April 1, 2007, and April 5, 2010, located in the Mid-Atlantic (Delaware, North Carolina, irginia, and West Virginia) and New England (Massachusetts, Maine, New Hampshire, and Vermont) regions of the United States were included. Patients who were older than 63 years, received inpatient chemotherapy, had no allowed costs, or were from Alexandria, Virginia (to avoid overlap with our pathways-participating oncologists), were excluded. Pennsylvania was excluded due to claims costs that were more than 20% higher than the included geographies and an active competitor to our pathways.

Propensity score weighting by cancer type (breast, colon, lung), age, and date of first chemotherapy treatment was used to further balance the 2 groups. We utilized a boosted decision tree model to calculate the probability of being in the treatment group based on known features.16 The predicted probabilities were then used to develop weights for the control group that distort the distribution of their features to match those of the pathways study group, estimating the average treatment effect on the study population. The Kolmogorov-Smirnov test was used to compare the distribution of the features.

Statistical Analysis

The primary outcome was the sum of a patient’s allowed cancer costs up to 270 days after initiation of chemotherapy. The primary linear model used the natural logarithm of cancer costs as the outcome and included at least a unique intercept for each data source, parallel cost curves, and a treatment indicator for patients who began treatment under pathways. The linear models used balancing weights derived from the propensity scores, unless otherwise stated in the sensitivity analysis. The complete list of features in the linear models included: cancer type, calendar time (splined), age (splined), data source indicator (our pathways, MarketScan Atlantic, MarketScan New England), treatment indicator (pathways or all other), nd MarketScan seasonality artifact (splined).

Sensitivity testing was performed using a variety of alternative model specifications to add credibility to the results of the primary linear model, including the following: less cost trend wiggle, reducing the degrees of freedom for the cost curve spline, which results in the cost trend component of the model absorbing less cost variation; more cost trend wiggle, expanding the degrees of freedom for the cost curve spline, which results in the cost trend component of the model absorbing more cost variation; Drop 5 (Cook’s distance), a model measuring the effect of deleting the 5 most influential observations; Drop 5 (DFBeta), a model removing the 5 most influential observations based on their direct influence on the treatment coefficient; gamma generalized linear model, a model changing response to its original scale, and then fitting a generalized linear model assuming a conditional gamma distribution and a log-link; and no MarketScan seasonality artifact, a model removing the feature that captures the fact that claims data from the last quarter of a calendar year may be incomplete in MarketScan.

A secondary outcome was the probability of an inpatient hospital admission during the 270 days after chemotherapy initiation. A logistic regression model was used that included the same feature set used in the primary linear model. Sensitivity testing was also performed using several alternative models.

Boosted decision tree models also were fit to the primary and secondary outcomes to explore all nonlinearities and interactions. In addition to the primary and secondary outcomes, analyses were performed to determine whether there was a reduction in the number of drug combinations used for patients treated under pathways.Only patients undergoing first lines of therapy were used in these analyses because the cohorts were limited to patients initiating treatment, and, therefore, fewer patients received late lines of therapy during the study period. Variability was calculated by cancer type; colon cancer was excluded due to the minimal number of drug combinations used in practice.

Propensity score weights were not utilized. Three different variability metrics were considered: (1) percentage of lines of therapy utilizing the 5 most common drug combinations; (2) percentage of lines of therapy utilizing the 10 most common drug combinations; and (3) effective number of drug combinations. The effective number of drug combinations measures the concentration in the most prevalent drug combinations. If all drug combination possibilities were equally utilized, the effective number would be the actual number. In reality, the effective number will always be ess than the number of possible drug combinations because it discounts those applied rarely. All of the variability metrics were computed for numerous centered rolling windows over the study period, which allowed visualization of smoothly changing results.


A total of 2424 patients from the payer database met eligibility criteria for the pathways cohort, and 1490 patients from the MarketScan database met eligibility criteria for the control group.Propensity score weighting further reduced the control group to an effective sample size of 1400.16 Table 1 shows the selection of the final cohorts. Propensity score analysis is shown in Table 2, which gives simple summary statistics of the known features before and after balancing. For the categorical variable (cancer type), the P value is approximated by a χ2 test and is only reported for a single value (breast cancer). The boosted decision tree model was optimized to minimize the largest Kolmogorov-Smirnov statistic. This value was minimized from 0.06 in the unbalanced table to 0.01 in the  balanced table. Most of the balancing effort was focused on the age and date of starting chemotherapy treatment. The boosted decision tree model additionally balanced the interaction of each of the features, such as the age curves by cancer type. The actual distortion was modest, however; the groups were already fairly well aligned for the known features.


Copyright AJMC 2006-2018 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!