We evaluated an alternative way to implement guidelines using an automated risk calculator and risk-based decision tool to calculate patients’ risk of cardiovascular disease (CVD) and recommend therapies. We compared such an approach with traditional guidelines.

A retrospective cohort study of 1,506,109 Kaiser Permanente Southern California members 35 years or older.

We estimated 3-year risks of fatal and nonfatal myocardial infarction and stroke using an independently developed risk calculator, then graphically compared risks with observed outcomes. We used the area under the receiver operating characteristics curve to assess discrimination, and the Hosmer-Lemeshow statistic to test fit. We compared the characteristics and outcomes of populations identified for medication therapy by the risk-based decision tool and traditional guidelines using bivariate statistics.

A risk score was obtained in 72% (1,082,158) of members. The risk calculator was fairly good in discrimination: the area under the curve was 0.774 (95% CI, 0.770-0.779) for myocardial infarction and 0.805 (95% CI, 0.801-0.808) for stroke. Predictiveness and fit was good based on graphical analysis and Hosmer-Lemeshow 

<.0001. The risk-based decision tool identified high-risk patients for treatment who were not identified by traditional guidelines (3.80% of all those identified for statins, 3.04% for antihypertensives), as well as low-risk patients who were identified by guidelines (3.80% for statins, 2.51% for antihypertensives).

The risk calculator provided risk estimates in most patients and demonstrated fairly good discrimination and predictiveness. The risk-based decision tool identified high-risk patients for treatment not identified by traditional guidelines, as well as low-risk patients for whom treatment may be unnecessary.

	Automated risk calculator and guideline decision support tools can move clinical care a step closer to systematically individualizing patient therapy and population healthcare.

		The proportion of the population for whom sufficient electronic data were available to make risk estimation was good.

		The risk-based decision tool performed well in discrimination and predictiveness.  

		The risk-based decision tool identified high-risk individuals missed by traditional guidelines, and low-risk individuals who were recommended treatment based on traditional guidelines.

Clinical practice guidelines traditionally have specified levels of biomarkers to determine whether or not patients should be treated and to specify treatment goals. For example, the Seventh Report of the Joint National Committee (JNC-7) guideline for cholesterol specifies 3 different risk strata based on LDL cholesterol levels.

 Traditional guidelines can include other risk factors that are typically secondary to the biomarker target, such as gender or smoking status. A notable exception to the traditional approach is the new American College of Cardiology (ACC)/American Heart Association (AHA) cholesterol guideline. Although not explicitly dropping the use of cholesterol treatment goals, these new guidelines endorse the use of a risk calculator to guide treatment.

 Depending on their risk of a cardiovascular event, as estimated by a calculator, patients with high cholesterol levels may not be treated and patients with low cholesterol levels may be treated.

We wished to understand the potential implications of using risk calculators and risk-based decision tools to help risk stratify a population and identify individual patients for treatment. In the first part of our study, we evaluated the population applicability, discrimination, and predictiveness of an independently developed cardiovascular disease (CVD) risk calculator.

 The risk calculator was part of a risk-based decision tool under consideration by Kaiser Permanente Southern California (KPSC) for use in place of standard clinical practice guidelines. The validation of the risk calculator against outcomes was considered an essential first step before comparing the risk-based decision tool to traditional care. In the second part of our study, we tested the potential population impact of the risk-based decision tool by comparing actual outcomes in patients selected for possible treatment by the risk-based decision tool versus those selected by traditional clinical practice guidelines.

We conducted a retrospective cohort study of KPSC members with active membership during 2007 to 2010. The ending time point was chosen because at the time of initial analyses, government mortality data were not available past 2010. In the first part of this study, we determined the proportion of an adult population to which the risk calculator could be applied, then we compared predicted and observed outcomes. In the second part of the study, we determined which patients would be selected for medication therapy by the risk-based decision tool and by traditional guidelines, then we compared observed outcomes in these groups. Archimedes, Inc developed the risk calculator and the risk-based decision tool, and was involved in study planning and interpretation, but not in data collection or analysis. No data from KPSC were used to create the risk calculator or the risk-based decision tool evaluated in this study. The study was approved by the KPSC Institutional Review Board.

KPSC is a prepaid, integrated health system with 3.5 million members of diverse race and ethnicity at 14 medical centers and 197 medical offices throughout southern California. Members have similar coverage benefits and a limited range of co-payments for services and medications. Subjects 35 years or older were selected from among health plan members who were alive on December 31, 2007, with no gap in enrollment of more than 45 days in the year prior to that date.

The electronic health record (EHR) was used to identify pre-existing conditions using 

International Classification of Diseases, Ninth Revision, Clinical Modification

) codes assigned during inpatient or outpatient healthcare encounters. We collected data on past cardiovascular risk—related disease history from January 1, 2004 (up to 4 years prior to study outcomes), including diabetes, stroke, myocardial infarction, atrial fibrillation, congestive heart failure, and chronic kidney disease. For 2007, we extracted all the clinical information needed for risk calculation from laboratory and pharmacy data files. Laboratory data included fasting plasma glucose, glycated hemoglobin, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, total cholesterol, triglycerides, serum creatinine, and measures of proteinuria (eg, microalbumin-to-creatinine ratio). We identified medication dispensing dates for antihypertensives (ie, angiotensin converting-enzyme inhibitors and angiotensin receptor blockers, calcium channel-blockers, thiazide diuretics, beta-blockers), cholesterol-lowering drugs (ie, statins and non-statins), and insulin. We identified additional key risk factors from the EHR, including recent smoking, nonprescription aspirin use, body mass index, and blood pressure measurements.

We identified fatal and nonfatal myocardial infarction and cerebrovascular (ie, stroke) events from January 1, 2008, to December 31, 2010. These 3-year outcomes were identified from the EHR using 

 hospital discharge diagnosis codes (myocardial infarction: 410; and stroke: 346.6x, 430, 431, 432.x, 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.x, 436). Hospitalization discharge diagnosis codes were collected from claims data if the hospitalization occurred at a non-KPSC facility. The date and cause of death were determined from the California State Death Statistical Master Files and the Multiple Cause of Death Files for all subjects, regardless of health plan membership status.

Risk Calculator and Risk-Based Decision Tool

 We predicted CVD risk for all patients using an updated version of a previously developed and described risk calculator.

 The models used in the risk calculator were in the form of a proportional hazards survival model and were independently derived using publicly available data sets from the following studies: Framingham, Framingham Offspring, Atherosclerosis Risk in Communities Study (ARIC), and Cardiovascular Health Study (CHS).

 The risk calculator was designed to determine the 5-year risk of fatal and nonfatal myocardial infarction, and, as a separate estimate, fatal and nonfatal stroke. Multiple statistical imputation is used to handle missing data when possible. Confidence guardrails prevent presentation of risk information when the data for a particular patient are insufficient for imputation and risk calculation.

The risk-based decision tool is the set of risk calculators and additional models that allow computations of health outcomes and the effects of interventions on risk at a person-specific level. This tool has a software application that utilizes EHR data to create an interactive visual display that quantifies health risks and the benefits of different healthcare interventions for individual patients.

 The risk-based decision tool interface can be used to assist collaborative provider-patient decision making in place of standard clinical practice guidelines. The risk-based decision tool sets a threshold of expected benefits and, for each patient, calculates a Health Impact score—the expected gain in 5-year quality-adjusted life-years, depending on whether or not they follow a treatment recommendation (see 

] for details). Information on the expected risks and the outcomes associated with possible therapies is presented to patients and physicians for individualized decision making.

 In a population management context, the risk-based decision tool can be used to prioritize patients by the potential benefit of actionable interventions and rank interventions in order of expected benefit. Some of the models for intervention impact on risk were derived independently, and others were derived from risk equations used in the Archimedes Model, which allows computations of health outcomes and effects of interventions on a person-specific level.

 The model is validated on a continuous basis to remain relevant and comply with new medical guidelines.

All of these models were derived from and validated against data from multiple studies and public sources, but not data from Kaiser Permanente.

Identification of Patients for Medication Therapy

Traditional guidelines and the risk-based decision tool were each used to identify patients for medication therapy. KPSC traditional guidelines were based on the JNC-7 blood pressure guideline and the Adult Treatment Panel III (ATP III) cholesterol guideline, which incorporated use of the Framingham risk calculator for borderline cholesterol values. These traditional guidelines were converted into an automated algorithm that could be applied to the entire population. For the risk-based decision tool, patients were chosen using a Health Impact score that resulted in the same number of patients being selected for treatment as by traditional guidelines (the “same-size” population threshold).

Our focus for this evaluation was therapy with statins (HMG-Co-A reductase inhibitors) and antihypertensives for CVD event prevention. For both traditional clinical practice guidelines and the risk-based decision tool, a patient was not identified for a medication therapy if they were already on it or there was a history of allergy or prior adverse events based on EHR data.

For the first part of this study, we determined the proportion of patients for whom a risk calculation was possible. The calculation was not performed if the data required by the calculator were not present in the EHR and could not be imputed. For comparisons with actual 3-year outcomes, we linearly interpolated the risk calculations from the risk-based decision tool from 5 to 3 years. We used the area under the receiver operating characteristic curve (AROC) to assess how well the risk calculator sorted those who had an outcome from those who did not (ie, discrimination accuracy). We used graphical methods, as described by Pepe et al,

 to assess the risk calculator predictiveness across a range of risk percentiles, which provided a visual calibration and goodness of fit assessment. We used the Hosmer-Lemeshow statistic to test risk calculator model goodness of fit.

For the second part of the study, we compared the characteristics of the “same-sized” populations identified for medication therapy by the risk-based decision tool and traditional guidelines using the 

 test. Any of 1 or more antihypertensive medications were counted as a single medication recommendation. We determined the outcomes (3-year events/1000 persons) of patients who were identified for possible treatment or not based on the risk-based decision tool, with those identified for treatment or not based on traditional guidelines. We also determined the outcomes in “indeterminate” patients for whom the risk-based decision tool did not calculate a Health Impact score due to insufficient data to calculate risk or to support a treatment recommendation. These indeterminate cases also included patients with medical conditions not modeled (about 5% of all patients): advanced heart failure, late-stage kidney disease, and a CVD event that occurred under age 45. When implementing traditional guidelines, if there was insufficient data to apply the guidelines, patients were assigned to “no treatment” by default since traditional guidelines only specified when to treat, and this approach was consistent with population-based implementation of guidelines.

The study cohort consisted of 1,506,109 patients who met inclusion criteria. 

 describes selected characteristics of the study cohort. A risk calculation was possible for 1,082,158 patients (72% of the cohort). The mean (SD) follow-up time for patients in which a calculation was possible was 2.61 (0.86) years.

The risk-based decision tool sorted patients fairly well by their propensity to have a myocardial infarction or stroke: the AROC was 0.774 (95% CI, 0.770-0.779) for myocardial infarction, and 0.805 (95% CI, 0.801-0.808) for stroke. The Hosmer-Lemeshow fit statistic indicated good fit (

 shows the predictiveness curves of the risk calculator for fatal and nonfatal myocardial infarction (graph A) and stroke (graph B) for 20 different ranked risk groups. The predicted rates were slightly higher than observed rates for myocardial infarctions in the top 15th risk percentile. The predicted rates for strokes were slightly lower than the observed rates in several high-risk groups, although not the highest-risk group.

Treatment Stratification by the Risk-Based Decision Tool and Traditional Guidelines

Table 1 shows the characteristics of 3 subgroups identified for possible new medication treatment by: 1) the risk-based decision tool but not by traditional guidelines, 2) traditional guidelines but not by the risk-based decision tool, and 3) both. For almost all characteristics, there were statistically significant differences (

 <.0001) between the groups. The most notable difference was the older age of patients identified for possible treatment by the risk-based decision tool alone versus those identified by traditional guidelines alone. Using the “same size” population identified for treatment by both approaches, more than 50% of patients to whom medications were recommended by the risk-based guidelines only were 70 years or older. The corresponding percentage was less than 5% for those to whom medications were recommended by traditional guidelines only. Compared with traditional guidelines, the risk-based decision tool also identified proportionally more men, more patients with high-normal blood pressures, and more smokers.

 shows the percentage of the total cohort eligible for new medication treatment based on the risk-based decision tool, as well as the percentage based on traditional guidelines. The risk-based decision tool made statin recommendations in 34.85% of patients (8.73% for, 26.12% against), and antihypertensive recommendations in 18.20% of patients (4.82% for, 13.38% against). Traditional guidelines made statin recommendations in 9.69% of patients and antihypertensive recommendations in 4.94% of patients, with all others not recommended treatment by default. The CVD event rates per 1000 members among those to whom statins were recommended were similar using either approach, and somewhat higher among those to whom antihypertensives were recommended using the risk-based decision tool (72.0 vs 63.1). The event rates per 1000 members identified for treatment by both the risk-based decision tool and traditional guidelines were greater than the event rates identified by either approach alone.

Although there was overlap in the patients identified for treatment by the risk-based decision tool and traditional guidelines, the risk-based decision tool identified a number of patients who were at high risk, but who were not identified for treatment by traditional guidelines. For statins, the risk-based decision tool identified an additional 3.80% of the population (beyond the 9.69% identified by traditional guidelines) who were not identified for treatment by traditional guidelines but who had a CVD event rate (45.3 events/1000 patients) that was not much less than that of patients to whom treatment was recommended by traditional guidelines (50.1/1000 patients). Similarly, for antihypertensives, the risk-based decision tool identified an additional 3.04% of the population who were not identified for treatment by traditional guidelines, but who had a similar CVD event rate (65.9/1000 patients) to those for whom treatment was recommended by traditional guidelines (63.1/1000 patients).

The risk-based decision tool also made recommendations against therapy for a number of patients for whom treatment was recommended by traditional guidelines. For statins, the risk-based decision tool made recommendations against treatment for 3.80% of the population. These patients had a CVD event rate that was relatively low (15.5/1000 patients), and virtually identical to the risk among patients to whom treatment was not recommended by traditional guidelines (14.3/1000 patients). For antihypertensives, the risk-based decision tool made recommendations against treatment in 2.51% of the population; this group’s CVD event rate (19.9/1000 patients) was actually lower than that of patients to whom treatment was not recommended by traditional guidelines (22.7/1000 patients).

We undertook this study to evaluate an alternative way to implement guidelines using an automated risk calculator and decision support tool that captures EHR data to calculate patients’ risk of CVD outcomes, and identifies the impact of potential treatments. Our objectives were to determine its applicability, the usefulness of its predictions, and how patient identification for possible treatment compares with traditional guidelines. In doing so, we hoped to assess the implications of using a risk-based decision tool for both individual and population care.

The risk calculator was widely applicable to the health plan’s general adult population (72% had a risk estimate given 1 year of clinical data) and demonstrated fairly good discrimination and predictiveness. Our results indicate that when tuned to treatment in the same number of patients as traditional guidelines, the risk-based decision tool identifies a group of individuals who are likely to benefit from treatment but who are missed by current guidelines, even when those guidelines include Framingham risk. It also identifies a group of individuals for whom treatment is recommended by current guidelines, but have risks that are sufficiently low enough that they may safely be left untreated.

In a similar type of comparison, the newer ACC/AHA guideline for statin therapy based on a risk calculator was compared with ATP III guidelines with its cholesterol targets supplemented by Framingham risk in an existing community-based cohort.

 This study compared a different risk-based calculator (ACC/AHA) with essentially the same traditional guidelines as used by KPSC. The findings mirror ours in that the calculator had “greater accuracy and efficiency in identifying increased risk of incident CVD and subclinical coronary artery disease”

—similarly, more higher-risk patients were identified. Additionally of note, the risk-based approach of the ACC/AHA guidelines was deemed, in a simulation study, to have an acceptable cost-effectiveness profile.

 In this simulation study, the cost-effectiveness of the ACC/AHA risk-based approach varied with risk cut points.

 In our study, the Health Impact score was used to introduce the variation of risk to the population affected in place of pure risks.

A significant problem facing all risk-based calculators and decision support tools that use EHR data is that they might not be able to perform the desired calculations, either because of missing data or limitations in patient groups to which they are applicable (eg, late-stage kidney disease). Some of the patients who are excluded from risk models are at higher risk due to relatively infrequent medical conditions; in these situations, decisions are left to the discretion of the treating physician. Hopefully, future risk calculators will address this restricted scope of application. Risk-based decision tools also suffer from the same lack of evidence about treatment as do traditional guidelines. This is especially evident when considering that 72% of patients had a risk calculation, but among patients potentially eligible for a medication, 65% (statins) and 82% (antihypertensives) could not be definitively classified into treatment or no treatment groups.

Our study has several limitations, including that the KPSC system may differ in patient outcomes from other health delivery models. Our determination of who would be recommended for treatment was based solely on EHR data and automated guideline implementation rather than clinical practice. Use in practice may increase or reduce the identification of high-risk patients and the differences between approaches. Not all potentially relevant characteristics of a patient, or clinical actions, are captured in the EHR. Additionally, prediction accuracy was not adjusted for subsequent changes in risk factors or clinical treatments. Our results may vary somewhat over shorter or longer follow-up time frames. The risk-based decision tool identified some patients at increased risk who were not identified by traditional guidelines; however, we did not assess the separate contributions of imputation, calculator variables, and estimator accuracy in determining this difference in identification.

Articles

WEB EXCLUSIVE

The authors evaluated the clinical applicability, accuracy, and implications of using an automated risk calculator and risk-based decision tool in an integrated health system.

Potential of Risk-Based Population Guidelines to Reduce Cardiovascular Risk in a Large Integrated Health System

Using Behavioral Risk Factor Surveillance System (BRFSS) telephone survey data, we evaluated whether individuals who reported access to enhanced primary care features experienced improved quality of care.

We assessed a sample of 16,731 Massachusetts residents contacted by telephone using random-digit dialing, to complete the BRFSS in 2008. A randomized subset of 5693 respondents received an additional 5-question enhanced primary care assessment module. We defined an “enhanced” primary care group as those who reported having a regular, personal provider and responded that their provider “always” or “almost always” 1) had knowledge of their medical history, 2) gave them an appointment right away when necessary, 3) was up-to-date in their knowledge of the patient’s specialist care, and 4) asked them about all of their medications. Multivariable logistic regression was used to compare the “enhanced” versus “usual care” groups and assess several quality process measures.

Nearly one-third of participants endorsed having indicators of enhanced care, and this group reported significantly higher rates of diabetes process measures (56% vs 38%), cholesterol screening (89% vs 81%), influenza vaccination (57% vs 49%), pneumonia vaccination (51% vs 43%), and lower cost and/or access barriers to care (22% vs 33%).

Enhanced primary care was associated with improved self-reported quality outcomes in a statewide telephone survey. A brief, 5-question module provided a novel population measure of access to enhanced primary care. This is a scalable option for other states hoping to characterize their own primary care improvement efforts through the patient-centered medical home model.

	We evaluated Behavioral Risk Factor Surveillance System telephone survey data from Massachusetts residents that included a patient-centered medical home (PCMH) assessment module. 

		Patient-reported indicators of the PCMH correlated with improved quality process measures in diabetes, cholesterol screening, vaccination, and access to care.   

		A brief, 5-question module provided a novel population measure of access to enhanced primary care. 

		This may be a scalable option for other states hoping to characterize their own primary care usage and utility as health insurance expansion continues nationally.

Massachusetts was one of the leaders in insurance expansion with the 2006 Massachusetts Mandated Health Insurance Law.

 Recent reports have cited positive outcomes from the Massachusetts reform effort, such as increases in the total number of Massachusetts residents insured, growing rates of employer-based insurance coverage, strong public and physician support, reductions in disparities in access based on race or ethnicity, increasing amounts of preventive services, fewer low-income or chronically ill adults reporting unmet medical needs,

 and even lower mortality rate among those getting insurance.

 However, insurance reform only addresses a small part of the deficiencies in the larger healthcare delivery system. Patient-centered medical homes (PCMHs) are generating significant state and national attention as one solution in what is generally a costly and ineffective system plagued with disparities in health outcomes and significant barriers to high-quality, integrated medical care.

Most PCMH assessment surveys are based on individual practice or demonstration reports, and such surveys have been at the forefront of reports on the success of the PCMH model.

 Although such demonstrations are a critical part of the evaluation process of new care models, they are context-specific and take time to generate evaluable data. Another approach to evaluate the effect of the PCMH-associated components is to use publically available survey data and parsimonious definitions of what factors compose a PCMH. One such evaluation was performed on the 2007 to 2008 National Survey of Children’s Health, and it identified improvements in 6 of 10 quality measures among children in a PCMH, as defined by a 5-component measure.

 Such patient care experience surveys are often unavailable, however, and to date, there is no state-level evidence on the availability of PCMH components in primary care practices or their association with quality of care.

The Behavioral Risk Factor Surveillance System (BRFSS) was developed by the CDC as a random-digit-dial landline telephone survey.

 The CDC collects data monthly, using this survey in all states and territories of the United States. In order to evaluate the availability of primary care providers, PCMH components in primary care, and sociodemographic predictors of the preceding variables in Massachusetts, a 5-question module was added to the BRFSS for a subset of responders. Using the data provided by this module, we then evaluated the association of PCMH components with quality of care.

In preparation for statewide initiatives in medical homes, one of the authors, who was a member of the Massachusetts Executive Office of Health and Human Services, worked with experts in PCMHs to develop a 5-question short form survey module (

). The questions were adapted from work on the Primary Care Assessment Survey (PCAS) by Safran et al.

 Officials from the Massachusetts Department of Public Health, which administers the survey in the state, gave permission for the module to be piloted in a subset of participants in the 2009 Massachusetts BRFSS (MA-BRFSS) survey. 

The MA-BRFSS survey was completed in 3 waves, and the experimental module was used in the second wave. We used descriptive statistics to characterize the subset of responders and a 

 test to look for statistical variance between the experimental module—wave 2&mdash;and the other survey waves. We then evaluated whether the responder had a primary care physician according to their survey results and used a multivariable logistic regression to test for associated subject characteristics between the subset of responders with a primary care physician and the subset without.

To determine whether having an enhanced primary care experience was related to completion of healthcare process measures, we divided responders into 2 groups. One group, termed the “enhanced primary care” group, endorsed having a primary care physician and responded that their provider “always” or “almost always” 1) had a knowledge of their medical history, 2) gave them an appointment right away when necessary, 3) was up-to-date in their knowledge of the patient’s specialist care, and 4) asked them about all of their medications. A multivariable logistic regression was used to test for associated subject characteristics between the “enhanced primary care” group and the “usual care” group without these PCMH attributes. A 

test was used to characterize the 2 groups.

We performed a multivariable logistic regression analysis to assess differences between the enhanced and usual care groups in several quality measures. We selected these indicators because they are common measures of clinical quality in the management of chronic disease and preventive care. Quality measures included diabetes annual care processes (which includes an annual visit, eye exam, foot exam, cholesterol screening, and hemoglobin A1C testing); lipid screenings for patients with cardiovascular disease, diabetes, and hyperlipidemia; annual influenza vaccination in eligible patients (all patients aged ≥6 months)

; and pneumonia vaccinations in eligible patients (all adults ≥65 years and adults aged 19-64 years who smoke, have chronic pulmonary disease, or are immunodeficient).

An additional binary quality measure assessing cost barriers and access to care was created based on the responses to the following: “Was there a time during the last 12 months when you needed to see a doctor but could not due to the cost?” and “About how long has it been since you last visited a doctor for a routine checkup?” Subjects who responded that they did forgo a visit to the doctor due to cost in the past year or that they had not had a checkup within 1 year were given a positive value on the “cost/access” variable; subjects who did not meet one or both criteria were not given a positive value. The regressions were controlled for age, sex, income, education, and insurance status as covariates.

This study was reviewed by the Institutional Review Board at both the Massachusetts Department of Public Health and the CDC, and was considered exempt due to being a low-risk study without any patient identifiers.

Of the 16,731 respondents to the MA-BRFSS in 2009, 5693 respondents were randomized to receive the short form survey module. 

 summarizes subject characteristics for the respondents to the short form survey module (wave 2) compared with other respondents (waves 1 and 3). There were no significant demographic differences between the 2 groups, confirming that the random sampling for the survey module was successful.

Among participants, 29.8% answered “always” or “almost always” to all 5 PCMH module questions, indicating that they had an enhanced primary care experience. Subjects who were aged 50 to 74 years (

 <.001) were more likely to report having an enhanced primary care experience; subjects who were either uninsured (

 <.001) or were Asian, Native Hawaiian or Pacific Islander, American-Indian, multiracial, or selected “other” for their racial affiliation (

A multivariate logistic regression adjusted for age, sex, income, education, and insurance status found significant differences between the enhanced and usual primary care groups in all quality measures tested. Subjects receiving enhanced care were more likely to receive diabetes care

 = .002). Subjects in this enhanced group were also less likely to report experiencing barriers to access to care, measured by whether they reported an instance in the past 12 months where they needed to see a doctor but could not due to cost and by whether more than 1 year had elapsed since their last checkup (

Almost one-third of residents reported having access to a primary care practice with enhanced primary care attributes. Importantly, having enhanced primary care was associated with better performance on preventive, access, and chronic disease measures. Our findings have favorable implications for the wider adoption of the PCMH model—the main mode of primary care improvement and transformation nationwide in the United States. Our findings also contribute to a growing body of research showing that the PCMH-enabled primary care transformation may result in modest increases in quality measures early on by improving care processes, particularly in the area of prevention.

For example, a recent prospective multi-payer study in Hudson Valley, New York, found that PCMH practices improved significantly more (from 1% to 9%) on 4 of 10 quality measures than practices with electronic health records and paper records.

 In a single private payer demonstration project in New Jersey in 2011 with 8 medical home practices, mammography and nephropathy screening increased, although changes in 7 other HEDIS measures were not significant.

 A separate randomized controlled trial in New York evaluating PCMH transformation over 2008 to 2010 among 32 practices (18 intervention practices) found improvements in 2 of 11 quality indicators: hypertensive blood pressure control and breast cancer screening.

Notably, other PCMH studies have not shown significant impact on quality measures,

 enforcing the need for additional research in this area to understand why some initiatives are associated with quality improvement while others are not. Understanding how the PCMH model can impact quality of care is critical for optimal primary care design, especially given the enormous and rapid uptake of the PCMH model. From 2009 to 2014, the number of patients covered by PCMH initiatives increased from nearly 5 million to almost 21 million across more than 100 state and federal initiatives.

In addition, we need more efficient and effective methods to evaluate primary care and access to care, with a particular focus on the patient experience as medicine becomes increasingly patient-centered. A PCMH transformation study evaluating patient experiences in a pilot practice in eastern Massachusetts with Lean enhancement and payment reform found that patient experience was sustained or improved across PCMH domains including the personal physician and communication domain.

 Our study builds on this finding, utilizing a simple tool for evaluating access to enhanced primary care directly from patients in a large, state-based population survey of patient respondents. This 5-question survey module was a valuable addition to the BRFSS and provided a new population measure of access to enhanced primary care. This survey of critical primary care functions may be a replicable and scalable strategy that other states and healthcare systems can use to characterize their own primary care transformation efforts and reach. 

While it may be that the enhanced primary care components evaluated are inseparable from what might be simply termed high-quality traditional primary care, such a distinction may be irrelevant. The PCMH model provides structure and script for the delivery of better primary care. This is invaluable, as it allows struggling practices to implement a comprehensive plan for improvement rather than disparate quality improvement efforts that may fail due to their lack of integration into the central practice philosophy and administration. However, there is still great variability in how PCMHs are implemented from site to site,

 and evaluations at a population level are important. In addition, evaluations within a single system may reflect characteristics of that system only and may not be generalizable to PCMHs implemented in a broad array of practice types.

In spite of the demonstrable increases in quality and access to healthcare brought about by the Massachusetts healthcare reform law of 2006, healthcare costs continue to increase steadily. Reform efforts have not contributed to any additional rise in healthcare costs, but neither have they led to a significant decrease in overall costs.

 One of the largest drivers of healthcare expenditure is preventable hospitalizations and emergency department visits.

 There is evidence to suggest that these expenditures can be mitigated by high-quality, long-term, prevention-based primary care focused on managing the chronic conditions that often result in hospitalization when mis- or undermanaged.

 Delivering such care is the goal of the PCMH, although it remains to be seen whether the continued expansion of this model results in the expected and sustained cost reductions.

This study is a cross-sectional pilot study that includes only 1 year of data, although the sample is large and representative of the population of Massachusetts. A majority of the differences in quality outcomes in the enhanced primary care group were modest; however, the differences in diabetes care were quite large and may have been even larger had we not used a bundled care process measure, which can result in lower scores.

 Furthermore, the 5-question module used in this study was validated as a measure of primary care quality,

 but not as an indicator of whether a patient was treated by a PCMH. Thus, it is an indirect measurement of medical home attributes. As such, we cannot say with certainty that these differences in quality are a direct result of receiving enhanced care or are indicative of practices that are themselves medical homes, and we had no direct measure of which patients were receiving their care in a PCMH. The study is also limited by its reliance on self-report for all measures; however, this direct feedback from patients provides an important window into the patient perspective and experience. 

Evaluation of the PCMH and how it affects patient experience and quality measures on a population level is critical to monitor the progress of reform measures, such as the ongoing large-scale adoption of medical homes. Other major health reforms, such as increased insurance coverage, are being deployed on a state level, and we need new population-level tools to evaluate these initiatives. The pilot implementation of the 5-question enhanced primary care module in the Massachusetts BRFSS provides a simple, low-cost model, which other states may use to gather this crucial experience and quality data. We found that higher levels of enhanced primary care attributes at practices were associated with substantially improved quality for an array of measures, suggesting that this benefit may be realized even at the population level.

The authors would like to thank Bruce Cohen, PhD, Director of Research and Epidemiology and Bureau of Health Statistics, Research and Evaluation at the Massachusetts Department of Public Health; Liane Tinsley, MPH, for assistance with implementing the PCMH module into the BRFSS survey; and also Jennifer Kincheloe, PhD, MPH.

 Brigham and Women’s Hospital (AB, AWB, JSW, DWB), Boston, MA; Department of Health Care Policy, Harvard Medical School (AB, JSW), Boston, MA; Ariadne Labs (AB), Boston, MA; Veterans Affairs Boston Healthcare System (AWB), Boston, MA; Pennsylvania State University College of Medicine (SC), Hershey, PA.

 Dr Bitton is a board member of Health Leads and is a senior advisor for the Center for Medicare & Medicaid Innovation. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (AB, JSW, DWB); acquisition of data (AB, JSW); analysis and interpretation of data (AB, AWB. JSW, SC, DWB); drafting of the manuscript (AB, AWB, JSW, SC, DWB); critical revision of the manuscript for important intellectual content (AB, AWB, SC, JSW, DWB); statistical analysis (AB, SC); obtaining funding (AB, DWB); administrative, technical, or logistic support (AB, DWB, SC, AWB); and supervision (AB, JSW, DWB).

 Asaf Bitton, MD, MPH, Assistant Professor of Medicine and Health Care Policy, Division of General Medicine and Ariadne Labs, Brigham and Women’s Hospital, 1620 Tremont St, 3rd Fl, Rm 3-002P, Boston, MA 02120. E-mail: abitton@partners.org.

In a statewide telephone survey, patient-reported indicators of the patient-centered medical home correlated with improved process measures in diabetes, cholesterol screening, vaccination, and care access.  

Enhanced Primary Care and Impact on Quality of Care in Massachusetts

A 21-gene test that predicts recurrence risk among women with hormone receptor positive (HR+), localized breast cancer was nationally recommended in 2007, but we know little about its subsequent impact. We evaluated: a) patient characteristics associated with test use, b) correlations between Recurrence Score (RS) and chemotherapy, and c) whether test introduction was associated with a reduction in chemotherapy use.

The Kaiser Permanente Northern California tumor registry and electronic health records from 2005 to 2012 were used to identify HR+, human epidermal growth factor receptor 2 negative, node-negative cancers. Analyses used logistic regression with propensity score matching and 2-level logistic regression.

Of the 7004 patients who met guidelines for testing, 22% were tested and 26% had chemotherapy. Test use was more likely in younger women (for ages 40-49 years vs 50-64 years: odds ratio [OR], 1.22; 95% CI, 1.04-1.44), in women with tumors sized 1.0 to 2.0 cm versus >2 cm (OR, 1.20; 95% CI, 1.03-1.40), and in women from higher-income neighborhoods (for each $10,000 increase in area median income: OR, 1.05; 95% CI, 1.03-1.07). Among patients with low RS, 8% had chemotherapy versus 72% among patients with high RS (

 <.01). In propensity score-matched analyses, testing was associated with an absolute reduction of 6.2% in the proportion of women receiving chemotherapy (95% CI, 2.9%-9.5%); the 2-level model showed a similar but nonsignificant (

The 21-gene test is used in a minority of eligible patients in this integrated plan. Its use appears to be associated with a modest decrease in overall chemotherapy use.

	In this retrospective cohort study, we evaluated the use of a multigene test that predicts recurrence risk in localized breast cancer, and associated trends in chemotherapy use in eligible patients.

		The multigene test was used in fewer than 1 in 3 eligible patients, with use plateauing in 2010 through 2012.

		Recurrence risk scores on the test were highly, but not perfectly, correlated with the use of chemotherapy. 

		Introduction of the test was associated with a modest decrease in overall chemotherapy use.

Genetic testing of tumor cells has the potential to revolutionize the care of patients with breast cancer and to accelerate the benefits of personalized medicine.

 Several studies have observed that these tests have been incorporated into clinical practice and seem to influence chemotherapy decisions.

 Recent US studies of claims data and the National Cancer Institute’s Surveillance Epidemiology and End Results (SEER) data found that only 20% to 30% of eligible women were being tested, with the claims data study observing that reimbursement by insurers has increased slowly.

 It is unknown whether acceptance and use of this test have been more complete in integrated, fully capitated systems where the costs of care are covered and decisions are less likely to be affected by financial incentives for or against test or chemotherapy use. These tests are potentially costly, and may be marketed directly to patients, as well as physicians.

 As genetic testing in cancer grows more common, healthcare systems need to systematically evaluate how consistently such tests are being used, and how much incremental benefit they add to baseline clinical practices.

Breast cancer genetic testing provides a useful paradigm for evaluating the impact of such tests at a population level. Decisions about the use of adjuvant chemotherapy, for women with early-stage breast tumors that are estrogen or progesterone receptor positive (ER/PR+), can be especially difficult, as most will never experience a recurrence, even without adjuvant chemotherapy. Currently, several gene-expression profiling tests are being marketed to clinicians and patients as tools to enhance the accuracy of predicting recurrence risk and the likelihood of realizing benefit from adjuvant chemotherapy. The 21-gene Oncotype DX (Genomic Health, Inc, Redwood City, California) breast cancer assay has been validated in clinical trials to predict risk of distant recurrence in patients with early-stage, node-negative, ER/PR+, human epidermal growth factor receptor 2 negative (HER2—) cancers.

 Guidelines for incorporating Oncotype DX testing into treatment decisions were published in 2007

; however, little is known about how this test and other genomic tests are being incorporated into real-world oncology practice and how they affect patterns of care.

This study’s overall objective was to assess how the 21-gene test is being used among patients with early-stage breast cancer in a large integrated health delivery system. Among patients who met current guidelines for use of this test, our aims were to: a) compare the demographic and clinical characteristics of patients who had the test with those who did not; b) describe chemotherapy use among women with test results that indicate low, intermediate, and high risk of recurrence; and c) evaluate whether the introduction of the test was associated with a change in chemotherapy use.

Kaiser Permanente Northern California (KPNC) is a nonprofit integrated healthcare delivery system that currently provides care to more than 3.9 million members. Within KPNC, essentially all primary and specialty care, and the vast majority of emergency and hospital care, is delivered by providers working within a single care system for patients of a single health plan.

Identification of Breast Cancers and Cancer Characteristics

The KPNC tumor registry—a contributor to the SEER program of cancer registries&mdash;was used to identify all female KPNC members diagnosed with invasive, nonmetastatic, incident breast cancer between September 1, 2005 (when significant use of Oncotype DX began at KPNC), and June 30, 2012. A member’s first, newly diagnosed breast cancer during this period was included. The tumor registry includes patient age, sex, diagnosis date, tumor size, node involvement, ER/PR status, stage, and initial chemotherapy treatment. HER2 status was determined using the results of Immunohistochemistry and Fluorescence In Situ Hybridization tests.

In 2007, both the National Comprehensive Cancer Network (NCCN) and the American Society of Clinical Oncology included Oncotype DX testing in their guidelines.

 Following NCCN guidelines, we selected women for whom Oncotype DX was to be considered: those with ER/PR+, HER2—, stage I and stage II breast cancers having primary tumors ≥0.51 cm with either no lymph node involvement or only ≤2 mm axillary node micro-metastases.

 The Oncotype DX assay analyzes the expression of 21 genes to provide a Recurrence Score (RS) corresponding to the risk of distant recurrence at 10 years among tamoxifen-treated patients not treated with chemotherapy.

 The RS is classified into 3 categories based on likelihood of distant recurrence: low risk (RS <18), intermediate risk (RS 18-30), and high risk (RS ≥31). Low RS has been shown to predict little benefit from chemotherapy, whereas high RS predicts greater benefit.

 The routine approach in this medical group was to follow NCCN recommendations; thus, other genetic tests for breast cancer were not commonly used.

Due to the timing of SEER registry reporting requirements, there is some underascertainment of the chemotherapy treatment status in the tumor registry. As part of an ongoing prospective study of newly diagnosed cases of breast cancer,

 a subset of cases were reviewed to validate the chemotherapy treatment status. Of 7004 eligible cancers, 62% were reviewed. Of 1071 patients validated to have used chemotherapy, 86.18% were correctly classified as such in the registry, and of 3240 patients validated not to have used chemotherapy, 99.97% were correctly classified in the registry. The chemotherapy status used was the validated one, if it existed; otherwise, the registry status was used.

Patients were assigned to a census block group (defined by the US 2010 Census) based on their home address at the time of cancer diagnosis. Block group income and education were based on the 2006 to 2010 American Community Survey.

 We used data from the year before the cancer diagnosis to create a modified Deyo version of the Charlson comorbidity index.

 From administrative databases, we extracted (from the year before the cancer diagnosis) the following additional variables for each patient for use in the propensity score matching: primary medical center used for care, number of clinic visits and hospital days, and their associated cost.

To identify the predictors of receiving 21-gene testing, we used logistic regression, in which receipt of the test was the dependent variable. Independent variables were calendar year of cancer diagnosis (as a categorical variable), age group (5 categories: aged <40, 40 to <50, 50 to <65, 65 to <75, and ≥75 years), race/ethnicity (Asian, black, white Hispanic, white non-Hispanic, and other/unknown), tumor size (3 categories: 0.5 cm to ≤1.0 cm, >1.0 cm to ≤2.0 cm, and >2.0 cm), comorbidity (3 categories: 0, 1-2, and ≥3 comorbidities), census block group median income, and proportion of adults in the block group with less than a high school degree.

To assess the potential impact of 21-gene testing on receipt of chemotherapy, we used 2 different analytic approaches. The first approach directly evaluated whether women who received the test were more or less likely to receive chemotherapy compared with women who were not tested. We ran a logistic regression in which the dependent variable was whether the woman was tested, and the independent variables were the same as those listed above, plus the following (to increase further the similarity of the matched cohorts): primary medical center for care, patient’s age-squared (for potential nonlinear age effects), costs of clinic and hospital services, as well as the number of clinic visits and hospital days in the year before cancer diagnosis. Model calibration and discrimination were good (C statistic = 0.81; Hosmer-Lemeshow goodness of fit test: 

= .32, with nonsignificance reflecting adequate fit). The resulting predicted probability of receiving the test was the patient’s propensity score.

We selected patients who received the 21-gene test and matched them 1-to-1 to patients who were not tested. Matching was performed using the Mahalanobis metric matching within calipers, defined as one-fourth of the standard deviation of the logit of the propensity score.

 Using this methodology, 93% (n = 1462) of tested women were matched to a nontested woman, and after matching, there were no significant differences between the cohorts with regard to the variables used in the propensity score calculation. However, women in the matched cohorts were younger and had fewer comorbidities than the general pool of women from whom they were drawn. This reflects the fact that among the entire cohort of patients with cancer, tested women tended to be different from nontested women.

Using the propensity-matched samples, we calculated the percent of women in each sample receiving chemotherapy, and the corresponding ratio of the odds of receiving chemotherapy among women tested, to the odds of receiving chemotherapy among matched women not tested. As a sensitivity analysis, the matched analysis was repeated among the subset of patients with validated chemotherapy use.

The second analytical approach was to treat the overall percent of women who received testing as a predictive variable for receiving chemotherapy. This approach is similar to “ecological” regression, wherein aggregates are used either in place of, or in addition to, individual-level predictors.

 Unlike an interrupted time-series analysis in which differences before and after some change in practice are assessed, this approach does not require us to arbitrarily define time periods as “before” or “after” the introduction of the test, given that testing was phased in over time. We started with an analytic data set with 1 record per woman. For each calendar year and age group, we calculated the percent of women who received the test, and added this variable to the analytic data set. Thus, a woman diagnosed with cancer in 2011 who was aged 50 to <65 years, had a variable added to her record that reflected the percent of women in her age group in 2011 who were tested. Using these patient-level records, we ran a logistic regression in which receipt of chemotherapy was the dependent variable, and the independent variable of interest was the percent of women in the strata who were tested. The woman’s own test status is not included in the model. Other independent variables included in the model were the same as those used in the model identifying predictors of testing. The results of this model were then used to predict the percent of women receiving chemotherapy, assuming 1 of 2 scenarios: 1) 0% of women in her age group and year were tested, and 2) 30% were tested (which was about the maximum percent of women who received the 21-gene test in any given year).

The first approach has been called an “individual-level analysis” and the second approach a “2-level analysis.”

 Each of these approaches has distinct advantages and disadvantages. The individual-level analysis has the advantage of directly measuring the relationship between the individual’s use of the test and their use of chemotherapy, but does not account for certain aspects associated with self-selection of testing. For example, women with a strong predilection for or against chemotherapy may choose not to be tested—a problem similar to “confounding by indication.” The 2-level “ecologic” analysis factors out some of the potential unmeasured confounders related to which women choose to be tested.

 However, that approach is prone to bias if there are other unmeasured factors that may have increased or decreased use of chemotherapy at the same time as the increase in the use of testing.

We identified 7004 women diagnosed with cancers meeting our inclusion criteria, who met guidelines for 21-gene testing (

). The majority of women were under 65 years of age (57%) and 71% were non-Hispanic white. Three-fourths of cancers were stage I, and 70% of tumors were >1.0 cm in size. Overall, 21-gene testing was performed for 22% of women. In adjusted analyses, compared with women aged 50 to <65 years, women aged 40 to <50 years were more likely to be tested (OR, 1.22; 95% CI, 1.04-1.44), whereas women aged 65 to <75 years and ≥75 years were less likely to be tested (OR, 0.42; 95% CI, 0.36 to 0.49; and OR, 0.04; 95% CI, 0.02-0.06, respectively) (

). Compared with women with tumors >2.0 cm, women with tumors 0.5 cm to ≤1.0 cm were less likely to be tested (OR, 0.51; 95% CI, 0.42-0.61), whereas women with tumors >1.0 to ≤2.0 cm were more likely to be tested (OR, 1.20; 95% CI, 1.03-1.40). Each $10,000 increase in block group median income was associated with increased odds of testing (OR, 1.05; 95% CI, 1.03-1.07).

Among women with 21-gene testing, 52%, 39%, and 9% had low-risk, intermediate-risk, and high-risk RS values, respectively (

). Among women who had the test, a slightly higher percentage (26%) received chemotherapy compared with those who did not (22%; 

 <.01). Among women with low-, intermediate-, and high-risk RS, 8%, 40%, and 72%, respectively, received chemotherapy.

Between 2005 and 2012, the percent of eligible women receiving 21-gene testing rose from 8% to more than 25%, while the percent of women receiving chemotherapy decreased modestly from 26% to 22% (

). Younger women were much more likely to receive chemotherapy, and the 2 age groups with the most women receiving chemotherapy had the most pronounced downward trends in chemotherapy use from 2005 to 2012 (from 59% to 47% for those aged 40 to <50 years, and from 35% to 24% for those aged 50 to <65 years) (

In analyses with individual-level propensity score matching (n = 2924), receipt of the 21-gene testing was associated with decreased odds of chemotherapy (OR, 0.74; 95% CI, 0.63-0.87), corresponding to a reduction in the percent of women receiving chemotherapy from 32.7% to 26.5%, or an absolute reduction of 6.2% (95% CI, 2.9%-9.5%). When including only women with validated chemotherapy treatment status, 21-gene testing was associated with lower odds of chemotherapy (OR, 0.64; 95% CI, 0.53-0.78) than in the primary analysis, corresponding to an absolute reduction of 9.5% (95% CI, 5.3%-13.6%). Among this matched cohort, women who had received Oncotype DX testing and went on to receive chemotherapy were older, and were more likely to have stage I cancer and smaller tumors, than women who had not received Oncotype DX testing and went on to receive chemotherapy.

In the 2-level multivariable (ecological) analysis, each 10% increase in the absolute percent of women tested was associated with a 0.92 decreased odds of chemotherapy, but this result was not statistically significant (95% CI, 0.83-1.03; 

 = .14). The point estimates of this model imply that 26% of women in the study would have received chemotherapy in the absence of any testing, while 23% would receive chemotherapy if 30% of women were tested.

This is among the first large investigations of the use and impact of the 21-gene test for breast cancer recurrence risk in a managed care population. In this integrated system, we found that although rates of use increased over time, only 20% to 25% of patients meeting guidelines received testing. In adjusted analyses, use of testing was differential by age, tumor size, and neighborhood-level income. Patterns of chemotherapy use were generally consistent with test results, with those having a low RS far less likely to have chemotherapy than those with a high RS. When used, the test was associated with a modest reduction in overall chemotherapy use.

The rates of testing we observed were similar to other reports from US populations,

 including those of a large, for-profit oncology network.

 One difference from past studies is that we found no racial/ethnic differences in use of the test. For example, Guth et al found that women treated at municipal hospitals—who were more likely to be of low income and nonwhite&mdash;were less likely to have the 21-gene test than socioeconomically similar women seen in tertiary care settings (3% vs 30%).

 The fact that the test was fully covered by insurance in our setting removed patient-level financial barriers and may have mitigated racial/ethnic variation in test use. However, patients living in higher-income areas were more likely to have the test than those living in lower-income areas.

The other clinical and demographic correlates of nonuse of 21-gene testing that we identified, such as smaller tumor size and older age, have also been observed in other settings.

 For these subgroups, clinicians and/or their patients may have decided that chemotherapy was not indicated, so that testing would not change treatment decisions. Clinicians may also feel that standard clinicopathologic prognostic factors, existing decision tools (eg, Adjuvant!),

 and/or a patient’s health status or preferences are more important in treatment decision making than 21-gene test results.

 However, it has been reported that, when obtained, the 21-gene results can change pre-testing treatment decisions in about 30% to 50% of cases.

 A survey of KPNC oncologists, completed in spring 2015, suggests that among those patients who have the 21-gene test, the results cause changes in chemotherapy decisions in approximately 40%—and that these changes are equally divided among changes away from having chemotherapy and changes toward having chemotherapy (Lieu et al [unpublished data from a survey of 85 KPNC oncologists via mail and e-mail, as part of the overall project that produced this paper]).

This 21-gene test and other multigene tests have been promoted as likely to be cost-effective since test costs are expected to be offset by decreases in chemotherapy use among women with low recurrence risk, and, hence, low predicted benefits of chemotherapy.

 In our matched analysis, we found that testing was associated with a 6% to 10% reduction in chemotherapy use. A recent meta-analysis estimated a somewhat higher percent reduction of 12%.

 Our “ecologic” analysis indicated that an increase in testing in the KPNC setting, from 0% to 30%, resulted in approximately a 3% absolute decrease in the percent of women getting chemotherapy. The 2-level analysis is a more indirect (and conservative) approach, with fewer observations due to the summarized nature of the analysis, and, therefore, has less power. Nevertheless, the direction of the result was the same as that of the propensity score-matched analysis. Regardless of approach, our estimated reductions in chemotherapy were far lower than the 31% reduction in chemotherapy observed in a smaller study of a younger cohort in Ireland.

 The lower reduction in chemotherapy we observed relative to other studies was most likely due to our patient population being older and having a lower baseline percent of women receiving chemotherapy, as well as a lower percentage of eligible women being tested, compared with the study from Ireland.

Our results should not be construed as suggesting that the 21-gene test is being underused, as the study only evaluated chemotherapy in relation to current levels of testing. Our results cannot be extrapolated to project whether, or by how much, chemotherapy use would decrease if the test were used for a higher percentage of patients. The current level of testing may reflect clinicians’ judgments that the nontested patients would not benefit from testing because the decision about chemotherapy is already clear in their cases.

Chemotherapy treatment decisions in our study were not always in accord with the RS. Among women with low RS, 8% received chemotherapy anyway, and 28% of those with high RS did not receive chemotherapy. Similar discordant use patterns were noted in a recent meta-analysis.

 Our data did not enable us to determine the reasons for these conflicting choices; however, based on other reports,

 it seems that discordant use patterns are not unexpected since multigene testing is only 1 factor in complex chemotherapy decisions, and test results are not an absolute mandate for or against chemotherapy. For instance, doctors may sometimes use the test to discourage a low-risk (based on tumor size, histology, grade) or unhealthy patient who wants chemotherapy from having it, or to encourage treatment in a high-risk, healthy patient who does not want it. In the latter situation, a patient may continue to refuse chemotherapy even after receiving a high-risk test result. Genomic testing may increase anxiety and impair decision making or results may be poorly understood.

This study has many strengths, including its fully enumerated managed care population, inclusion of only those cases with clinical indications for 21-gene testing, and ability to relate test results to chemotherapy use. Although these findings from an integrated health plan population in California may not be representative of all practice settings, the testing and treatment patterns we found were remarkably similar to those reported from other settings.

 This suggests that clinical norms, randomized controlled trial evidence, and national recommendations may be more important to physicians’ ordering behavior than the costs of the test or who is covering those costs. That chemotherapy is sometimes discordant from therapy suggested by the RS indicates that patient factors may be as, or more important than the healthcare structure. These hypotheses will need to be tested explicitly in future research across diverse healthcare systems and populations.

Overall, this study suggests that in a large integrated healthcare system, the 21-gene test is used in a minority of eligible patients, but when used, it is leading to clinically appropriate patterns of chemotherapy use. Optimizing the benefit and efficiency of this and other genomic tests for cancer patients will require additional research on the factors that drive test use and subsequent decisions about chemotherapy. 

A multigene test for breast cancer recurrence risk was used in a minority of eligible patients, yet was associated with a decrease in chemotherapy use.

Breast Cancer Multigene Testing Trends and Impact on Chemotherapy Use

To estimate the costs and benefits of over-the-counter (OTC) statins under the 2013 American College of Cardiology/American Heart Association guidelines.

A 10-year cost-effectiveness model using a health system perspective was developed to analyze the impact of making an OTC statin drug available.

We calibrated the model by using nationally representative survey data on statin use and cardiovascular risk, data from clinical studies on the safety and efficacy of statins, and data from a study on consumer decisions to use an OTC statin.

This paper estimates the costs and benefits of over-the-counter (OTC) statins using data on statin use and cardiovascular risk, clinical studies of statin safety and efficacy, and an OTC statin use trial.

May 2016

Potential of Risk-Based Population Guidelines to Reduce Cardiovascular Risk in a Large Integrated Health System

Enhanced Primary Care and Impact on Quality of Care in Massachusetts

Breast Cancer Multigene Testing Trends and Impact on Chemotherapy Use

A Cost-Effectiveness Analysis of Over-the-Counter Statins

Referring Wisely: Orthopedic Referral Guidelines at an Academic Institution

Physician Perceptions of Choosing Wisely and Drivers of Overuse

Greater Potential Cost Savings With Biosimilar Use

Implementing a Hybrid Approach to Select Patients for Care Management: Variations Across Practices

Clinical Interventions Addressing Nonmedical Health Determinants in Medicaid Managed Care

Medicaid Managed Care Penetration and Drug Utilization for Patients With Serious Mental Illness