Medication Adherence and Measures of Health Plan Quality

Seth A. Seabury, PhD; Darius N. Lakdawalla, PhD; J. Samantha Dougherty, PhD; Jeff Sullivan, MS; and Dana P. Goldman, PhD

Measuring the quality of care provided to patients by health plans and providers is an increasingly common element of evaluating and improving healthcare delivery. Efforts to increase value in healthcare often include incorporating measures of quality into provider compensation or plan ratings.1-3 This is reshaping how health plans evaluate themselves, how employers choose plans, and how consumers decide who provides their care. At present, many healthcare organizations participate in quality reporting, whether for internal quality improvement or for public reporting. New payment models are emerging that directly incorporate these quality data into coverage and reimbursement decisions.
Many quality measures have been developed to evaluate the appropriateness of the prescribing, monitoring, and use of prescription medicines in the treatment of both acute and chronic conditions. Medication adherence is a natural candidate for quality measurement, as individuals with higher adherence consistently demonstrate better clinical outcomes as well as lower use of and spending on other medical services.4-7 For example, a recent study found that poor adherence can lead to higher costs of up to $702 per month for Medicare beneficiaries with diabetes and up to $840 for those with heart failure.8 This persistent empirical relationship was reflected recently in the decision by the Congressional Budget Office to incorporate a reduction in spending on medical services to reflect the health benefits associated with medication use among Medicare beneficiaries.9
However, there is little understanding of the ways in which explicitly measuring and tracking medication adherence and outcomes at the plan level will incentivize health plans to implement measures that improve upon those process and outcome metrics. In particular, it is not known whether plans with higher average adherence exhibit better health outcomes or lower health spending. There is considerable research demonstrating that medication adherence is associated with improved clinical outcomes and reduced spending for individuals,9 although this does not necessarily mean that average medication adherence rates can provide useful signals of health plan performance. There are many drivers of patient outcomes and plan performance, and it is possible that medication adherence does not vary enough across plans, or that the patient populations differ too much, to isolate a clear relationship between plan-level adherence and outcomes. To better understand the implications and possible value of using medication adherence as a measure of health plan performance, we need to understand the link between adherence and outcomes at the plan level after controlling for patient differences across plans.
We used a large database of claims from private sector health plans to compute commonly used measures of plan performance on outcomes and plan-level rates of medication adherence for patients with diabetes and congestive heart failure (CHF). We selected these 2 conditions because they are relatively common and lead to significant social costs, and also because there is ample evidence supporting a relationship between adherence to guideline-recommended medications and improved clinical outcomes for these patients on an individual level.1,6,10-15 We tested the relationship between plan-level quality measures of medication adherence and health outcomes to assess whether better adherence was associated with lower disease complication rates and spending.
We estimated the relationship between plan-level adherence and outcomes according to a 2-step process. We first computed average rates of adherence and health complications related to diabetes and CHF at the plan level. We computed unadjusted averages, and we also used multivariate regression to adjust for patient characteristics and to predict the adherence and complication rates that would result if every plan had patients with the same observable characteristics. In the second stage, we estimated the association between adherence and disease complication rates in order to test whether higher adherence was associated with better outcomes. Here we describe these procedures and the data used in greater detail.
Data and Study Sample
We used a database of medical and pharmacy claims data for individuals enrolled in health plans of large private employers from 2000 to 2009. These data have been used extensively to study medication adherence and patient outcomes in past work.16-22 The database contained medical claims or encounter data from all available healthcare sites, including both inpatient and outpatient care. The billing data were broken down by payer (eg, out-of-pocket or health plan) and provided information on up to 4 associated diagnosis codes and 1 associated procedure code. The database also included pharmacy claims for all outpatient pharmaceutical purchases; each claim included unique drug identifiers, the number of days of medication supplied, and total payments by all payers. Enrollment records provided information on basic demographics, including age, gender, and 3-digit zip code of residence. These claims data were rolled up to a patient level to identify individual diagnoses, characteristics, adherence, and hospitalizations, and then aggregated at the plan level for analysis.
Study Sample
We restricted the sample to individuals aged 18 to 64 years who were diagnosed with either diabetes or CHF. We excluded individuals 65 years or older to focus on private health plans and to exclude patients covered by Medicare. Patients with diabetes were identified based on an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code of 250.xx listed as the primary or secondary diagnosis in 1 inpatient claim, or at least 2 outpatient claims within a 30-day window over 1 calendar year. We required at least 2 outpatient codes to eliminate cases where the ICD-9-CM code was listed on a claim as a possible diagnosis but was later ruled out. CHF was identified the same way, using ICD-9-CM diagnosis codes of 428.xx.
Using these inclusion criteria, we created separate cohorts of individuals with diabetes and with CHF. Any individual diagnosed with both diseases in the same year was included separately in both samples. Once identified as being in either or both samples, individuals were grouped into health plans according to their enrollment records. Individuals were included from the year of initial observed diagnosis for as long as they remained enrolled. We required that individuals be continuously enrolled over a full calendar year to ensure that we observed all their utilization throughout the year. When aggregating the information to the plan level, we also required that plans have at least 100 patients to be included in the analysis.
Plan Performance Measures
Many quality measures have been developed to evaluate plan performance, and a full accounting of the relationship between all quality measures tracking medication adherence and health outcomes is beyond the scope of this study. We evaluated a set of quality metrics specific to medication adherence and health outcomes that were endorsed by the National Quality Forum (NQF), relevant for diabetes or CHF, and observable in retrospective claims analysis.23 The NQF is a nonprofit organization whose mission is to build consensus around approved quality measures for health plan performance. The NQF measures are used by CMS in quality reporting programs for physicians, plans, and hospitals.
Medication adherence. We measured medication adherence using the percentage of days covered (PDC) over a 1-year period. PDC is an NQF-endorsed quality measure for adherence that is commonly used by researchers to evaluate patient adherence.23 Therapeutic classes were selected based on treatment guidelines for diabetes and CHF. Diabetes medications included beta-blockers, angiotensin-converting enzyme (ACE) inhibitors, angiotensin receptor blockers (ARBs), calcium channel blockers, oral diabetes medications, and statins. For CHF, we included beta-blockers, ACE inhibitors, ARBs, and diuretics. Within each health plan, we identified all patients who were prescribed each type of medication at least once and computed the PDC for the calendar year after the date of initiation. Patients who met the PDC threshold of 80% or greater over the year were considered to have “good” adherence. The 80% PDC threshold is often used as an NQF-endorsed measure of quality; for example, CMS uses it for certain drug classes in its performance measurement system for Medicare Part D plans.24
Patient outcomes. We measured patient outcomes using quality measures endorsed by NQF that reflect disease complications generally considered preventable for the 2 relevant diseases. For diabetes, this includes the rate of inpatient admissions for uncontrolled diabetes, emergent care for hypoglycemia or hyperglycemia, rates of short-term diabetes complications, and rates of long-term diabetes complications. Given that some outcomes are relatively uncommon, we reported them in terms of rates per 1000 patients. CHF outcome measures included hospitalizations or emergent care visits for CHF or other comorbid conditions, including diabetes, coronary artery disease, or hypertension. As these were comparatively more common than the diabetes complications, they were computed in terms of percent of patients per year. We also computed the average annual total medical expenditures for patients in each sample, normalized to 2009 dollars using the Consumer Price Index.
Statistical Analysis
Regression adjustment. The goal of our analysis was to describe how variation in medication adherence across plans correlated with variation in plan-level patient outcomes. However, this correlation could be confounded by the presence of differences across plans; for example, plans with sicker patients will on average tend to have patients using more medications and more medical services, regardless of their adherence patterns. To address for potential bias driven by systematic differences in patients across health plans, we adjusted for patient characteristics using multivariate regression. We then aggregated adjusted adherence and outcome values up to the plan level to construct our key analytic plan-level quality measures of disease complications, spending, and adherence. In the eAppendix (available at, we expand on this and provide a more detailed discussion of our empirical strategy.
We estimated separate regression models for each medication class and each health outcome. All regressions included controls for patient age, gender, and number of comorbidities, as well as fixed effects for state of residence and year. Note that while our data identify plans consistently within a year, the coding of plans is not fully consistent within an employer over time. Thus, a plan in our analysis really refers to a plan-year, which is our primary unit of analysis, and we were unable to use plan-fixed effects. In the eAppendix, we discuss the results of a sensitivity analysis in which we included employer fixed effects and found that our results were largely unchanged. Comorbidities were identified from the medical claims and included ulcer, depression, allergic rhinitis, migraine, osteoarthritis, chronic sinusitis, anxiety or tension disorder, epilepsy, gastric acid disorder, glaucoma, irritable bowel syndrome, malignancies, psychotic illness, thyroid disorder, rheumatoid arthritis, tuberculosis, HIV, anemia, and chronic obstructive pulmonary disease. Other common conditions that were related to cardiovascular health (eg, acute myocardial infarction or hypertension) were excluded because the onset of claims could be endogenously related to individuals’ medication use or other healthcare related to their diabetes or CHF.
We calculated adjusted plan-level adherence and health outcomes as follows. For medication adherence, we regressed the percent of individuals with PDC greater than 80% on these covariates using a linear probability model to obtain predicted PDC values for each individual. We then generated predicted values for adjusted adherence holding other factors fixed at their mean values, so that the remaining variation would come from fixed differences across plan-years and not from differences in observable patient characteristics. We replicated this procedure for health outcomes and spending, leaving us with a set of adjusted, plan-level averages for medication adherence and outcomes. The same process was conducted separately for CHF and diabetes.

This regression adjustment eliminates observed heterogeneity in the patient population across plans but does not necessarily address unobserved heterogeneity across plans that could be correlated with both medication adherence and health outcomes. For example, significant changes in the plan population in a given year (if, say, the plan benefits changed in such a way as to attract sicker patients with poorer adherence) could drive a correlation between adherence and outcomes. Thus, the association between plan-level adherence and outcomes are not necessarily causal, and the findings should be interpreted accordingly.
An index of medication adherence. We computed the average level of medication adherence across plans separately for beta-blockers, ACE inhibitors, ARBs, calcium channel blockers, oral diabetes medications, and statins in the diabetes sample, and beta-blockers, ACE inhibitors, ARBs, and diuretics in the CHF sample. We then combined the PDC measures into a single index of medication adherence for each condition. Based on the value of the index, we grouped health plans into categories of high, moderate, and low levels of adherence.
Specifically, to construct the index we applied the following procedure separately in both disease cohorts. First, for each health plan we computed the average share of patients with at least an 80% PDC separately for each medication class. Then, for each plan, we calculated the weighted average of adherence across drugs for each plan-year, where adherence was weighted by the number of patients in the plan taking each drug. Using this weighted average, we identified the upper and lower quartile values of adherence across plans (ie, the values of adherence that 25% of plans were above and below, respectively). Finally, we stratified plans into 3 categories of adherence—high, moderate, or low—based on whether their weighted average was above the 75th percentile, between the 75th and 25th percentile, or below the 25th percentile across plans, respectively. We focused our analysis on the top and bottom quartiles of plans because we wanted to emphasize the impact of plans that did particularly well or particularly poorly—the “outlier” plans.
We used the same process to categorize plans into different adherence categories using both the adjusted and unadjusted values of the PDC, and in both disease cohorts. For our main analysis, we regressed the adjusted health outcomes against the medication adherence variables as the independent variables using ordinary least squares regression. Note that in a 2-step estimation procedure, where we use regression to adjust for health plan characteristics in the first step and regress adjusted outcomes against adjusted adherence in the second step, the variance estimates are biased in the second step.25 To statistically compare differences across the low, moderate, and high adherence plans, we used a bootstrap procedure. Specifically, we resampled individuals with replacement and computed P values for differences across low, moderate, and high adherence plans using bootstrap variance estimates based on 200 draws.
Table 1 presents the share of individuals within plans who show evidence of good adherence, as identified by a PDC of 80% or better. Only adherence to medications used in plan quality measures (as specified above) are computed for each sample; for example, we do not report adherence to antidiabetes medications in the CHF sample, even though CHF patients with diabetes may take them. For patients with diabetes, 54% to 56% of patients within a plan exhibit good adherence to therapy, on average, except in the case of statin medications, for which just 45.2% exhibit good adherence. Adherence to ACE inhibitors and ARBs is generally lower for CHF patients within a plan, while adherence to diuretics is particularly low, at 37%. Individuals taking diuretics have been shown in past research to exhibit lower adherence levels than other agents for treating cardiovascular disease.26,27 As a result, the weighted average of plan-level adherence across therapies is significantly lower for the CHF sample (46.8%, compared with 52.9% for diabetes).
We report plan-level characteristics in Table 2. The table reports the mean values of adherence and other characteristics, including health outcomes, plan size, and average total medical expenditures, for different plan years. The table shows that medication adherence is slightly higher in the diabetes sample (approximately 53% compared with 47%). Also, as we would expect, there are many more diabetes patients per plan, while patients with CHF are sicker on average, demonstrated by their higher rates of complications and higher total healthcare expenditures. Note that because we required at least 100 patients in a plan to be included in the analysis, the number of plans included is much smaller in the CHF sample than the diabetes sample (195 vs 919).
The thresholds below and above which plans are defined as having low or moderate adherence, respectively, were defined using the 25th and 75th percentiles of the adherence across plans. We classified plans with adherence below the 25th percentile as having low adherence, those above the 75th percentile as having high adherence, and those with adherence between these levels as having moderate adherence. We found that there was substantial variability in average medication adherence across plans. In the diabetes sample, the threshold for low plan-level adherence is 48.3%, meaning that less than half of plan enrollees with diabetes are considered to have good adherence. The threshold for high plan-level adherence, the 75th percentile, is 58.6%. The significant differences in adherence across plans are important, because they suggest that there is heterogeneity in adherence across plans that can potentially be targeted for intervention by policy makers. For CHF patients, adherence ranges from 43.5% at the 25th percentile to 50.9% at the 75th percentile. In the eAppendix, we provide some additional detail on the sample and the variation in key variables across plans.
In Table 3, we compare the variation in plan-level health outcomes to plan-level adherence in the diabetes sample. Specifically, we report the average health outcomes for patients in the low, moderate, and high adherence plans. The top panel reports the association between unadjusted outcomes and unadjusted adherence, while the bottom panel reports the association between adjusted outcomes and adjusted adherence. P values are reported in parentheses reflecting statistically significant differences between the moderate and high adherence plans and the poor adherence plans.
There is a positive association between poor adherence and adverse outcomes for diabetes patients. In the top “unadjusted” panel, low adherence plans exhibit significantly more uncontrolled diabetes, emergent care for glycemic events, and long-term complications, compared with the high adherence plans. The patterns for diabetes complications are less clear, but adjusting for differences in patient characteristics across plans clarifies the associations.
After adjusting for the characteristics of individual patients, patients in low adherence plans exhibit statistically worse outcomes compared either with those in high adherence plans, or with those in moderate adherence plans. Significance is achieved at the 1% level for all the high adherence comparisons, and at the 5% or 10% level for the moderate adherence comparisons.
Table 4 reports the associations between plan adherence measures and outcome measures for CHF patients. As with the previous table, the top panel reports unadjusted values and the bottom panel reports adjusted outcomes, with bootstrap P values in parentheses.
For the unadjusted values, low adherence plans have statistically higher (P <.001) rates of hospitalizations and emergency department (ED) visits for CHF itself and all the co-morbidities considered in quality measurement, compared with high adherence plans. The same is true when comparing low adherence plans to moderate adherence plans, where the highest P value is .01. These findings are largely unchanged by the regression adjustment, as shown in the bottom panel. Low adherence plans perform significantly worse than those with moderate or high adherence. For example, rates of CHF hospitalization are 3.1 percentage points higher in low versus high adherence plans, a 25% difference. Rates of CHF ED visits for low adherence plans are 3.9 percentage points higher, nearly double those for plans with moderate or high adherence. The adjusted comparison of adherence and coronary artery disease hospitalizations are not significantly different for high adherence versus low adherence plans.
In addition to directly assessing the relationship between quality measures for adherence and health outcomes, we also explored the implications for healthcare spending on patients with diabetes and CHF. Accordingly, we compared adjusted average annual non-drug medical expenditures (defined as the sum of inpatient and outpatient expenditures) by diabetes and CHF patients according to plan adherence (Figure). Results demonstrated that better adherence is associated with substantially lower average spending per year among both the diabetes and CHF samples. For example, average annual expenditures for CHF patients in plans that had low average adherence were approximately $31,500, compared with $20,407 for patients in high adherence plans (P <.001). Similarly, annual expenditures for patients with diabetes in low adherence plans were $8784, compared with $6766 in high adherence plans (P <.001).
We used the relative differences in predicted expenditure values to simulate potential cost savings associated with improving performance among plans with low adherence. Our estimates suggest that moving all low adherence plans to the moderate category would, on average, reduce aggregate spending among patients with diabetes by 1.6% and among patients with CHF by 6.3%. Similarly, moving all the low and moderate adherence plans to high adherence would reduce spending among diabetes patients by 14.1% and among CHF patients by 13.7%. Note that because there is overlap between the diabetes and CHF samples, some of the improvement in diabetes patients could be due to improved treatment of CHF and vice versa. Thus, these differences in spending are not additive in terms of the implications for total spending at the plan level.
We used a large database of private sector health claims to estimate the association between quality metrics of medication adherence and health outcomes at the health plan level for patients with diabetes and CHF. Our results demonstrated that there was significant systematic variation in medication adherence across plans. Moreover, we found a consistently positive correlation between high plan-level adherence and good health outcomes. Patients with CHF or with diabetes in health plans with low and moderate levels of adherence are significantly more likely to experience disease-related complications than patients in plans with high adherence. These findings suggest that medication adherence can provide a useful marker for good health plan performance.
The positive association we find between adherence and outcomes at the plan level could be due to a constellation of factors, including the positive health benefits of adherence itself, along with other quality-improving strategies that may promote both good adherence and good outcomes. For example, plans that reward and retain physicians that promote good medication adherence might end up exhibiting both good adherence and good outcomes. Nevertheless, our results support the use of adherence metrics as a potentially important way to separate better-performing plans from their peers.
Moreover, if the estimated associations were causal in nature, our findings would demonstrate the possible value of improving plan-level adherence as a potentially significant source of cost savings. To understand the magnitude of potential savings, consider that excess medical costs associated with diabetes and CHF were estimated to be $116 billion and $25 billion, respectively, in 2007 (the most recent year available).15,28 Applying our estimated association between cost reductions and improved adherence nationwide, plans with low adherence metrics could save $2.1 billion annually on patients with diabetes and $1.9 billion annually on patients with CHF by improving the adherence of their enrollees to a moderate level in 2012 dollars. Similarly, plans could save $19.3 billion annually on patients with diabetes and $4.1 billion annually on patients with CHF if all plans achieved high adherence. These figures are rough estimates, as our findings are based only on the commercial sector while the burden estimates are for the entire population. There also could also be overlap in savings between the 2 disease states, so the total savings are not additive across the 2 populations. Nevertheless, the potential for savings is significant.
Our findings have several policy implications. For example, providing incentives for plans to improve adherence by tying measures of medication adherence to reimbursement could be an effective lever to improve the overall quality of care while reducing unnecessary expenditures. This could be done by expanding the use of performance measures based on medication adherence in public schemes, such as Medicare Part D or the health insurance exchanges. Under current Medicare policy, Part D plans are required to report on quality measures for use of and adherence to medications used to treat diabetes, hypertension, and high cholesterol, and use them to evaluate Part D plan performance in publicly reported Star ratings. Our findings suggest this could provide significant value for health plans and patients, along with providing support for consideration of a more comprehensive set of measures.
However, this discussion rests on the important assumption that plans have the ability to drive medication adherence. Our study demonstrates that plans differ substantially in terms of the medication adherence of their beneficiaries, and those plans with patients who have better adherence also have better outcomes. But we do not demonstrate why some plans have better adherence than others, or to what extent plans can actively influence enrollee medication-taking behavior. There are many reasons why medication adherence might systematically differ across plans, and which could provide potential levers for plans to improve. For example, formulary design and cost sharing can both influence patient use of medicines and other medical services, as well as adherence.11,13,23-25,29-33 In addition, synchronizing multiple medication refills to take place on a single monthly pickup date, improving care coordination, adopting health information technology, and providing plan incentives are promising interventions, though questions about effectiveness, scalability, and generalizability persist.34 It will be important to provide health plans with a strong evidence base for cost-effective interventions, as simply tying reimbursement to adherence will not necessarily improve outcomes if plans or providers lack the knowledge or ability to change patient behavior.
Our study had several limitations. While our data are national and cover a large and diverse set of patients, they do not comprise a nationally representative random sample. This lack of geographic representativeness could compromise the generalizability of our findings; however, these same data have been shown to provide accurate and generalizable measures of patient behavior and spending levels in numerous past studies.16-19 We were also limited in the range of quality measures we could evaluate given data availability, and further work should extend these analyses using quality measures that require additional data elements unavailable in medical claims (eg, laboratory and/or chart records). Our analysis was based on claims data and lacked more direct measures of disease severity, such as glycated hemoglobin levels for diabetic patients or ejection fraction for CHF patients. Also, while we defined good adherence as above 80% PDC to make it consistent with quality measures, further study could evaluate whether the relationship between adherence and outcomes varies across other levels of adherence.
As we have noted throughout, a potentially important limitation of this study is that we do not estimate a causal relationship between plan-level adherence and health outcomes. There is a clear association between adherence and plan outcomes, but understanding whether that is driven causally by the effects of adhering to a medication regimen or by unobserved heterogeneity among patients or plans is crucial for interpreting the study findings. More work is needed to understand the causal mechanisms that drive plan differences in medication adherence and the implications for patient outcomes.
Despite these limitations, our study provides a promising first step toward demonstrating that adherence is a promising marker for good performance in a health plan. The constellation of activities that high adherence plans undertake to improve quality seems to produce gains in health outcomes for patients. Gaining more insight into these specific activities emerges as a critical question for research that can help inform policy development.

Print | AJMC Printing...