Currently Viewing:
The American Journal of Managed Care May 2017
Drivers of Excess Costs of Opioid Abuse Among a Commercially Insured Population
Lauren M. Scarpati, PhD; Noam Y. Kirson, PhD; Miriam L. Zichlin, MPH; Zitong B. Jia, BA; Howard G. Birnbaum, PhD; and Jaren C. Howard, PharmD
Critical Incident Stress Debriefing After Adverse Patient Safety Events
Reema Harrison, PhD, MSc, BSc, and Albert Wu, MD, MPH
Assessing the Effect of the VHA PCMH Model on Utilization Patterns Among Veterans With PTSD
Ian Randall, PhD; Charles Maynard, PhD; Gary Chan, PhD; Beth Devine, PhD; and Chris Johnson, PhD
State Prescription Drug Monitoring Programs and Fatal Drug Overdoses
Young Hee Nam, PhD; Dennis G. Shea, PhD; Yunfeng Shi, PhD; and John R. Moran, PhD
Disparities in Diabetes and Hypertension Care for Individuals With Serious Mental Illness
Junqing Liu, PhD; Jonathan Brown, PhD; Suzanne Morton, MPH; D.E.B. Potter, MS; Lisa Patton, PhD; Milesh Patel, MS; Rita Lewis, MPH; and Sarah Hudson Scholle, DrPH
Currently Reading
The Cost of Adherence Mismeasurement in Serious Mental Illness: A Claims-Based Analysis
Jason Shafrin, PhD; Felicia Forma, BSc; Ethan Scherer, PhD; Ainslie Hatch, PhD; Edward Vytlacil, PhD; and Darius Lakdawalla, PhD
Opioid Prescribing for Chronic Pain in a Community-Based Healthcare System
Robert J. Romanelli, PhD; Laurence I. Ikeda, MD; Braden Lynch, PharmD; Terri Craig, PharmD; Joseph C. Cappelleri, PhD; Trevor Jukes, MS; and Denis Y. Ishisaka, PharmD
The Association of Mental Health Program Characteristics and Patient Satisfaction
Austin B. Frakt, PhD; Jodie Trafton, PhD; and Steven D. Pizer, PhD
Medicaid Prior Authorization and Opioid Medication Abuse and Overdose
Gerald Cochran, PhD; Adam J. Gordon, MD, MPH; Walid F. Gellad, MD, MPH; Chung-Chou H. Chang, PhD; Wei-Hsuan Lo-Ciganic, PhD, MS, MSPharm; Carroline Lobo, MS; Evan Cole, PhD; Winfred Frazier, MD; Ping

The Cost of Adherence Mismeasurement in Serious Mental Illness: A Claims-Based Analysis

Jason Shafrin, PhD; Felicia Forma, BSc; Ethan Scherer, PhD; Ainslie Hatch, PhD; Edward Vytlacil, PhD; and Darius Lakdawalla, PhD
This study demonstrates that common pharmacy claims-based measures underestimate the effect of actual adherence on inpatient costs among patients with serious mental illness.

To quantify how adherence mismeasurement affects the estimated impact of adherence on inpatient costs among patients with serious mental illness (SMI).

Study Design: Proportion of days covered (PDC) is a common claims-based measure of medication adherence. Because PDC does not measure medication ingestion, however, it may inaccurately measure adherence. We derived a formula to correct the bias that occurs in adherence-utilization studies resulting from errors in claims-based measures of adherence. 

Methods: We conducted a literature review to identify the correlation between gold-standard and claims-based adherence measures. We derived a bias-correction methodology to address claims-based medication adherence measurement error. We then applied this methodology to a case study of patients with SMI who initiated atypical antipsychotics in 2 large claims databases.

Results: Our literature review identified 6 studies of interest. The 4 most relevant ones measured correlations between 0.38 and 0.91. Our preferred estimate implies that the effect of adherence on inpatient spending estimated from claims data would understate the true effect by a factor of 5.3, if there were no other sources of bias. Although our procedure corrects for measurement error, such error also may amplify or mitigate other potential biases. For instance, if adherent patients are healthier than nonadherent ones, measurement error makes the resulting bias worse. On the other hand, if adherent patients are sicker, measurement error mitigates the other bias. 

Conclusions: Measurement error due to claims-based adherence measures is worth addressing, alongside other more widely emphasized sources of bias in inference. 

Am J Manag Care. 2017;23(5):e156-e163
Takeaway Points

Using pharmacy claims data on patients with serious mental illness (SMI), this study demonstrates that increased medication adherence correlates with lower inpatient costs. A bias-correction formula was used to show that measurement error in claims-based adherence measures results in the effects of adherence being underestimated by a factor of 5.3. 
  • Medication nonadherence is a burdensome problem in SMI, substantially contributing to healthcare costs. 
  • Few studies address errors in adherence measurement, which can introduce bias when estimating the impact of adherence on inpatient costs. 
  • Using a reliable estimate of the correlation between true adherence and claims-based adherence can identify the effects of measurement error on the estimated relationship between adherence and cost.
Nonadherence to medication is a prevalent and burdensome problem among patients with serious mental illness (SMI).1 Estimates of adherence to antipsychotics among patients with schizophrenia-spectrum disorders, for instance, range from 47% to 95%.2 The consequences of poor adherence include suboptimal health outcomes and higher avoidable healthcare costs.3,4 In patients with schizophrenia, medication nonadherence impedes recovery,5-7 increases the risk of hospitalization,6,8-12 and extends the length of in-hospital stays.6,11 Overall, hospitalizations due to medication nonadherence have been estimated to cost more than $100 billion annually in the United States,13 and hospitalization costs due to antipsychotic nonadherence specifically have been estimated at $1.5 billion annually.14

CMS includes medication adherence as a rating measure when determining healthcare quality.15 A common method to indirectly assess adherence is the proportion of days covered (PDC),16 which typically uses prescription claims data and is calculated as the proportion of days in the measurement period, usually 1 year, for which the patient has medication on hand. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has determined that PDC is one of the preferred methods to calculate medication adherence.17 PDC also is used to measure adherence to antipsychotic medications as part of the Healthcare Effectiveness Data and Information Set quality measures.18 

Although PDC is widely used, it suffers from 2 key shortcomings. First, PDC underestimates adherence when patients pay cash for medication or use other coverage options that fail to result in a recorded insurance claim. Second, PDC overstates adherence when patients purchase but do not take a given medication. New technologies, such as electronic pillboxes, smart caps, or ingestible sensors, may provide more accurate adherence measurements, but currently, payers and providers rarely use these technologies to monitor adherence. 

Neither the magnitude nor the direction of the bias associated with PDC adherence estimates are widely discussed or incorporated into adherence analyses. This study aimed to quantify how adherence mismeasurement affects the estimated impact of adherence on inpatient costs among patients with SMI. 


Our methodology relies on a 3-step approach to estimate the potential impact of measurement error on inpatient spending. We mathematically derived a formula for a bias-correction factor. As this factor depends principally on the link between true and measured adherence, we next conducted a review of the literature to identify studies that measured the relationship between a “gold standard” measure of adherence (eg, Medication Event Monitoring System [MEMS] caps, electronic pill counts) and adherence measured in claims data. Finally, we applied the bias-correction factor to a case study of patients with SMI who initiated therapy with an atypical antipsychotic. 

Bias Derivation

Consider the case where a researcher wants to measure the relationship between patient adherence and inpatient spending. One commonly used approach is an ordinary least squares (OLS) regression, such as the following: 

Yi=β0+β1PDCi+Wi' γ+u  

In this case, the dependent variable Yi represents inpatient spending for patient i, PDCi represents patient adherence to atypical antipsychotics, and the vector Wi' contains other patient covariates of interest. The coefficient on adherence, β1, is the primary parameter of interest. 

Mismeasurement in PDC biases the estimated effect of adherence on inpatient spending (ie, β1 ̂) toward 0. However, the true effect of adherence on inpatient costs can be derived if the relationship between measured adherence in claims data and true adherence is known. As shown in eAppendix 1 (eAppendices available at, one can correct for measurement error bias using the following formulation:

β1 = β1 ̂ 1-R2PDCi,Wi    

where β1  is the true effect of adherence on inpatient spending after adjusting for adherence mismeasurement, β1 ̂is the OLS estimator from the regression in equation 1, and (1-R2PDCi,Wi )/(ρ2-R2PDCi,Wi) is the correction factor for the mismeasurement in PDCi. The term R2PDCi,Xi is the R-squared value of linear regression of measured adherence (PDCi) on all other patient covariates (Wi ), and ρ2 is the square of the correlation between measured and true adherence. 

The true effect of adherence is feasible to estimate, but problematic. First, one must assume that there cannot be any unobserved patient characteristics that affect both medication adherence and inpatient spending. For instance, patients with less severe forms of SMI may be more likely to be adherent to their medication and have lower inpatient costs.19,20 To address this, we derived the bias that would remain in the case where medication adherence was an endogenous variable in eAppendix 1. The formula we derived fully addresses the bias due to measurement error, even when adherence is endogenous, so that researchers will recover the same parameter that would have been estimated if they had had access to correctly measured adherence. To be clear, our approach does not simultaneously address the endogeneity itself; rather, it represents a full solution to the problem of measurement error in adherence due to the use of claims data. 

Second, one needs a reliable estimate of the correlation between true medication adherence and claims-based adherence. The following section describes our review of the literature that was used to identify this parameter. 

Literature Review

A targeted literature review was conducted in Google Scholar and PubMed to identify estimates of the relationship between gold-standard adherence measures and claims-based adherence measures. The search combined free text and medical subject headings (MeSHs; in PubMed only) that describe various measures of adherence (MEMS caps, direct observation, PDC, medication possession ration, self-report, prescription claims, lab tests, pill count, and physician estimate) and the search term “adherence accuracy.” Searches were conducted without disease specification, with the term “schizophrenia,” and with the term “serious mental illness.” After reviewing the results of this initial search, the search was conducted again with other central nervous system diseases, specifically multiple sclerosis, Alzheimer’s disease, and Parkinson’s disease. A maximum of 50 titles were screened in each search or until titles were no longer generally relevant to the research questions. 

Abstracts were screened from relevant titles, which were defined as papers that discussed or compared multiple measures of medication adherence. Titles were not relevant if they indicated an intervention to improve adherence, factors influencing adherence, or outcomes associated with adherence or nonadherence, without language suggesting relevance. Titles that were non-English, nonhuman, and did not focus on schizophrenia or another SMI were also not investigated. Full texts were assessed if their abstracts included adherence measures collected using different methodologies and these data were explicitly compared in the results or conclusions sections. Abstracts and full texts were not identified or screened more than once if they appeared as results from multiple searches or were duplicated in databases. Additional citations were identified through previous literature searches, forward reference searches of each manuscript, and the references used in each manuscript. A complete description of the search terms used, number of full texts, abstracts, and articles reviewed is contained in eAppendix 2. 

Empirical Analysis Case Study

We used the Truven Health Analytics MarketScan (MarketScan) Commercial Claims and Encounters Database and the Medicaid Multi-State Database from October 1, 2007 through December 31, 2013 to identify patients with SMI. The commercial database included medical and pharmacy claims for individuals and their dependents who were covered by employer-sponsored private health insurance. The Medicaid database included medical and pharmacy claims of Medicaid beneficiaries from 11 deidentified but geographically dispersed states. We limited the sample to individuals aged 18 or older who had at least 1 inpatient or 2 outpatient claims with a diagnosis code for schizophrenia (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] diagnosis code: 295.x), bipolar disorder (ICD-9-CM diagnosis code: 296.0x–296.1x, 296.4x–296.8x), or major depressive disorder (ICD-9-CM diagnosis code: 296.2x–296.3x, 311.x). 

To measure adherence and healthcare utilization among patients initiating therapy, patients were required to have a new prescription for an antipsychotic and to be continuously enrolled for 6 or more months before and 12 or more months after the date they filled the new prescription. We required that a patient have an SMI diagnosis and no antipsychotic prescriptions during the 6-month “clean” period before the medication initiation date. Both atypical (eg, aripiprazole, asenapine, iloperidone, lurasidone, olanzapine, paliperidone, quetiapine, risperidone, ziprasidone) and typical (eg, chlorpromazine, fluphenazine haloperidol, perphenazine) oral antipsychotics were included in the analysis. Patients using clozapine were excluded from the sample, as clozapine is typically used only for patients who do not respond to at least 2 other antipsychotic medications.21 Also excluded were patients missing data on age or patients who received antipsychotics via mail order.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up