Currently Viewing:
The American Journal of Managed Care November 2017
Using the 4 Pillars to Increase Vaccination Among High-Risk Adults: Who Benefits?
Mary Patricia Nowalk, PhD, RD; Krissy K. Moehling, MPH; Song Zhang, MS; Jonathan M. Raviotta, MPH; Richard K. Zimmerman, MD, MPH; and Chyongchiou J. Lin, PhD
The Influence of Provider Characteristics and Market Forces on Response to Financial Incentives
Brock O’Neil, MD; Mark Tyson, MD; Amy J. Graves, SM, MPH; Daniel A. Barocas, MD, MPH; Sam S. Chang, MD, MBA; David F. Penson, MD, MPH; and Matthew J. Resnick, MD, MPH
Patients' Perspectives of Care Management: A Qualitative Study
Ann S. O’Malley, MD, MPH; Deborah Peikes, PhD, MPA; Claire Wilson, PhD; Rachel Gaddes, MPH; Victoria Peebles, MSW; Timothy J. Day, MSPH; and Janel Jin, MSPH
Impact of Health Reform on Young Adult Prescription Medication Utilization
Amy Pakyz, PharmD, PhD, MS; Hui Wang, PhD; and Peter Cunningham, PhD
Reframing the Unaffordability Debate: Patient Responsibility for Physician Care
Katherine Hempstead, PhD; Josh Gray, MBA; and Anna Zink, BA
Electronic Reminder's Role in Promoting Human Papillomavirus Vaccine Use
Jaeyong Bae, PhD; Eric W. Ford, PhD, MPH; Shannon Wu, BA; and Timothy Huerta, PhD, MS
Improving Antibiotic Stewardship: A Stepped-Wedge Cluster Randomized Trial
Adam L. Sharp, MD, MS; Yi R. Hu, MS; Ernest Shen, PhD; Richard Chen, MD; Ryan P. Radecki, MD, MS; Michael H. Kanter, MD; and Michael K. Gould, MD, MS
Changes in Cardiovascular Care Provision After the Affordable Care Act
Joseph A. Ladapo, MD, PhD; and Dave A. Chokshi, MD, MSc
Diabetes Care Improvement in Pharmacist- Versus Nurse-Supported Patient-Centered Medical Homes
Lillian Min, MD, MSHS; Christine T. Cigolle, MD, MS; Steven J. Bernstein, MD, MPH; Kathleen Ward, MPA; Tisha L. Moore, MPH; Jinkyung Ha, PhD; and Caroline S. Blaum, MD, MS
Currently Reading
Validation of a Claims-Based Algorithm to Characterize Episodes of Care
Chad Ellimoottil, MD, MS; John D. Syrjamaki, MPH; Benedict Voit, MBA; Vinay Guduguntla, BS; David C. Miller, MD, MPH; and James M. Dupree, MD, MPH

Validation of a Claims-Based Algorithm to Characterize Episodes of Care

Chad Ellimoottil, MD, MS; John D. Syrjamaki, MPH; Benedict Voit, MBA; Vinay Guduguntla, BS; David C. Miller, MD, MPH; and James M. Dupree, MD, MPH
The Michigan Value Collaborative has created a claims-based algorithm that categorizes claims into episode components. This manuscript describes the validation of this algorithm.

Objectives: Although hospitals face increasing pressure from payers to improve the efficiency of healthcare delivery beyond the index hospitalization, they often lack information on postdischarge events. The Michigan Value Collaborative (MVC) developed a claims-based algorithm to provide hospitals with data on events that occur to patients beyond the hospitalization. Herein, we discuss the validation of MVC’s claims-based algorithm. 

Study Design: Retrospective analysis of a claims-based algorithm’s ability to identify specific medical events, such as index hospitalizations, 30-day readmissions, emergency department visits, skilled nursing facility admissions, home health visits, and rehabilitation services. The claims-based events were validated using a primary review at 63 hospitals. 

Methods: We selected 1830 Blue Cross Blue Shield of Michigan episodes from MVC data and asked 63 Michigan hospitals to query their medical records for the presence or absence of specific events. We then calculated agreement statistics and improved our algorithm using feedback from hospitals. 

Results: All 63 hospitals participated in the validation process and successfully identified 99% of episodes in their medical records. The initial agreement between our algorithm and medical records was moderate for 4 postdischarge events (kappa ranging from 0.62-0.78) and poor for rehabilitation services (0.16). Much of the disagreements occurred because hospitals could not identify postdischarge events occurring outside of their hospital systems. Other disagreements occurred because of hospital coding practices. Through this analysis, the claims-based algorithm was improved to better reflect real-world coding practice.

Conclusions: Our findings suggest that the MVC claims-based algorithm identifies and classifies claims with high fidelity and outperforms medical records in the identification of postdischarge events. These findings provide important insight to policy makers, payers, and hospital administrators about the value of claims-based data for the implementation of episode-based programs.

Am J Manag Care. 2017;23(11):e382-e386
Takeaway Points

The Michigan Value Collaborative (MVC)’s claims-based algorithm identifies postdischarge events with high fidelity, often outperforming medical records.
  • The creation of this algorithm and its subsequent validation demonstrates that deriving episode-level utilization from administrative claims is achievable. 
  • With an increasing focus on improving efficiency of care after the index hospitalization, hospitals need accurate information. To this end, MVC’s claims-based algorithm provides precise episode-level data on postdischarge care. 
  • Given that administrative claims data outperformed medical records, hospitals should seek to collect such data, possibly by participating in a statewide, regional, or health-system value improvement collaborative or asking payers for claims-based analytic data.
Hospitals are increasingly being held accountable for services and expenditures that occur beyond the hospitalization through episode-based performance measures.1-6 Postdischarge expenditures, such as postacute care and readmissions, have been cited as the fastest growing spending categories over the last 2 decades and have been the target of many national programs focused on reducing healthcare costs.3 For example, CMS recently implemented the Comprehensive Care for Joint Replacement bundled payment program, which will hold hospitals financially accountable for expenditures occurring from admission through 90 days post discharge.7 In addition, accountable care organizations were developed to reduce costs that occur both outside of and during hospitalizations.8 Based on the prevalence and growth of episode-based payment programs, it is evident that many payers believe the key to reducing healthcare expenditures is to hold hospitals responsible for efficiency along the entire patient care episode. 

Despite enthusiasm for increased episode efficiency, identifying specific high-cost events, such as readmissions, can be challenging for hospitals for several reasons. First, it is difficult for hospitals to track events outside of the initial hospitalization. In fact, hospitals are often not even aware of postdischarge events that occur at outside facilities. Secondly, unless directly affiliated with the hospital, postdischarge providers are not incentivized to report utilization patterns to hospitals. Furthermore, many small hospitals may not have internal resources to monitor and track postdischarge events and spending. 

In Michigan, one response to these challenges was the development of the Michigan Value Collaborative (MVC).9 Established in 2012 and funded by Blue Cross Blue Shield of Michigan (BCBSM), MVC’s mission is to provide hospitals with episode-level data and promote high-quality care at the lowest reasonable costs.10 One particular area of interest has been postacute care, specifically rehabilitation. MVC hospitals have begun to use episode-level data to monitor rehabilitation expenditures, especially for conditions such as acute myocardial infarction (AMI) and hip replacement, where more than one-third of all patients are discharged to rehabilitation facilities (Table 1). However, a necessary step in this process is to ensure that MVC’s claims-based identification of postdischarge services is accurate. 

In this context, we describe a large-scale medical records–based validation of the algorithm used by MVC to define clinical episodes of care in commercial claims. We believe that the MVC validation experience will provide useful insight to hospitals and payers about the advantages and limitations of using claims to track events that occur after hospitalization.


Data Sources 

The MVC collects claims data for BCBSM beneficiaries admitted to 1 of 63 Michigan hospitals for 21 medical and surgical conditions. We consulted clinical experts who used International Classification of Diseases, Ninth Revision (ICD-9) diagnosis/procedure and Current Procedural Terminology (CPT) codes to define the conditions. Then, we utilized variables such as revenue codes, diagnosis-related groups (DRGs), facility identification, and CPT codes to classify individual claims into: inpatient, skilled nursing facility (SNF), home health, emergency department (ED), inpatient rehabilitation, outpatient rehabilitation, and general outpatient claims. Cases with readmissions to hospitals other than the index facility were excluded from the readmissions validation step. eAppendix A (eAppendices available at further details the entire attribution process. Table 1 presents the characteristics of episodes identified from claims data by condition type, including use of postdischarge services. This study was deemed exempt from review by the Michigan Institutional Review Board. 

Validation Process 

Our validation process occurred in 2 phases (pilot and full validation). During the pilot, 6 hospitals were asked to review 10 to 20 BCBSM preferred provider organization (PPO) cases from between January 1, 2013, and October 31, 2014. For matching purposes, the clinical condition, national provider identifier, date of birth, gender, admission date, and discharge date for each patient was provided. Specifically, participants were instructed to indicate if the patient listed had records demonstrating that a specified event occurred within 90 days of their discharge date. These events included 30-day readmissions, ED visits, SNF admissions, home health visits, and rehabilitation services (inpatient and outpatient).

The lessons learned from the pilot were used to inform the full validation. Here, we distributed the same key variables to all 63 MVC participants. Each hospital was provided 30 cases to review, with the exception of hospitals that participated in the pilot (which were asked to review 20 cases). We selected conditions based on volume, prevalence of associated postdischarge services, and suggestions from clinical experts. The conditions included colectomy, coronary artery bypass graft, AMI, pneumonia, congestive heart failure, hip replacement, knee replacement, cesarean delivery, vaginal delivery, trauma, and spine surgery. In total, 1830 BCBSM PPO cases were selected for data validation from these 11 conditions. 

Statistical Analyses 

We identified areas of agreement and disagreement between MVC’s claims-based algorithm and medical records. First, we looked for agreement that these episodes occurred and were attributed to the correct hospital and to the correct condition. Next, we used a kappa statistic to assess agreement for postdischarge services (eAppendix B). For each disagreement in a case, 2 members of the MVC team, a clinician and an analyst, reviewed the specific claims. After determining the cause of the discordance, we adjusted our algorithm to better capture hospital events and re-evaluated the level of agreement. 

In the cases where MVC reported a postdischarge event that did not match a hospital’s medical records, we used confirmatory evidence from the claims to improve our confidence that the event occurred. Specifically, we examined the following items: 1) Did the postdischarge event have more than 2 claim line items with dates following the index admission?; 2) Did the place of service designation on the claims support the assignment of the claim to the postdischarge service?; and 3) Did the revenue center code on the claims support the assignment of the claim to the postdischarge service? 


One hundred percent of hospitals (n = 63) participated in the validation process. Hospitals matched 1812 of the 1830 (99%) MVC episodes to records in the hospital’s medical charts. 


The agreement for the occurrence of postdischarge services ranged from 0.16 to 0.78, with rehabilitation services having the poorest agreement. eAppendix B contains details of the validation process. 

30-Day Readmissions 

Using administrative claims, we were able to identify 183 readmissions for 1812 episodes (10% readmission rate). Of these, there were 15 cases in which MVC observed a 30-day readmission that was not evident in the hospital data (Table 2). These were due to readmissions to a hospital different than the index facility and, per the methods section, were excluded. There were 24 discordant readmissions which MVC did not identify and a hospital did; these cases were classified as observation unit stays in MVC data (eAppendix B). 

ED Visits 

We identified 452 ED visits (25% of episodes). Of these visits, 292 (65%) were identified by hospitals. Importantly, there were 160 out of 452 cases (35%) in which MVC observed an ED visit that was not reported in the hospital clinical data (Table 2). Of the 160 discordant cases, 122 (76%) of these ED visits occurred at a different facility than the index admission. All 160 cases had confirmatory evidence of a valid ED visit. 

There were 85 episodes (5%) in which a hospital reported an ED visit that was not evident in the MVC data. Upon review, many of these were classified elsewhere. For example, 38 preceded an index admission or a readmission and the ED services are grouped with this hospitalization in the MVC episode (eAppendix B). 

SNF Admissions 

We identified 64 SNF admissions (4% of episodes). Of these, 43 (67%) admissions were identified by hospitals. There were 21 cases (33%) in which MVC identified a SNF admission but the admissions were not reported by hospitals (Table 2). We found evidence in the claims data to confirm that all 21 visits occurred. There were only 9 cases in which hospitals reported care in a SNF, and we found no evidence of a SNF admission in the MVC claims data (eAppendix B). 

Home Health Visits 

We identified 949 home health visits (52% of episodes). Of these visits, 781 (82%) were identified by hospitals (Table 2). Of those reported by hospitals, there were 168 cases (21%) in which MVC observed a home health visit that was not evident in the hospital clinical data. There were 99 cases (10% of episodes) in which a hospital observed a home health visit that was not evident in the MVC data (eAppendix B). 


The algorithm identified 1223 rehabilitation visits (67% of episodes). Of these visits, 350  (29%) were reported by hospitals. There were 873 cases (71%) in which MVC observed a rehab visit that was not evident in the hospital clinical data (Table 2). Of the 873 discordant cases, 851 (97%) had confirmatory evidence in the MVC claims for utilization of rehabilitation services. There were only 42 cases (3% of episodes) in which a hospital observed a rehab visit that was not evident in the MVC data (eAppendix B). 

Improvements to the Claims-Based Algorithm 

After reviewing all cases of discordance between the claims-based algorithm and the medical records, we identified several areas for improvement, the most important being to update the ICD-9 codes used to identify postdischarge claims to better reflect the coding practices used by hospitals. For example, ICD-9 code V57.8 (care involving other specified rehabilitation procedure) was initially not considered by MVC as a related diagnosis code for patients after joint replacement. However, this code was used for a large number of rehabilitation services after joint replacement. After making this and other improvements, we re-evaluated the level of agreement between our algorithm and the medical records and found improvement in agreement for all services (eAppendix B). 


In this study, we validated the MVC’s claims-based algorithm for the identification and classification of postdischarge events. During the process, we found a high episode match rate between MVC data and medical records. Much of the disagreement was due to the inability of hospitals to identify readmissions, rehabilitation services, and other postdischarge events. Collectively, these findings suggest that this claims-based algorithm outperforms medical records in identifying postdischarge events. 

Copyright AJMC 2006-2018 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!