Identifying which patients are likely to benefit from care coordination is important. We evaluated the performance of 6 risk-screening instruments in predicting healthcare utilization.
Published Online: September 17, 2013
Lindsey R. Haas, MPH; Paul Y. Takahashi, MD; Nilay D. Shah, PhD; Robert J. Stroebel, MD; Matthew E. Bernard, MD; Dawn M. Finnie, MPA; and James M. Naessens, ScD
Background: Care coordination is a key component of the patient-centered medical home. However, the mechanism for identifying primary care patients who may benefit the most from this model of care is unclear.
Objectives: To evaluate the performance of several risk-adjustment/stratification instruments in predicting healthcare utilization.
Study Design: Retrospective cohort analysis.
Methods: All adults empaneled in 2009 and 2010 (n = 83,187) in a primary care practice were studied. We evaluated 6 models: Adjusted Clinical Groups (ACGs), Hierarchical Condition Categories (HCCs), Elder Risk Assessment, Chronic Comorbidity Count, Charlson Comorbidity Index, and Minnesota Health Care Home Tiering. A seventh model combining Minnesota Tiering with ERA score was also assessed. Logistic regression models using demographic characteristics and diagnoses from 2009 were used to predict healthcare utilization and costs for 2010 with binary outcomes (emergency department [ED] visits, hospitalizations, 30-day readmissions, and highcost users in the top 10%), using the C statistic and goodness of fit among the top decile.
Results: The ACG model outperformed the others in predicting hospitalizations with a C statistic range of 0.67 (CMS-HCC) to 0.73. In predicting ED visits, the C statistic ranged from 0.58 (CMSHCC) to 0.67 (ACG). When predicting the top 10% highest cost users, the performance of the ACG model was good (area under the curve = 0.76) and superior to the others.
Conclusions: Although ACG models generally performed better in predicting utilization, use of any of these models will help practices implement care coordination more efficiently.
Am J Manag Care. 2013;19(9):725-732
For care coordination to be used effectively, it is important to identify which patients are likely to benefit from it. Little evidence exists as to which risk-stratification method performs better in predicting hospitalizations, emergency department visits, 30-day readmissions, and high-cost users.
There appeared to be good concordance among 6 different risk screening instruments in predicting hospitalization.
Adjusted Clinical Group classification was generally better than other models in predicting healthcare utilization.n
Focusing care coordination efforts within the medical home on patients likely to benefit most requires appropriate identification of the highest risk, highest utilizing patients.
With rising prevalence of chronic disease, patient-centered medical homes (PCMHs) were created to deliver higherquality, more cost-effective primary care in the United States.1 A key component of PCMH is the implementation of care coordination, especially if targeted to the right people.2 While some states have developed risk-adjusted care coordination payments, methods for effective identification of individuals likely to benefit from care coordination are not clear.3 Patients with multiple chronic conditions may see many providers, often in an uncoordinated manner, adding to their increased risk of high healthcare utilization through test duplication, edication management conflicts, and medical errors.4-6 These patients are also at increased risk for hospital admissions and emergency department (ED) visits. Primary care doctors may not routinely have the time or resources to properly manage these complex patients who have multiple chronic health problems. Thus, care coordination for the right patients could decrease unnecessary care and prevent adverse outcomes.
Most existing PCMH programs identify care coordination patients through physician referrals.7,8 However, this method may not identify all potential beneficiaries. Care coordination should be focused on patients who will benefit most, maximizing the impact on both quality and costs. Risk-stratification models can be efficient tools for both providers and health plans to screen populations and select individuals for care management programs who are at risk of future hospitalization and functional decline.9-11 Several methods using administrative data have been described to help with the identification of high-risk patients.12 However, it remains unclear which method will perform best in identifying patients in need of care coordination. To answer this important and fundamental question, we compared the performance of 6 common risk-adjustment methods in predicting hospitalizations, ED visits, 30-day readmissions, and high expenditures. This study extends the existing literature on patient selection for care coordination with results from direct comparison of the following models: Adjusted Clinical Groups (ACGs), Hierarchical Condition Categories (HCCs), Elder Risk Assessment (ERA), Chronic Comorbidity Count (CCC), Charlson Comorbidity Index, and Minnesota Health Care Home Tiering (MN Tiering). The identification of an optimal method may guide implementation of care coordination programs within PCMHs. Furthermore, identifying these predictable high-utilization cases may lead to other effective interventions.
A retrospective cohort analysis compared multiple methods to identify high-risk patients. These methods were applied to all primary care patients 18 years or older empaneled to the Employee and Community Health (ECH) practice (family medicine, primary care internal medicine, and community pediatric and adolescent medicine) at Mayo Clinic, Rochester, Minnesota, the community-focused primary care arm of a large integrated multispecialty group practice.
To be a patient of ECH, one must be a Mayo Clinic employee or live in the community, identify an ECH primary care provider, and be enrolled within his/her panel. Participants assigned to a primary care provider for all 12 months in 2009 (base year) and throughout 12 months or until death in 2010 (assessment year) based on the electronic medical record were included in the analysis. The organized development of care coordination within ECH occurred during 2011; hence, usual care (encounter driven) was provided to all patients during the study. Subjects were excluded if they refused consent for medical record review in accordance with Minnesota state law.
All information was electronically abstracted from the electronic medical record and administrative databases within Mayo Clinic’s health records system. Mayo Clinic maintains all electronic medical record information within 1 system, including inpatient and outpatient visit information.13
Demographic variables collected in the base year included age, sex, marital status, and insurance status. Age was grouped into 4 categories of 18 to 44 years, 45 to 64 years, 65 to 84 years, and more than 85 years. Insurance status was also grouped into 4 categories: public (Medicare or Medicaid), private, no insurance, and unknown. Diagnosis codes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM]) for each patient encounter, as well as utilization and cost information, were extracted from institutional billing data for year 2009. Number of hospital days was also retrieved from 2008 to calculate ERA scores. We included all diagnosis codes from hospitalizations, ED visits, and primary and specialty care evaluation and management visits.
Adjusted Clinical Groups. The ACG is a commonly used classification system based on administrative diagnosis data and developed by Johns Hopkins to measure morbidity.14 The ACG methodology was developed to predict the utilization of medical resources using the presence or absence of specific diagnoses from both inpatient and outpatient services for a specified period of time, along with age and sex, to classify each person into 1 of 93 discrete ACG categories with similar expected utilization patterns. Adjusted Clinical Groups can be used to improve accuracy and fairness in forecasting healthcare utilization15 and have been found to predict inpatient hospitalizations as well as or better than other case-mix tools in many health systems.9
Minnesota Tiering. The MN Tiering model is based on a product of ACGs, Major Extended Diagnostic Groups.3 MN Tiering is currently being used by the State of Minnesota to determine management fees for care coordination among medical home plans. The purpose of MN Tiering is to group patients into “complexity tiers” based on the number of major condition categories from which they suffer.3 The total sum of conditions is grouped into the following 5 patient complexity levels: low (tier 0): 0 conditions; basic (tier 1): 1 to 3 conditions; intermediate (tier 2): 4 to 6 conditions; extended (tier 3): 7 to 9 conditions; and complex (tier 4): 10 or more conditions.
Hierarchical Condition Categories. In 2004, the Centers for Medicare & Medicaid Services (CMS) implemented HCCs to adjust Medicare capitation payments for health expenditure risk of Medicare Advantage Plan (health maintenance organization) enrollees.16 In the HCC model ICD-9-CM diagnosis codes and demographic data for each patient are aggregated into 70 condition categories that contribute to a single risk score.17 Several studies among Medicare patients have provided evidence that HCC scores for risk adjustment can be effective at predicting hospitalizations.18,19 Elder Risk Assessment Index. The ERA Index was developed to identify patients at risk for hospitalization and ED visits in adults 60 years or older.20 The ERA Index incorporates a weighted score of age, sex, number of hospital days in the prior 2 years, and marital status, as well as selected medical conditions (diabetes, coronary artery disease, congestive heart failure, stroke, chronic obstructive pulmonary disease,and dementia).20 The minimum score on the index is –1 and the maximum score possible is 34.
Chronic Condition Count. The CCC method was derived by Naessens and colleagues21 from a modification of the method of Hwang and colleagues,22 based on the publicly available Agency for Healthcare Research and Quality’s Clinical Classification Software. The CCC method is more comprehensive than most comorbidity counts. The total sum of chronic conditions was grouped into 6 categories: 0, 1, 2, 3, 4, and 5 or more. Comorbidity counts have been shown to be associated with high annual costs as well as persistence in high costs.21
Charlson Comorbidity Index. The Charlson Comorbidity Index, originally derived to classify comorbidities affecting 1-year mortality in cancer patients, sums weights for 17 specific conditions.23 In our study, the sum of Charlson Comorbidity Index counts was used to predict future outcomes. The performance of the Charlson Comorbidity Index in predicting poor outcomes has been assessed in various large populations, and its validity as a prognostic measure of outcomes has been consistently demonstrated.24
Hybrid Model. A care coordination patient enrollment process was developed in our primary care practice based on combining MN Tiering and the ERA score as a hybrid model. Patients eligible for enrollment in care coordination included all patients in MN tier 4 (individuals with more than 10 comorbid conditions) and those in MN tier 3 with ERA scores greater than 10. Although all of the previously mentioned risk-stratification instruments have been validated, it remains unclear how the hybrid method of combining MN Tiering and ERA score will compare with them.
PDF is available on the last page.