Currently Viewing:
The American Journal of Managed Care September 2014
Impact of Atypical Antipsychotic Use Among Adolescents With Attention-Deficit/Hyperactivity Disorder
Vanja Sikirica, PharmD, MPH; Steven R. Pliszka, MD; Keith A. Betts, PhD; Paul Hodgkins, PhD, MSc; Thomas M. Samuelson, BA; Jipan Xie, MD, PhD; M. Haim Erder, PhD; Ryan S. Dammerman, MD, PhD; Brigitte Robertson, MD; and Eric Q. Wu, PhD
Effective Implementation of Collaborative Care for Depression: What Is Needed?
Robin R. Whitebird, PhD, MSW; Leif I. Solberg, MD; Nancy A. Jaeckels, BS; Pamela B. Pietruszewski, MA; Senka Hadzic, MPH; Jürgen Unützer, MD, MPH, MA; Kris A. Ohnsorg, MPH, RN; Rebecca C. Rossom, MD, MSCR; Arne Beck, PhD; Kenneth E. Joslyn, MD, MPH; and Lisa V. Rubenstein, MD, MSPH
Is All "Skin in the Game" Fair Game? The Problem With "Non-Preferred" Generics
Gerry Oster, PhD, and A. Mark Fendrick, MD
Targeting High-Risk Employees May Reduce Cardiovascular Racial Disparities
James F. Burke, MD, MS; Sandeep Vijan, MD; Lynette A. Chekan, MBA; Ted M. Makowiec, MBA; Laurita Thomas, MEd; and Lewis B. Morgenstern, MD
HITECH Spurs EHR Vendor Competition and Innovation, Resulting in Increased Adoption
Seth Joseph, MBA; Max Sow, MBA; Michael F. Furukawa, PhD; Steven Posnack, MS, MHS; and Mary Ann Chaffee, MS, MA
Out-of-Plan Medication in Medicare Part D
Pamela N. Roberto, MPP, and Bruce Stuart, PhD
New Thinking on Clinical Utility: Hard Lessons for Molecular Diagnostics
John W. Peabody, MD, PhD, DTM&H, FACP; Riti Shimkhada, PhD; Kuo B. Tong, MS; and Matthew B. Zubiller, MBA
Should We Pay Doctors Less for Colonoscopy?
Shivan J. Mehta, MD, MBA; and Scott Manaker, MD, PhD
Long-term Glycemic Control After 6 Months of Basal Insulin Therapy
Harn-Shen Chen, MD, PhD; Tzu-En Wu, MD; and Chin-Sung Kuo, MD
Characteristics Driving Higher Diabetes-Related Hospitalization Charges in Pennsylvania
Zhen-qiang Ma, MD, MPH, MS, and Monica A. Fisher, PhD, DDS, MS, MPH
Quantifying Opportunities for Hospital Cost Control: Medical Device Purchasing and Patient Discharge Planning
James C. Robinson, PhD, and Timothy T. Brown, PhD
Effects of a Population-Based Diabetes Management Program in Singapore
Woan Shin Tan, BSocSc, MSocSc; Yew Yoong Ding, MBBS, FRCP, MPH; Wu Christine Xia, BS(IT); and Bee Hoon Heng, MBBS, MSc
Currently Reading
Predicting High-Need Cases Among New Medicaid Enrollees
Lindsey Jeanne Leininger, PhD; Donna Friedsam, MPH; Kristen Voskuil, MA; and Thomas DeLeire, PhD

Predicting High-Need Cases Among New Medicaid Enrollees

Lindsey Jeanne Leininger, PhD; Donna Friedsam, MPH; Kristen Voskuil, MA; and Thomas DeLeire, PhD
Self-reported health measures embedded in a Medicaid application can comprise a predictive model identifying new and returning enrollees at risk of high healthcare utilization.
with representing the mean value of the predicted probabilities resulting from the logistic regression model of the dependent variable event on a given set of predictors. Alternately stated, the discrimination slope is the difference between the average predicted probabilities of sample members experiencing the outcome and the average predicted probabilities of sample members not experiencing the outcome. Improvements in the discrimination slope, termed integrated discrimination improvement (IDI), are reported both as the level difference between a baseline and augmented model and as the percent improvement associated with the augmented model relative to the baseline model. We employed a split-sample approach to compute all C-statistics and discrimination slopes. This approach involves randomly dividing the sample into 2 subsamples, the first of which is used to fit the model (n = 17,043). The resulting estimates are then applied to the withheld validation sample (n = 17,044), with which the metrics of interest and associated 95% confidence intervals are computed using a 500 replicate bootstrap procedure. We also bootstrapped the difference in the C-statistic and discrimination slope between each augmented model and the baseline model to determine the statistical significance of any marginal gain in predictive performance.

Finally, we computed measures of sensitivity, specificity, and positive and negative predictive values associated with the baseline demographic model compared with the specification employing all the HNA measures. In keeping with the related literature, we chose the 50th, 75th, and 90th percentiles in the predicted risk distribution as our threshold values.15,21 These results are particularly important for case-finding applications, as stakeholders must decide upon a risk threshold at which a program (or additional screening measure) will be administered.


Descriptive statistics are displayed in Table 1. Excepting cancer and top ED use, each condition was positively associated with membership in the top utilization and top cost deciles. Similarly, the behavior, prescription drug, and previous year’s utilization measures were all positively correlated with membership in the top utilization and cost deciles. Both access to care measures exhibited modest negative associations with membership in the top ED decile and modest positive associations with the hospitalization and cost outcomes.

Predictive performance of the multivariate specifications is displayed in Tables 2 and 3. For top ED utilization, Cstatistics ranged between 0.67 for the baseline specification and 0.74 for the richest HNA specification. The past-year utilization and condition specifications provided the greatest incremental predictive improvement over baseline; both behaviors and prescriptions were also associated with appreciable increases in predictive performance. In contrast, the access-to-care domain provided no meaningful improvement in predictive ability. Comparing discrimination slopes yielded similar conclusions (Table 3).

Predictive performance, measured by either the C-statistic or the discrimination slope, was lowest for the hospitalization outcome (C-statistic for richest specification = 0.67). Here again the conditions and utilization domains offered the greatest incremental increases in predictive ability over baseline (C-statistic of 0.65 and 0.63 for conditions and past utilization specifications, respectively, vs 0.59 for the baseline model). Prescriptions and behaviors both contributed meaningful improvements in predictive accuracy over baseline (IDI of 90% and 60%, respectively), while the access-to-care specification constituted a negligible (albeit statistically significant) improvement.

Similar to the progression of models predicting high ED use, the inclusion of the HNA predictors improved the performance of models predicting membership in the top cost decile sufficiently, such that the richest specification met the Hosmer-Lemeshow rule-of-thumb threshold for acceptability. Specifically, the C-statistics ranged from 0.61 for the baseline model to 0.72 for the richest HNA specification. For the cost outcome the block of condition predictors was associated with the greatest marginal improvement (C-statistic of 0.69; IDI of 267%). In contrast to the other 2 outcomes, the past year’s utilization specification was ranked third with respect to incremental performance improvement; however, it is important to note that while the specification’s relative performance was lower, the level of incremental predictive ability remained considerable (IDI of 153%). Also of note is that the relative contribution of prescription drugs was much higher for the cost outcome compared with the other 2 outcomes (IDI of 248% for cost, compared with 48% and 90% for ED and inpatient utilization, respectively). Importantly, the incremental predictive contribution of the HNA measures was highest for the cost outcome (IDI of 413% for specification including all HNA measures).

Table 4 displays the sensitivity, specificity, and positive and negative predictive values by risk threshold associated with the baseline model and the specification including all HNA measures. For each outcome, the HNA specification was associated with appreciable improvements in sensitivity—especially at the 75th and 90th percentiles—with no resulting decreases in specificity. Similarly, the HNA specifications improved positive predictive value across all thresholds, with especially large improvements seen at the 90th percentile, with no associated decline in negative predictive value. The tradeoff between sensitivity and specificity at different risk thresholds is striking, and underscores the tensions inherent in choosing a threshold at which to target case-finding applications of the underlying predictive model. As is expected given the low prevalence of the outcome measures and in keeping with similar studies,15 positive predictive values were fairly low, even at the 90th percentile of the risk distribution (HNA specification: 0.27 for the ED outcome; 0.22 for any hospitalization; and 0.30 for high cost).

Sensitivity Analysis and Limitations

We also performed a number of sensitivity analyses to assess the robustness of these findings across additional specifications. First, we estimated an additional specification including a dummy variable reflecting having a comorbidity (2 or more enumerated conditions) in addition to the full set of HNA measures.22,23 This additional covariate added no incremental predictive power. Second, we estimated models employing top-decile, ambulatory care–sensitive ED visits as the ED outcome measure, using the algorithm created by Billings and colleagues.24 Results were very similar to the specifications modeling top-decile total ED visits (available from the authors upon request).

A potential limitation of our analysis is that it was constrained to use the particular HNA as designed by the Wisconsin DHS. The HNA did not include several of the best established predictors of future health costs and utilization, including general health status and functional and activity limitations, and all-cause past utilization over the past year.25-28 Such omissions, therefore, suggest that our estimates represent a conservative estimate of the potential predictive ability associated with HNA administration to new adult Medicaid enrollees.


We found that a simple, self-reported health needs assessment collected via a Medicaid enrollment system was meaningfully predictive of future healthcare utilization for a sample of new childless adult enrollees. For the outcomes of high ED utilization and high cost, the HNA measures combined with demographic measures demonstrated acceptable predictive performance and were associated with large incremental predictive improvements over demographic variables alone for each of the 3 outcomes, with the largest incremental improvements achieved for the high cost outcome. It is encouraging that the predictive performance of the HNA approaches that achieved in a claims-based study on a comparable Medicaid population in Vermont.15 Two corroborating studies using within-sample comparisons found that the predictive ability of a self-reported health screener approaches but does not quite meet that exhibited by recent claims history.29,30

The Wisconsin experience shows that the use of HNAlike instruments via Medicaid application systems holds great promise for prospective assessment of new enrollees. Medicaid agencies deciding whether and how to use an HNA-like instrument in predictive modeling applications face several important issues, however. As is the case with all risk adjustment applications, agencies will need to work assiduously to ensure that provider groups believe in the legitimacy and fairness of an HNA-based risk model. Medicare’s long standing experience with using survey data as a frailty risk adjuster could serve as an instructive guide in navigating this and other issues inherent in implementing survey-based risk adjustment.27,28,31,32 Agencies interested in using an HNA to target case management and/or other specialized services must be mindful that the positive predictive value of the resulting model is likely to be low. In recognition of this limitation, traditional disease management programs often use predictive modeling as a first screen, complemented by a subsequent screen typically involving follow-up by a nurse case manager.29 Additionally, conducting a business case analysis similar to that pioneered by Billings and colleagues33,34 would give stakeholders a sense of the likely fiscal impacts associated with a case-finding intervention employing a predictive model. We conclude with a final note that, as was the case in Wisconsin, HNA instruments are often designed for several purposes, many of which are not predictive in nature.35 Designing an effective HNA will require balancing its predictive goals with

the demands of its other stated objectives.

Author Affiliations: Department of Health Policy and Administration, School of Public Health, University of Illinois at Chicago, IL (LJL); Population Health Institute, School of Medicine and Public Health, University of Wisconsin–Madison (DF, KV); and McCourt School of Public Policy, Georgetown University, Washington, DC (TD).

Funding Source: This study received financial support from the Robert Wood Johnson SHARE initiative.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (LJL); acquisition of data (DF, KV); analysis and interpretation of data (LJL, KV, TD); drafting of the manuscript (LJL, TD); critical revision of the manuscript for important intellectual content (DF, KV, TD); statistical analysis (LJL); provision of study materials or patients (DF); obtaining funding (LJL); administrative, technical, or logistic support (DF); and supervision (LJL, DF, TD).

Address correspondence to: Lindsey Jeanne Leininger, PhD, Assistant Professor, Department of Health Policy and Administration, University of Illinois – Chicago School of Public Health, 1603 W Taylor St, Chicago, IL 60612. E-mail:
1. Medicaid benefits: targeted case management. The Henry J. Kaiser Family Foundation website. Accessed June 19, 2013.

2. Llanos K, Rothstein J, Dyer MB, Bailit M. Physician pay-forperformance in Medicaid: a guide for states. Center for Health Care Strategies website. Accessed February 19, 2012.

3. Winkelman R, Damler R. Risk adjustment in state Medicaid programs. Society of Actuaries website. Accessed February 18, 2012.

4. Klein K, Glied S, Ferry D. Entrances and exits: health insurance churning, 1998-2000. Issue Brief (Commonw Fund). 2005;855:1-12.

5. Sommers BD. 2009. Loss of health insurance among non-elderly adults in Medicaid. J Gen Intern Med. 2009;24:1-7.

6. Wisconsin Department of Health Services. Health Insurance for Childless Adults. Waiver Proposal Submitted to the Centers for Medicare and Medicaid Services. Madison, WI: Wisconsin Department of Health Services; 2008.

7. Perrin NA, Stiefel M, Mosen DM, Bauck A, Shuster E, Dirks EM. Self-reported health and functional status information improves prediction of inpatient admissions and costs. Am J Manag Care. 2011;17(12):e472-e478.

8. Maciejewski ML, Liu CF, Derleth A, McDonnell M, Anderson S, Fihn SD. The performance of administrative and self-reported measures for risk adjustment of Veterans Affairs expenditures. Health Serv Res. 2005;40(3):887-904.

9. Byrd VL, Verdier J. Collecting, Using, and Reporting Medicaid Encounter Data: A Primer for States. Washington, DC: Mathematica Policy Research; 2011.

10. Rudd RE, Moeykens BA, Colton TC. Health and literacy: a review of medical and public health literature. [Originally in The Annual Review of Adult Learning and Literacy. 1999;1(5).] National Center for the Study of Adult Learning and Literacy website. Accessed April 19, 2013.

11. Knutson D, Bella M, Llanos K. Predictive modeling: a guide for state purchasers. Center for Health Care Strategies website. Published 2009. Accessed April 19, 2013.

12. Verdier JM, Byrd V, Stone C. Resource paper – Enhanced primary care case management programs in Medicaid: issues and options for states. Center for Health Care Strategies website. Published 2009. Accessed April 19, 2013.

13. Ash AS, Ellis RP. Risk-adjusted payment and performance assessment for primary care. Med Care. 2012;50(8):643-653.

14. Medicare—hospital quality initiatives—outcome measures. CMS website. Accessed April 19, 2013.

15. Weir S, Aweh G, Clark RE. Case selection for a Medicaid chronic care management program. Health Care Financ Rev. 2008;30(1):61-74.

16. Shwartz M, Ash AS. Empirically evaluating risk adjustment models. In: Iezzoni LI, ed. Risk Adjustment for Measuring Health Care Outcomes, 4th ed. Chicago, IL: Health Administration Press; 2013:249-300.

17. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157-172.

18. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138.

19. Hosmer DW, Lemeshow S. Applied Logistic Regression, Second Edition. San Francisco, CA: John Wiley & Sons; 2000.

20. Pencina MJ, D’Agostino RB, Vasan RS. Statistical methods for assessment of added usefulness of new biomarkers. Clin Chem Lab Med. 2010;48(12):1703-1711.

21. Billings J, Mijanovich T. Improving the management of care for high-cost Medicaid patients. Health Aff (Millwood). 2007;26(6):1643-1654.

22. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383.

23. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27.

24. Billings J, Parikh N, Mijanovich T. Emergency department use: the New York story. Issue Brief (Commonw Fund). 2000;434:1-12.

25. Fleishman JA, Cohen JW. Using information on clinical conditions to predict high-cost patients. Health Serv Res. 2010;45(2):532-52.

26. Fleishman JA, Cohen JW, Manning WG, Kosinski M. Using the SF-12 health status measure to improve predictions of medical expenditures. Med Care. 2006; 44(5 suppl):I54-I63.

27. Gruenberg L, Kaganova E, Rumshiskaya A. Updating the Social/HMO AAPCC. Report to the Health Care Financing Administration. Cambridge, MA: DataChron Health Systems Inc.; 1993.

28. Gruenberg L, Silva A, Leutz W. An Improved Disability-based Medicare Payment System for the Social/HMO. Report to the Health Care Financing Administration. Cambridge, MA: The Long-Term Care Data Institute; 1993.

29. Andersen DR, Mangen DJ, Grossmeier JJ, Staufacker MJ, Heinz BJ. Comparing alternative methods of targeting potential high-cost individuals for chronic condition management. J Occup Environ Med. 2010;52(6):635-646.

30. Chaudhry S, Jin L, Meltzer D. Use of a self-report-generated Charlson Comorbidity Index for predicting mortality. Med Care. 2005; 43(6):607-615.

31. Robinson JM, Karon SL. Modeling Medicare costs of PACE (Program of All-Inclusive Care for the Elderly) populations. Health Care Financ Rev. 2000;21(3):149-170.

32. Kautter J, Ingber M, Pope GC. Medicare risk-adjustment for the frail elderly. Health Care Financ Rev. 2008;30(2):83-93.

33. Billings J, Dixon J, Mijanovich T, Wennberg D. Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. BMJ. 2006; 333(7563):327.

34. Billings J, Mijanovich T. Improving the management of care for high-cost Medicaid patients. Health Aff (Millwood). 2007;26(6): 1643-1654.

35. Goetzel RZ, Staley P, Ogden L, et al. A framework for patientcentered health risk assessments – providing health promotion and disease prevention services to Medicare beneficiaries. CDC website. Published 2011. Accessed February 26, 2013.
Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up