Hybrid approaches allow for clinician input into case finding for care management, but training and monitoring is required to protect against unintentional biases.
Objectives: Appropriate selection of patients is key to the success of care management programs (CMPs). Hybrid patient selection approaches, in which large data assets are culled to develop a list of patients for more targeted clinical review, are increasingly common. We sought to describe the patient and practice characteristics associated with high-risk patient identification and selection for a CMP during clinical review, and to explore variation across primary care practices.
Study Design: Retrospective cohort study.
Methods: Standardized estimates of Medicare beneficiaries identified as high risk for poor outcomes and high medical expense, and appropriate for a CMP within a large Pioneer Accountable Care Organization, were developed using mixed effects logistic models. Study subjects were 2685 Medicare beneficiaries aged over 18 (includes individuals eligible for Medicare due to a disability) aligned to 35 primary care practices in 2013.
Results: Independent predictors of patient identification as high risk include older age; higher risk score; recent increases in medical conditions; higher numbers of medical hospitalizations, skilled nursing facility days, and primary care physician visits; and shorter relationships with the primary care physician. Older age, and lower income, but no prior hospice use were independently associated with patient selection for a CMP among the subset of patients identified as being high risk. Adjusted predicted percents of high-risk patients varied significantly across practices overall and for 5 of the 6 patient characteristics that were independently associated with identification as high risk.
Conclusions: Inconsistency in high-risk patient identification and selection for a CMP may reflect differences in practice resources, but also highlight the need for continual training and feedback in order to protect against unintentional biases.
Am J Manag Care. 2016;22(5):358-365
Clinician review within hybrid case selection approaches includes decisions about which patients are high risk and, among high-risk patients, identification of the subset of patients most appropriate for a care management program (CMP).
The observation that health expenditures are concentrated within a relatively small subset of the population is well known by now, with 5% of the population accounting for 50% of medical expense in any given year. One-third of the individuals in this group—frequently those with poor health status and multiple chronic conditions—remain high-cost the following year,1,2 making them appropriate for primary care—based care management approaches.3,4 However, although effective care management targeted at the right individuals has been shown to moderate costs and utilization,5,6 inconsistent results have highlighted the need to appropriately select patients.7
There are a host of available case-finding tools and algorithms that can be used to identify individuals appropriate for care management; nevertheless, there is also increasing understanding that these tools have limitations.8,9 Many case-finding tools are proprietary “black boxes” that create a single score to rank patients according to risk.10 Additionally, these tools can vary considerably in terms of their input criteria, and there is no standardization of definitions across tools8—most importantly, the input criteria cannot be modified to meet specific program characteristics and operational realities.
Data timing and completeness are a challenge for all case-finding algorithms, especially those based on paid claims. Delays in data capture can result in the identification of patients who are no longer enrolled within the health plan or practice, as well as identification of patients who no longer require intervention.8,11,12 Incomplete data, such as removal of substance use, HIV/AIDS, and in some states, mental health claims,13 paint incomplete pictures of patient needs. In addition, demographic and psychosocial characteristics are often incomplete or missing from readily available administrative data (eg, poverty, education, living situation). The absence of patient psychosocial data is particularly challenging, as these data could help to identify cohorts of patients who would be most likely helped by care management programs (CMPs)5,14; for example, addressing cost-related medication underuse among low-income patients may improve diabetes control and reduce inappropriate emergency department utilization.15 Although many healthcare systems do collect psychosocial data on patients, system alignment issues often preclude use of more granular and descriptive data for case finding.
Because of these limitations, case finding based on administrative data is typically not the only method used.16 Patient health-risk assessments and clinician referrals are techniques frequently used to identify patients for case management and can provide more detailed and actionable data, but they require time and resources to implement.3 Often, multiple case-finding methodologies, including predicted risk scores, utilization triggers/markers, medication adherence and gaps in care, and clinician referrals, are used in concert.11,14
Case-finding approaches that rely on multiple methodologies are typically termed “hybrid approaches” and can vary widely, but generally refer to the use of multiple methodologies—often in sequence—to refine case finding. Within this evaluation, we studied a sequential approach to case finding that uses an algorithm to comb through administrative claims and identify a preliminary set of potentially high-risk patients. This preliminary set of patients is then reviewed by physicians and nurse care managers who make the final decision of whether to refer a patient for care management. This sequential hybrid approach is thought to be reliable3 and can offer a number of advantages.
Clinician referral, within the boundaries of a pre-selected list, focuses the referral process and reduces the burden on clinicians, while at the same time ensuring primary care practice and physician (PCP) involvement. This involvement may increase buy-in of the program, which is especially important for programs embedded within physician practices. Clinician input can also help to identify patients with more modifiable health risks, including those with sub-optimal health literacy and health insight, social and home environment challenges, and/or inadequate coping skills and financial resources.11,17,18 For example, primary care physicians are more likely to describe patients with poorly controlled diabetes, inadequate insurance, or with mental health or substance use disorders, as complex.19
However, clinician referral may also introduce unanticipated variations in patient selection.8 Often, there are no specific criteria that physicians are asked to consider when identifying appropriate patients, leaving the door open to unintentional biases.14 Appropriate clinician referral into CMPs requires that clinicians have a clear understanding of the characteristics that place patients at higher risk for poor health outcomes, as well as the benefits the CMP could provide.
In this study, we sought to understand the unique patient characteristics associated with both patient identification as high risk for poor outcomes and subsequent clinician referral to care management. We also sought to describe variations in patient identification and selection for care management across PCPs overall, and for important patient characteristics. We examined the selection decisions during the second year of the CMP to avoid biases associated with the ramp-up of the program.
The Partners Healthcare Pioneer Accountable Care Organization (ACO)—1 of 4 Pioneer ACOs operating in eastern Massachusetts—is among the largest nationwide, and the only Pioneer ACO that exclusively aligns patients to primary care physicians. One of the signature initiatives within the Partners Pioneer ACO is the CMP. The CMP is a longitudinal program that relies on nurse care managers working in conjunction with primary care physicians to improve care quality and control healthcare utilization for referred patients. After a successful trial within the Medicare High Cost Beneficiary Demonstration Program,20 the CMP was implemented within all Partners Healthcare PCPs under the Medicare Pioneer ACO contract, starting in January 2012. Partners Healthcare uses a hybrid approach for patient selection, combining a claims-based algorithm with clinician review to identify high-risk patients who could benefit from the CMP.
The population for this study included traditional Medicare beneficiaries within the Partners Healthcare Pioneer ACO who were identified using a claims-based algorithm as potentially high risk for poor outcomes in the second year of the program (2013). By the second year, all practices had experience performing the clinical review and had 1 or more ACO patients who were actively engaged in CMP. In total, 7651 (12%) of 63,629 Medicare patients were identified by the Partners algorithm. Among individuals with a predictive risk score for future total medical expense via Optum ImpactPro software version 6.0 (Optum, Inc, Eden Prairie, Minnesota) that exceeds a minimum threshold, the Partners algorithm uses combinations of chronic conditions and patterns of healthcare utilization to identify individuals potentially at high risk and appropriate for subsequent clinical review. The Partners algorithm was developed with extensive input from an advisory committee of operational and clinical experts within the organization. During the clinical review process, PCPs receive detailed information on chronic conditions, utilization patterns, and predictive scores that triggered identification by the algorithm. PCPs had considerable latitude when selecting patients for care management, but were provided additional guidance through reference guides and regional medical directors who received orientation and training on the list review process.
In order to study clinician review decisions during the second year, this analysis excluded 2605 patients who had been identified and reviewed in the prior year, as well as 1267 patients who had moved, died before the review process, were not community-dwelling (eg, residing in long-term care), switched to a Medicare Advantage plan, or did not have a relationship with a Partners primary care clinician. Lists of the remaining 3779 patients (5.9% of 63,629 total Medicare ACO patients) were distributed to practice management organizations and, eventually, to PCPs and clinicians for final identification of individuals appropriate for the primary care—based CMP. We then further excluded 788 patients aligned to 44 PCPs that did not make differential review decisions (eg, identifying none or all patients as high risk for poor outcomes or requiring care management), as well as 306 patients who were not classified by the practice prior to the release of the 2014 list. Thus, the final study population consisted of 2685 patients who received primary care within 35 practices.
The dependent variables are the decisions made by primary care clinicians and their practices regarding whether patients were identified as high risk for poor outcomes and whether patients could benefit from CMP.
The independent variables are listed in and include patient demographic characteristics, area-level poverty (American Community Survey 2012 5-Year Estimates21), prospective risk score and changes in the number of conditions from calendar year 2012 to the year directly preceding the first list review (May 1, 2012-April 30, 2013) (CMS-Hierarchical Condition Categories Risk Adjustment Model22), healthcare utilization (ED visits, medical hospitalizations, skilled nursing and hospice service use) in the 12 months prior, and characteristics of the patient’s relationship with their PCP. Practice characteristics include practice size, level of affiliation with the healthcare organization, whether the practice is a community healthcare center, years of experience with care management, CMP capacity, and level of practice readiness for patient-centered medical home transformation. Models were also adjusted for primary care physician characteristics, including primary care physician age, full-/part-time status, and duration of employment within the healthcare organization.
Descriptive analyses of the patient- and practice-level characteristics associated with patient identification as high risk for poor outcomes and selection for care management were explored using bivariate analyses. Standardized estimates of patient identification as high risk and selection for CMP were developed using logistic regression models with robust standard errors clustering at the practice. For regression models, we used the subset of explanatory variables that were independently predictive of either patient identification as high risk for poor outcomes or selection for care management at P <.20, and used a similar approach to develop standardized estimates at the practice level.
We explored the extent to which practices consistently identified patients with specific characteristics as high risk for poor outcomes. Specifically, we examined 6 patient characteristics that were independently associated with being considered high risk for poor outcomes and that were present in at least 10% of patients in our sample; similar models were used for each, as described above, to develop standardized practice-level estimates of identification as high risk among patients with and without the characteristic. Full interaction models did not enhance predictive ability and decreased the number of practices with stable estimates; therefore, we only included interactions between PCP and the characteristic of interest in our final models.
Global tests were used to test for differences in overall practice marginal estimates and interaction marginal estimates. Chi-square statistics were used to describe the relationship between overall practice marginal estimates and practice estimates for individuals with specific characteristics.
The study was approved by the Partners Institutional Review Board.
Among Medicare ACO patients identified by the claims-based algorithm and who also passed an administrative review (roughly 5.9% of total Medicare ACO patients), 86.3% were thought to be high risk for poor outcomes by their primary care providers; 77.2% of these patients were referred to a nurse-led CMP within the PCP.
A number of patient characteristics were independently associated with patient identification as high risk during the clinician review. Specifically, independent predictors of high risk include advanced age, higher prospective risk score, a recent increase in diagnosed conditions, and higher numbers of medical hospitalizations and days in skilled nursing facility (SNF) care in the prior year. Patients identified as high risk also were more likely to have more visits with their PCP in the prior 12 months, but shorter relationships with their PCP than patients not identified as high risk. The patient characteristics with the strongest independent association with being identified as high risk include age 85 years or older (93.1%), 2 or more hospitalizations in the prior year (91.5%), and 30 or more SNF days during the prior year (92.7%) (Table 1).
Among high-risk patients, advanced age was also an independent predictor of patient referral to a CMP. However, when determining whether high-risk patients would benefit from care management, clinicians were also more likely to select individuals without prior hospice use and those living in lower-income areas. The adjusted proportions of high-risk patients 85 years or older selected for care management was 80.1%, and between 81.2% and 81.9% among patients residing in areas with more than 10% of individuals living in poverty (Table 1).
Lists of patients identified by the claims-based algorithm were reviewed within 35 practices where patients received their primary care. On average, practices reviewed 76 algorithm-identified Medicare ACO beneficiaries (range = 5-248) in 2013. Adjusted proportions of patients identified as high risk and referred to a CMP varied according to practice characteristics. Specifically, after adjusting for patient characteristics and other practice characteristics, practices with 6 or more physicians (88.8%), non—community health centers (87.3%), and practices without the capacity to provide care management to all patients (93.5%) had the largest adjusted proportions of algorithm-identified ACO patients identified as high risk during clinical review. Among high-risk patients, adjusted proportions selected for care management were highest within practices employed by the health delivery organization (80.8%) and non–community health center practices (78.4%) ().
After adjusting for patient and primary care physician characteristics and controlling for the clustering of patients within PCPs, practices varied significantly in terms of both the adjusted proportion of patients identified as high risk (36% to 99%; P <.001) and the proportion of patients referred to the CMP (18% to 96% ; P <.001) ().
We further explored the extent to which patients with specific characteristics were consistently categorized as high risk across practices. For this exploratory analysis, we selected the 6 patient characteristics that were both independently associated with patient identification as high risk and were present in at least 10% of the patient population. Practices varied significantly in how they categorized patients for 5 of 6 patient characteristics studied (, available at www.ajmc.com).
Hybrid patient identification approaches, that use algorithms to identify a subset of patients who warrant more careful clinical review, hold promise as a feasible method for incorporating clinician judgment and knowledge of patient psychosocial needs into case findings. However, we know little about the clinical review process, including the criteria used by PCPs to identify patients as high risk, how practices identify which high-risk patients would be appropriate for care management, and how these decisions differ across PCPs.
Consistent with findings from our interviews of primary care physicians and care managers,18 we found that primary care teams used additional information about their patients’ clinical profiles to decide whether a patient on an algorithm-generated list was high risk. Patients identified as high risk tended to be older, sicker, and with greater healthcare utilization, including primary care visits, than patients not identified as such. Patients with shorter relationships with their primary care physician were more likely to be identified as high risk, suggesting that PCPs might either have different thresholds for identifying patients who are new to their practice as high risk or may not have had the opportunity to establish necessary supports for their newer patients. Lack of a long-term patient—PCP relationship may compromise effective communication with patients and their caregivers, which is particularly important in older, complex patients.23
As a second step, primary care teams were asked to identify which high-risk patients were appropriate for care management, and when selecting, patient clinical need was far less of a consideration. In addition to patient age and no prior hospice use, patient area—level poverty was independently associated with selection of high-risk individuals for CMP services. Thus, although predicted risk and prior utilization are important for identifying patients who are high risk, they are not the primary factors associated with the decision to refer patients to a CMP.
Our study extends prior work by clearly differentiating clinical assessments of whether patients are high risk from determinations of whether high-risk patients would be appropriate for CMP services. Additionally, our findings support other studies that have found that one of the benefits of including clinician input is the ability to consider nonclinical factors, such as modifiable health risks, patient health literacy, social and home environment challenges, social support, or inadequate coping skills and financial resources.11,17,18,24 Patients appropriate for care management often have issues with medical decision making and care coordination in addition to poorly controlled disease.19 Physicians may also preferentially select patients who they feel are more willing to participate in a CMP.24
We found considerable variation in the overall adjusted proportions of patients identified as high risk across practices, as well as significant variations in practice-level identification rates for specific cohorts of patients, suggesting that practices may not use the same criteria to identify high-risk patients. Although practices were provided with central guidance on how to perform the clinical review, patient-by-patient decisions are left to the practices. Some of this diversity in identification of high-risk patients may appropriately reflect unmeasured differences in practice resources and access to specialty care and services, such as behavioral health specialists and addiction counselors, for their patient population. Differences could also reflect variations in program implementation, such as nurse skill levels or comfort with different patient populations. Care management, by nature, is a diverse intervention for individuals with complex healthcare needs.5 The multifaceted nature of care management not only complicates the identification of patients, but may bias selection toward patients whose needs can be met by the skills of the care manager available to the practice. Training for care managers and feedback to practices regarding the benefits and limitations of CMP may help minimize unintentional disparities in patient selection.
We excluded 43 PCPs (763 patients) that did not appear to differentially classify Medicare ACO patients from this analysis; it is unclear whether these practices made appropriate decisions or elected to not fully participate in the review process. Training and feedback may be particularly important for practices without differential review decisions (eg, those practices where all or no patients were identified as high risk). Alternatively, lower-intensity versions of clinician review may be necessary to ensure that all practices are able to participate in a meaningful way. In addition, this study was conducted within a single large provider network, and therefore may not be generalizable to other systems.
Hybrid approaches that use quantitative methods to prospectively identify a list of patients for subsequent clinical review are commonly used to identify and select patients for care management.9 Although clinical review may enhance PCP buy-in and allow for the consideration of patient psychosocial factors and appropriateness for care management, it may also introduce biases in patient selection. This study found evidence of significant variation across practices, both in the identification and the selection of high-risk patients for care management. However, it is not yet clear if this variation reflects differences in patient need and existing clinical services or biases related to training and feedback. CMPs using hybrid approaches need to provide adequate training and feedback to primary care clinicians and practices; this additional support could ensure that unintentional biases do not impact decision making and that selection decisions evolve to target patients at the highest risk for future poor outcomes and high medical expense who can benefit most from care management. Finally, additional on-going research is necessary to identify the characteristics of patients most successful in primary care—based care management. The findings from this work will inform the design of care management interventions, particularly in relation to the initial selection of patients for CMPs.
Partners HealthCare Population Health Management funded this study but was not involved in the analysis and interpretation of the data. The authors would like to acknowledge their operational colleagues for their support with this research.
Author Affiliations: Mongan Institute for Health Policy (CV, JS, JH) and Division of General Medicine (EW), Massachusetts General Hospital, Boston, MA; Population Health Management, Partners HealthCare (CV, EW, NM, TGF), Boston, MA; Department of Medicine, Harvard Medical School (CV, JH, TGF), Boston, MA; Department of Epidemiology and Biostatistics, University of California at San Francisco (RB), San Francisco, CA; Los Angeles County Department of Health Services (CH), Los Angeles, CA.
Source of Funding: This study is based on a quality improvement investigation funded by Partners HealthCare.
Author Disclosures: Dr Brand was a biostatistical consultant for the team that prepared the manuscript. Drs Vogeli, Hsu, Weil, Mohta, and Ferris and Ms Sprit are employees of Partners HealthCare or one of its affiliated hospitals. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (CV, NM, CH, EW, TGF); acquisition of data (CV, JS, EW); analysis and interpretation of data (CV, JS, RB, JH, NM, CH, EW); drafting of the manuscript (CV); critical revision of the manuscript for important intellectual content (JS, RB, JH, NM, CH, EW, TGF); statistical analysis (CV, JS, RB); obtaining funding (CV); administrative, technical, or logistic support (NM); and supervision (CV, JH, TGF).
Address correspondence to: Christine Vogeli, PhD, Mongan Institute for Health Policy, Massachusetts General Hospital, 50 Staniford St, 9th Fl, Boston, MA 02114. E-mail: firstname.lastname@example.org.
1. Cohen SB. The concentration and persistence in the level of health expenditures over time: estimates for the U.S. population, 2011—2012 [statistical brief #449]. Agency for Healthcare Research and Quality website. http://meps.ahrq.gov/mepsweb/data_files/publications/st449/stat449.pdf. Published September 2014. Accessed June 1, 2015.
2. Naessens JM, Stroebel RJ, Finnie DM, et al. Effect of multiple chronic conditions among working-age adults. Am J Manag Care. 2011;17(2):118-122.
3. Hong CS, Siegel AL, Ferris TG. Caring for high-need, high-cost patients: what makes for a successful care management program? Issue Brief (Commonw Fund). 2014;19:1-19.
4. Chen A, Brown R, Archibald N, Aliotta S, Fox PD; Mathematica Policy Research, Inc. Best practices in coordinated care. CMS website. https://www.cms.gov/Medicare/Demonstration-Projects/DemoProjectsEvalRpts/downloads/cc_full_report.pdf. Published March 22 2000. Accessed June 1, 2015
5. Bodenheimer T, Berry-Millett R. Care management of patients with complex health care needs. The Robert Wood Johnson Foundation website. http://www.rwjf.org/content/dam/farm/reports/issue_briefs/2009/rwjf49853/subassets/rwjf49853_1. Published December 2009. Accessed June 1, 2015.
6. Mullahy CM. Science versus art in case management. Case Manager. 2003;14(1):4.
7. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301(6):603-618. doi: 10.1001/jama.2009.126.
8. Wharam JF, Weiner JP. The promise and peril of healthcare forecasting. Am J Manag Care. 2012;18(3):e82-e85.
9. Hong CS, Hwang AS, Ferris TG. Finding a match: how successful complex care programs identify patients. California HealthCare Foundation website. http://www.chcf.org/~/media/MEDIA%20LIBRARY%20Files/PDF/PDF%20F/PDF%20FindingMatchComplexCare.pdf. Published March 2015. Accessed June 1, 2015.
10. Winkleman R, Mehmud S. A comparative analysis of claims-based tools for health risk assessment. Society of Actuaries website. https://www.soa.org/Files/Research/Projects/risk-assessmentc.pdf. Published April 20, 2007. Accessed June 1, 2015.
11. Bernstein RH. New arrows in the quiver for targeting care management: high-risk versus high-opportunity case identification. J Ambul Care Manage. 2007;30(1):39-51.
12. Cousins MS, Shickle LM, Bander JA. An introduction to predictive modeling for disease management risk stratification. Dis Manag. 2002;5(3):157-167.
13. Confidentiality of alcohol and drug abuse patient records (CFR Title 42: part 2). Government Publishing Office website. http://www.ecfr.gov/cgi-bin/text-idx?rgn=div5;node=42%3A188.8.131.52.2. Accessed June 1, 2015.
14. Shadmi E, Freund T. Targeting patients for multimorbid care management interventions: the case for equity in high-risk patient identification. Int J Equity Health. 2013;12(1):70. doi: 10.1186/1475-9276-12-70.
15. Berkowitz SA, Meigs JB, DeWalt D, et al. Material need insecurities, control of diabetes mellitus, and use of health care resources: results of the Measuring Economic Insecurity in Diabetes study. JAMA Intern Med. 2015;175(2):257-265. doi: 10.1001/jamainternmed.2014.6888.
16. Freund T, Gondan M, Rochon J, et al. Comparison of physician referral and insurance claims data-based risk prediction as approaches to identify patients for care management in primary care: an observational study. BMC Fam Pract. 2013;14:157. doi: 10.1186/1471-2296-14-157.
17. Lewis GH. “Impactibility models”: identifying the subgroup of high-risk patients most amenable to hospital-avoidance programs. Milbank Q. 2010;88(2):240-255. doi: 10.1111/j.1468-0009.2010.00597.x.
18. Haime V, Hong C, Mandel L, et al. Clinician considerations when selecting high-risk patients for care management. Am J Manag Care. 2015;21(10):e576-e582.
19. Grant RW, Ashburner JM, Hong CS, Chang Y, Barry MJ, Atlas SJ. Defining patient complexity from the primary care physician’s perspective: a cohort study. Ann Intern Med. 2011;155(12):797-804. doi: 10.7326/0003-4819-155-12-201112200-00001.
20. McCall N, Cromwell J, Urato C. Evaluation of Medicare Care Management for High Cost Beneficiaries (CMHCB) Demonstration: Massachusetts General Hospital and Massachusetts General Physicians Organization (MGH). Massachusetts General Hospital website. http://www.massgeneral.org/News/assets/pdf/FullFTIreport.pdf. Published September 2010. Accessed April 2016.
21. 2012 American Community Survey: 5-year estimates. US Census Bureau website. http://www2.census.gov/programs-surveys/acs/summary_file/2012/data/. Accessed June 1, 2015.
22. Pope GC, Kautter J, Ingber MJ, Freeman S, Sekar R, Newhart C. Evaluation of the CMS-HCC Risk Adjustment Model. CMS website. https://www.cms.gov/Medicare/Health-Plans/MedicareAdvtgSpecRateStats/downloads/evaluation_risk_adj_model_2011.pdf. Published March 2011. Accessed June 1, 2015.
23. Starfield B. Primary Care: Concept, Evaluation, and Policy. New York, NY: Oxford University Press; 1992.
24. Freund T, Wensing M, Geissler S, et al. Primary care physicians’ experiences with case finding for practice-based care management. Am J Manag Care. 2012;18(4):e155-e161.