This study demonstrates that the predictive accuracy of primary care physicians’ assessment of future hospitalization risk is comparable to commonly used quantitative risk stratification instruments.
Published Online: April 12, 2017
Andrew S. Hwang, MD, MPH; Jeffrey M. Ashburner, PhD, MPH; Clemens S. Hong, MD, MPH; Wei He, MS; and Steven J. Atlas, MD, MPH
Prospectively identifying patients at increased risk for hospitalization is a key prerequisite to designing interventions to prevent avoidable admissions. Primary care physicians (PCPs), with their comprehensive understanding of the clinical and psychosocial needs of their patients, may be uniquely qualified to identify risk factors missed by existing predictive algorithms. We examined whether PCPs could accurately identify patients at high risk for hospitalization in the subsequent year.
We invited PCPs in our network managing a panel of at least 100 patients in 2013 to review a list of 100 randomly selected patients from their panel.1 After verifying that the patient was theirs, PCPs were asked, “Would you be surprised if this patient was admitted to the hospital in the next year?” Patients were categorized as high or low admission risk based on whether PCPs answered “no” or “yes,” respectively.
We compared baseline demographics and Charlson Comorbidity Index (CCI) scores between patients in the PCP-designated high versus low admission risk groups using χ2 or t tests, as appropriate. Unadjusted rates of hospitalization in the prior year and subsequent 1 year were compared using zero-inflated Poisson (ZIP) regression while accounting for clustering by PCP. To determine if PCP assessment remained an independent predictor of future hospitalization, we used ZIP regression to adjust for patient demographics, PCP characteristics, past acute care utilization, and CCI score, while accounting for clustering by PCP. We also calculated the C statistic for PCPs’ prediction of subsequent hospitalization.
Among eligible PCPs, 54% (99 of 182) reviewed their patient list. There were no statistically significant differences between participating and nonparticipating PCPs. Of 9900 eligible patients, PCPs reviewed 9832 patients and designated 238 (2.4%) as not being their patient. Among the remaining 9594 patients, PCPs designated 2037 (21.2%) as high admission risk and 7538 (78.6%) as low admission risk. Nineteen (0.2%) patients were unassigned and excluded from analyses.
Compared with patients in the low admission risk group, patients in the high admission risk group were significantly older, more likely to be white, insured by Medicare, lived in neighborhoods with a low median household income, had a higher Charlson Comorbidity Index score, and had increased unadjusted rates of acute care utilization in the past year and subsequent 1 year (Table).
PCPs’ qualitative assessment of future hospitalization risk among patients in their panels was an independent predictor of subsequent hospitalization, identifying a group of patients with a 172% increased rate of hospitalization (adjusted rate ratio, 2.72; 95% confidence interval [CI], 1.95-3.78). Predictive accuracy of PCP assessment in our study (C statistic, 0.77; 95% CI, 0.75-0.79) was comparable to the reported C statistic of other commonly used risk stratification instruments.2 There was no major difference in C statistic among those with (0.77; 95% CI, 0.72-0.81) or without (0.75; 95% CI, 0.73-0.77) hospitalization in the prior year, highlighting the fact that PCPs were not simply identifying those with past hospitalizations. Given the predictive accuracy of PCPs’ clinical assessment, efforts to identify patients at high risk for future hospitalization should aim to incorporate the unique insight that PCPs have about predisposing biopsychosocial factors.
Author Affiliations: Division of General Internal Medicine, Massachusetts General Hospital (ASH, JMA, WH, SJA), Boston MA; Los Angeles County Department of Health Services (CSH), Los Angeles, CA.
Source of Funding: Funded by Partners Community Healthcare, Inc, and by the Massachusetts General Hospital Primary Care Operations Improvement Program.
Author Disclosures: The authors report no other relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (JMA, SJA, ASH, CSH); acquisition of data (JMA, SJA, CSH, WH); analysis and interpretation of data (JMA, ASH, WH); drafting of the manuscript (ASH); critical revision of the manuscript for important intellectual content (JMA, SJA, ASH, CSH); statistical analysis (ASH); provision of patients or study materials (WH); obtaining funding (CSH); administrative, technical, or logistic support (CSH); and supervision (SJA, CSH).
Address Correspondence to: Andrew S. Hwang, MD, MPH, Gray Bigelow 730, 55 Fruit St, Department of Medicine, Boston, MA 02114. E-mail: firstname.lastname@example.org.
1. Atlas SJ, Chang Y, Lasko TA, Chueh HC, Grant RW, Barry MJ. Is this “my” patient? development and validation of a predictive model to link patients to primary care providers. J Gen Intern Med. 2006;21(9):973-978.
2. Haas LR, Takahashi PY, Shah ND, et al. Risk-stratification methods for identifying patients for care coordination. Am J Manag Care. 2013;19(9):725-732.