Evelyn T. Chang, MD, MSHS; Rebecca Piegari, MS; Edwin S. Wong, PhD; Ann-Marie Rosland, MD, MS; Stephan D. Fihn, MD, MPH; Sandeep Vijan, MD; and Jean Yoon, PhD, MHS
The development of advanced healthcare analytics allows large healthcare organizations to use risk stratification as a population management strategy to proactively identify patients at high risk for poor health outcomes. These high-risk patients may benefit from customized interventions with the goals of improving outcomes and providing healthcare more efficiently.1
Risk stratification or risk prediction tools are often based on historical administrative claims and utilization data.2
However, the stability of risk estimates over time is uncertain. Programs that target patients identified as being high-risk at one point in time do not take into account the dynamic nature of changing risk over time; therefore, such programs may vary in effectiveness across patients.1
Different clinical interventions may be indicated for patients who are persistently versus temporarily high-risk.3-5
Studies have supported the notion that many high-cost, high-needs patients are just temporarily high-risk.3-5
Documented predictors of hospital and emergency department (ED) utilization include schizophrenia, homelessness, opioid prescriptions, depression, substance use, social isolation, and heart failure.6-8
However, little information exists to characterize patients who remain persistently high-risk for hospitalization. Prediction of persistent high risk over time may enhance risk stratification as a population management strategy and inform efforts to target and tailor resource-intensive clinical interventions.
The Veterans Health Administration (VHA) incorporated risk stratification measures into practice nationally in 2012. It reports weekly estimates of the probability of patient hospitalization in the next 90 days for the entire primary care patient population for use by primary care teams to inform clinical decision making.9-11
We aimed to inform population management programs for high-risk patients12
by following a cohort of high-risk patients over a 2-year period. Our objectives were to identify and describe groups of high-risk patients with distinct longitudinal risk score patterns and identify baseline characteristics that predicted which patients were likely to remain high-risk over time.
Data for this study were obtained from the VHA Corporate Data Warehouse, which included patient demographics, clinical diagnoses, admission and discharge status, patient risk scores, and utilization records.13
We also obtained data from the VHA Vital Status file to ascertain patients’ dates of death.
We identified a national cohort of 258,759 patients who had a 10% or greater absolute probability of 90-day hospitalization on our index date (November 23, 2012), were assigned to a VHA primary care provider, and were not hospitalized in the VHA on the index date, because risk scores are not generated for hospitalized patients. We followed patients for 2 years after the index date.
Measurement of Risk
We obtained patients’ monthly risk using the predicted probability of hospitalization in the next 90 days in the Care Assessment Need (CAN) model, a validated risk prediction algorithm.9
The CAN model (eAppendix Table 1
[eAppendix available at ajmc.com
]) uses demographics, utilization of VHA health services, comorbidities, prescribed medications, vital signs, and veteran-specific variables.9
Risk scores were generated on a weekly basis for all patients in VHA primary care. Patient-month measures of risk were calculated as the mean of weekly probabilities for that calendar month, and missing values occurred when veterans were hospitalized (including acute care, residential treatment facility admission, nursing home admission, or hospice admission) or dropped out of VHA care.
We defined patients as “high-risk” if the probability of 90-day hospitalization was 10% or greater, which corresponds to approximately the top 5% of VHA patients (n = 258,759). Our main dependent variable was a patient-month measure of risk persistence, defined hierarchically (Figure 1
Patients with at least 1 missing CAN probability in a calendar month were categorized into 1 of the following mutually exclusive outcomes: death, hospitalization in VHA, or dropped out of VHA care (no VHA encounters in previous 24 months). By definition, patients can move into and out of the “hospitalization” or “dropped out of care” categories. All patients except those who died were recategorized the following month.
We categorized patients with nonmissing CAN scores as persistently high-risk, intermittently high-risk, or initially high-risk then persistently low-risk for every month based on all patient-month observations in the study period. Patient-month observations were coded as “persistently high-risk” if hospitalization probability was 10% or greater during the current month and all prior months. “Initially high-risk” was defined as probability less than 10% during all subsequent months. “Intermittently high-risk” was defined as having a combination of high (≥10%) and low (<10%) hospitalization risk across the months and not meeting the criteria for persistently high-risk or initially high-risk. These categorizations ignore the patient-months designated as “hospitalization” or “dropped out of care.” By definition, patients can only move out of the persistently high-risk group, patients can only move into the initially high-risk group, and patients may move into or out of the intermittently high-risk group.
Patient Characteristics and Utilization
We examined patient characteristics measured at baseline: demographics (age, gender, marital status, ethnicity, urban vs rural residence), medical and psychiatric comorbidities, any cancer diagnosis, and markers of social instability.14
Markers of social instability included discharges against medical advice4
; number of zip code changes in the year prior4
; International Classification of Diseases, Ninth Revision
) diagnosis of nonadherence to medical treatment; and homelessness. Homelessness was measured by use of housing services15
diagnosis codes indicating lack of housing. ICD-9
diagnoses for comorbidities were adapted from previously published research16
and from the VHA Health Economics Resource Center17
(eAppendix Table 2
). Numbers of hospitalizations and outpatient visits and no-show rates were calculated from the index date over the subsequent 2 years. No-shows were recorded for each instance in which a patient did not arrive to a scheduled appointment.18
For outpatient visits during the follow-up period, we counted VHA encounters that occurred face-to-face or by telephone with any healthcare provider (eg, physician, nurse, psychologist). We categorized outpatient visits based on the specific type of service (ie, primary care, mental health, palliative care, ED).19
Mental health visits included primary care–mental health integration encounters.20
Inpatient admissions were categorized into medical, surgical, and psychiatric admissions.19
Our dependent variable in bivariable and multivariable analyses was the patient’s risk category (ie, persistently high, intermittently high, or initially high) in the final study month (n = 194,597). We used analysis of variance to examine whether there were differences in means among the continuous variables (inpatient/outpatient utilization, ED visits, and zip code changes) across the high-risk categories. In bivariable analyses, we performed ordered logit regression to estimate a single equation for each independent variable over the 3 ordered levels of the dependent variable: persistently, intermittently, and initially high-risk. Proportional odds ratios (ORs) were generated from the ordered logit models to estimate the change in odds in each predictor for patients at (1) persistently high-risk versus intermittently high-risk and (2) intermittently high-risk versus initially high-risk.
We performed multivariable logistic regression to predict being persistently high-risk versus the other 2 groups after adjusting for independent variables found to be statistically significant (P
<.05) from the ordered logit regression analyses described previously. Model covariates included age, gender, and race/ethnicity, and we excluded variables used to calculate the CAN probabilities (eAppendix Table 1).
For continuous independent variables that are not normally distributed (ie, number of discharges against medical advice, palliative care encounters, mental health visits, and zip code changes), we applied a log (base 10) transformation to create a less skewed distribution. As a result, one would need to have a 10-fold increase in the independent variables to achieve the ORs indicated for the transformed variables.
Because the ordered logit model assumes proportional odds, we also estimated a generalized ordered logistic regression in sensitivity analysis. This model estimates ORs for variables that violate the proportional odds assumption separately for each comparison.21
OR estimates from the generalized ordered logit model were of similar size and significance as those from the ordered logit model (eAppendix Table 3
). In sensitivity analyses, we also used a multinomial logit model to compare patients who were persistently high-risk with those who were intermittently high-risk and to compare those who were persistently high-risk with those who were initially high-risk. Directionality and significance were similar to the ordered logit model (eAppendix Table 4
We conducted stratified analyses by age group (≥65 years and <65 years) to examine whether there were differences in risk patterns between VHA enrollees eligible for Medicare and younger veterans; results were similar for both age groups (eAppendix Table 5
), so we report only the combined results.
Analyses were conducted with SAS version 9.4 (SAS Institute Inc; Cary, North Carolina) and SAS Enterprise Miner.
Institutional Review Board
This evaluation was designed to support VHA operations and quality improvement for the VHA Offices of Primary Care and Clinical Systems Development and Evaluation and was exempt from institutional review board approval or waiver.22,23
shows the monthly risk status of the high-risk patient cohort over a 24-month period (quantitative results in eAppendix Table 6
). After 1 year, 28.6% (n = 74,060) were persistently high-risk for hospitalization, 10.3% (n = 26,541) had died, 23.8% (n = 61,556) were initially high-risk for hospitalization, and 30.6% (n = 79,078) were intermittently high-risk for hospitalization. By the end of the 2-year period, just 13.8% (n = 35,770) remained persistently high-risk for hospitalization; 17.7% (n = 45,805) had died, 41.5% (n = 107,473) were initially high-risk, and 19.9% (n = 51,354) were intermittently high-risk, and the remaining 7.1% (n = 18,357) were hospitalized or left VHA care at study end.
Patients who were older than 45 years, male, unmarried, black, and living in an urban area had greater odds of being persistently high-risk for hospitalization (vs intermittently high-risk) and of being intermittently high-risk for hospitalization (vs initially high-risk) (Table 1 [part A and part B]
). Patients with nearly all medical and mental comorbidities had greater odds of being persistently or intermittently high-risk for hospitalization. The largest differences between the risk categories were found with congestive heart failure (OR, 2.60; 95% CI, 2.55-2.65), chronic kidney disease (OR, 2.48; 95% CI, 2.39-2.58), and dementia (OR, 1.96; 95% CI, 1.90-2.00). Patients with social instability, such as discharges against medical advice, number of zip code changes, and diagnoses of nonadherence, had greater odds of being both persistently and intermittently high-risk for hospitalization. Median no-show rates in a given year were similar for all risk categories.
Patients who were persistently high-risk for hospitalization were more likely than those who were intermittently high-risk or initially high-risk to have a VHA hospitalization (89%, 75%, and 41%, respectively) or ED visit (93%, 86%, and 57%, respectively) during the follow-up period (Table 2
). Also, persistently high-risk patients had more frequent VHA hospitalizations per year than the initially high-risk group, with a mean (SD) of 1.9 (3.5) hospitalizations per year; most (1.5 [3.1]) of these admissions were medical.
Patients who were persistently high-risk for hospitalization also had higher rates of outpatient utilization: They had a mean (SD) of 83.2 (81.2) total VHA outpatient encounters (telephone and in-person visits) per year, almost double the number of outpatient encounters per year among patients who were only initially high-risk for hospitalization (Table 2). Persistently high-risk patients had a mean (SD) of 10.6 (15.9) encounters in primary care, 18.1 (64.9) encounters in mental health, and 0.1 (1.9) encounters in palliative care per year. In contrast, patients who were only initially high-risk for hospitalization had, on average, 44.4 (58.4) outpatient encounters per year, 6.7 (11.0) in primary care, 9.5 (41.1) in mental health, and 0.02 (0.8) in palliative care. Intermittently high-risk patients had mean utilization in between the persistently high-risk and initially high-risk groups.
In multivariable analyses, we identified 20 statistically significant (P
<.05) predictors of being persistently high-risk for hospitalization, including age, gender, urban residence, visual and hearing impairment, chronic pain (back and neck pain, arthritis, and headache), chronic medical comorbidities (chronic kidney disease, coronary artery disease, congestive heart failure, diabetes, hypertension, and nicotine use), number of visits to palliative care, cancer diagnosis, number of mental health visits, and all markers of social instability (Table 3
In this study, we examined the patterns of risk status over time among a population of high-risk patients receiving continuous VHA care. Consistent with findings of prior research on costs and readmissions,3,24
the majority of these patients did not remain persistently high-risk during 2 years of follow-up; just 29% were persistently high-risk after 1 year, and 14% remained persistently high-risk after 2 years. Almost half (42%) were classified as being persistently low-risk by study end.
These findings may partly explain seemingly positive results from interventions that enroll patients who are high-risk at baseline but that report pre–post findings without a comparison group. The improvement in patient outcomes could, in fact, be due to naturally decreasing risk over time rather than due to the intervention itself. Several randomized studies of care management interventions for high-need, high-cost patients have found no differences in cost and utilization between intervention and usual-care groups.25-27
We found significant and meaningful differences in sociodemographics, clinical comorbidities, and utilization among the 3 trajectory groupings of high-risk patients. The presence of chronic medical or behavioral conditions (particularly heart failure and chronic kidney disease), a nonadherence diagnosis, and higher VHA utilization rates were associated with greater duration of remaining persistently high-risk for hospitalization, which indicates that programs for high-risk patients can focus efforts on patients with these characteristics. Persistently and intermittently high-risk patients used a remarkable amount of VHA services—as many as an average of 63 to 83 outpatient visits and 0.9 to 1.9 admissions per year—suggesting that they were actively engaged with ambulatory care providers. Some acute care may be unavoidable despite high utilization of outpatient services, especially for patients who rapidly decompensate or cannot be managed effectively with ambulatory care alone.28
Persistently high-risk patients were more likely to live in urban areas, but many (30%) lived in rural areas, indicating that smaller, rural community clinics or virtual care modalities may have an important role to play in mitigating risk for these patients.
Our results confirm published findings described in other predictive models for high utilization, readmission, and mortality: Age, gender, number of zip code changes, number of discharges against medical advice,6,7,29
frequent use of outpatient medical and mental health services,6
nonadherence, and tobacco use29
significantly predicted long-term persistence of high risk. Other known predictors of high utilization (eg, socioeconomic status, depression, alcohol use disorder, marital status) were not included in our model due to endogeneity. Unlike previous studies, we report that hearing and visual impairment, cancer diagnosis, and palliative care utilization were also predictors of patients being persistently high-risk. Interestingly, unlike prior studies6,29
that preceded the VHA comprehensive primary care teams for homeless patients,30
our findings show that homeless patients who used housing support services were less likely to be persistently high-risk. It is possible that VHA programs that address housing instability were able to effectively change the duration for which a patient is high-risk.
Developing effective programs for high-risk patients has proved challenging because most randomized trials in this area have not shown improvement in outcomes.26,31,32
One possibility is that high-risk patients are so heterogeneous that a one-size-fits-all approach may not be effective. Different clinical interventions may be more suitable for patients who are persistently high-risk versus those who are only initially or intermittently high-risk or for patients with various combinations of factors that predict risk.33,34
For instance, persistently high-risk patients may benefit from longitudinal primary care programs tailored for specific vulnerable populations (eg, primary care for homeless, homebound, or elderly patients; those receiving dialysis; those with chronic pain or serious mental illness); other high-risk patients may benefit from time-limited care management models that augment primary care, such as collaborative care models for high-needs patients.26,35
Large healthcare systems may find it helpful to use empiric clustering models to further distinguish subgroups of high-risk patients based on clinical characteristics.4,5,36,37
Our findings suggest ways to tailor care management interventions for patients with sustained needs. Although the extent to which a patient’s underlying risk for hospitalization is modifiable remains unclear, evidence-based practices can be implemented to possibly reduce a patient’s risk for hospitalization. For instance, interventions to target housing instability, such as the Housing First model, may reduce a patient’s risk for future hospitalization.30
Healthcare teams who target persistently high-risk patients could include behavioral health specialists familiar with managing treatment nonadherence, holistic approaches to chronic pain, and guideline-concordant care for ambulatory care–sensitive conditions (eg, congestive heart failure, diabetes, hypertension). Comprehensive assessments could identify depression and patient factors underlying nonadherence and social instability in high-risk patients and then prioritize which patients should receive the most intensive interventions to reduce future risk. Future studies could examine whether intervening on predictors of persistent high risk is effective in changing a patient’s risk trajectory.
Measures of socioeconomic status, including income, employment status, and education status, were likely predictors of risk persistence, but they were not included due to lack of availability in VHA administrative data.29
To avoid a problem in tautology, we excluded VHA risk prediction model variables (eg, mental health conditions, cognitive impairment) from our model to predict whether or not patients remain at high risk. However, we do report some clinical characteristics not included in the risk prediction model that contribute to hospitalization risk, such as chronic pain conditions, markers of social instability, and sensory impairments. Lastly, our sample is limited to the VHA population, which includes veterans who are less likely to have childhood-onset medical conditions due to military medical eligibility, are more likely to have psychiatric conditions, and may use services in multiple healthcare settings, not just the VHA.38
We were unable to draw from non-VHA data sources for this analysis to account for healthcare services that were not paid by the VHA. Although the actual percentage of persistently high-risk patients might differ by payer,39
the overall patterns are likely to be similar.
Because most high-risk patients may be only briefly high-risk for hospitalization, nonrandomized evaluations of interventions that identify patients based on a single risk score may appear to have positive effects on patient health due to naturally decreasing risk over time in the majority of patients. Identified predictors of patients remaining persistently high-risk could inform efforts to tailor the intensity and type of interventions to improve health for veterans with the most sustained needs.
This study was funded by the Veterans Health Administration (VHA) Patient Aligned Care Team Demonstration Laboratory Coordination Center (XVA-61-041). Dr Wong is supported by VHA Career Development Award CDA 13-024. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the University of California, the University of Washington, the University of Pittsburgh, or the University of Michigan.