Implementing a claims-based algorithm and disease management program may be an effective strategy to reduce relapse and cost among patients with schizophrenia.
Objectives: To refine a payer algorithm identifying patients with schizophrenia at high risk of relapse within a managed Medicaid population and evaluate its effectiveness in a case management (CM) program.
Study Design: Cross-sectional and longitudinal study design.
Methods: The algorithm used a single payer’s Medicaid medical and pharmacy claims (August 1, 2009, to July 31, 2014) for patients with schizophrenia (N = 12,353) to predict those at high risk for hospitalization. The final algorithm was used in a CM program (outbound communication to providers) at 3 payer service centers in 3 states. Based on the algorithm, 60 patients (20 from each site) with the highest risk scores were targeted for CM (CM group) and 60 (those patients ranked 21st-40th most at-risk at each site) comprised the control group. Chi-square tests compared groups on frequency measures (hospitalizations, emergency department [ED] visits). Pre- to postimplementation differences were tested using McNemar’s test. A pre—post analysis of variance assessed mean numbers of inpatient admissions, inpatient days, and ED visits for both groups.
Results: The algorithm had good positive predictive power (64.0%), negative predictive power (94.7%), sensitivity (40.2%), and specificity (97.9%). Following CM, the proportion of patients with at least 1 inpatient admission in the CM group decreased (23.3% to 13.3%), as did the rate of ED visits per month (by approximately 15%), whereas increases were observed in the control group.
Conclusions: Although not all of these differences were statistically significant, they suggest that the algorithm may be an effective case-finding tool for plans attempting to mitigate hospitalizations among high-risk patients with schizophrenia.
Am J Manag Care. 2019;25(12):e373-e378Takeaway Points
We developed a regression-based payer algorithm to identify patients with schizophrenia at the greatest risk of relapse and hospitalization. Further, we assessed the effectiveness of this algorithm in a disease management program.
Schizophrenia is a chronic remitting and relapsing mental illness that affects approximately 1.1% of the US population and between 1% and 2% of the Medicaid population.1,2 Schizophrenia has one of the highest aggregate costs of all mental illnesses, estimated at $37.7 billion for direct medical costs and an additional $117 billion for indirect costs in the United States in 2013.3 Relapse of schizophrenia is the largest cost driver,4,5 with these patients incurring up to 5-fold more medical costs than those who do not experience relapse, largely due to the inpatient costs associated with relapse.4,6-8 Therefore, improved methods to identify patients at greatest risk of relapse may reduce the number of relapses and the use of associated high-cost services.9
There are many early warning signs that can precede relapse in patients with schizophrenia, including nonadherence to treatment, ongoing depression, poor therapeutic alliance, and persistent substance abuse.10-12 The presentation of schizophrenia can be heterogeneous and, therefore, some clinicians believe that schizophrenia is a clinical syndrome with different patterns of symptoms rather than a singular, uniform disease.13,14 One model available to payers to estimate trajectory to relapse is to use claims-based indicators of patient behaviors (eg, antipsychotic medication adherence) or consequences (eg, prior psychiatric hospitalization, emergency department [ED] visits).4,11 However, neither of these are effective as standalone predictors.15 By contrast, earlier research employed a predictive model that allowed any combination of 6 claims-based proxy measures of patient instability (ie, patient instability events [PIEs]). These 6 proxy measures were summed using unit weighting into a composite score, which predicted decompensation with more accuracy than any single measure alone in a commercially insured population.16 Use of composite measures may assist payers in identifying those patients at a high risk of relapse and targeting interventions to avoid costly relapses. Interventions designed to reduce relapse in patients with schizophrenia vary from intensive cognitive behavioral therapy to broader community-based psychoeducational formats.17,18
The goals of this study were (1) to translate and refine the PIE algorithm concept for use as a case-finding algorithm that identified patients with schizophrenia at greatest risk for relapse and hospitalization within a managed Medicaid population and (2) to evaluate the effectiveness of a case management (CM) program that included the case-finding algorithm and outreach to identified patients’ healthcare providers to help avoid relapse. The ultimate goal of the program was to reduce relapse rates and associated healthcare expenditures while improving patient outcomes.
Data derived from a single payer’s Medicaid claims inclusive of medical and pharmacy claims from August 1, 2009, to July 31, 2014, were used to refine the schizophrenia case-finding algorithm originally developed using a commercial claims database. The data set included medical and pharmacy claims and eligibility data for all patients who had at least 1 inpatient or 2 outpatient claims containing a diagnosis for schizophrenia as identified by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 295.x or International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) code F20.x between July 1, 2012, and June 30, 2013 (case-finding period), as well as continuous eligibility during both the case-finding and measurement (July 1, 2013, to June 30, 2014) periods (Figure 1). Excluded from the study sample were cases with diagnoses of schizophreniform disorder (ICD-9-CM code 295.4x or ICD-10-CM code F20.81) or schizotypal disorder (ICD-9-CM code 295.7x or ICD-10-CM code F25.90) in the absence of other forms of schizophrenia, based on consultation with the health plan clinical leads.
Medicaid claims were used to assess the relationship between instability of patients with schizophrenia and relapse during the study period to identify indicators of instability and combinations that adequately predict patient relapse. Eight proxies of instability—6 of which were identified in a prior study16 and 2 of which were added at the request of clinical staff at the payer—were computed for each of the first 2 quarters of the measurement period (predictor assessment period) and were used in linear combination to detect subsequent relapse: Charlson Comorbidity Index (CCI) score; medication switch, defined as receipt of an antipsychotic medication other than the index antipsychotic medication; count of hospital admissions with a primary diagnosis of schizophrenia; count of ED visits with a psychiatric diagnosis (ICD-9-CM codes 295, 296, 300-305; ICD-10-CM codes F209, F200, F201, F202, F203, F205, F208, F2081, F2089, F250, F251, F258, F259, F39, F319, F419, F21, F609, F659, F102, F112, F1219, F1319, F1499, F1519, F1699, F1819, F1019); any fill for a psychiatric medication other than an antipsychotic; presence of a diagnosis of depression (ICD-9-CM codes 296.2x, 296.3x, 300.4x, 309.1x, 311.x; ICD-10-CM codes F329, F3340, F349, F4321); presence of a diagnosis of bipolar disorder (ICD-9-CM code 296.x except 296.2x and 296.3x; ICD-10-CM codes F209, F200, F201, F202, F203, F205, F208, F2081, F2089, F250, F251, F258, F259); and presence of a diagnosis of psychosis other than the primary diagnosis (ie, nonorganic psychosis diagnosis).
Relapse was the outcome of interest in this study and was defined as a psychiatric hospital admission with a primary diagnosis of schizophrenia (ICD-9-CM code 295.x or ICD-10-CM code F20.x) or an ED visit with any psychiatric diagnosis. The broader definition of any psychiatric diagnosis for ED visits was used because sometimes psychiatric illnesses are similar in their presentation in the ED and may be miscoded. These outcome measures were chosen because they are the clinical manifestations of a relapse and are costly to a health system. Relapse events were measured in the final 2 quarters of the measurement period (January 1, 2014, to June 30, 2014), which immediately followed the assessment period of the predictor variables (July 1, 2013, to December 31, 2013). Inpatient stays for schizophrenia were defined by place of service codes 21 or 51, with each stay being defined as having at least a 2-day gap from any previous stay. ED visits for a psychiatric diagnosis were defined by place of service code 23.
Presence of relapse was regressed onto demographic characteristics, quarterly PIEs, and a summed total of all PIEs. A forward conditional stepwise regression model was utilized, in which predictors were selected in the order of the magnitude of the amount of variance in the criterion variable accounted for by the model. A random segment or split half design was then used to evaluate the positive predictive power (PPP) and negative predictive power (NPP) of the algorithm at each phase of development. The variables listed as possible predictors and outcomes were inserted into the case-finding algorithm programming, which was run with the original outcomes and the new independent variables using stepwise regression with backward elimination. Results from the refined algorithm were compared with those from the original algorithm for change in predictive properties. Any statistically significant improvement (ie, increase in either or both sensitivity or PPP, while increasing or holding constant specificity and NPP) was evaluated for potential use in the case-finding algorithm. This refinement process continued iteratively until the final algorithm was agreed on.
Disease Management Program
The single Medicaid payer chose 3 regional service centers at which to incorporate the final algorithm within an existing high-risk CM program. Based on the final algorithm, patients at each of the service centers were dichotomized into high- or low-risk relapse groups. Patients who had values that were greater than or equal to 80% of the range of scores were classified as high-risk, with the remaining patients being classified as low-risk. Forty patients with the highest overall risk scores from each of 3 local service centers within the payer system were identified for analysis. From that selection of patients, the 20 patients with the highest scores from each service center were targeted for CM (CM group) and the next 20 patients from each of the 3 service centers, whose rank orders of total risk scores were 21 through 40, were retained as a proxy for a high-risk control group (Figure 2).
All aspects of the CM program were managed by payer staff at the 3 regional service centers serving the 3 demonstration states. The CM program consisted of outbound correspondence (letters or telephone calls) to providers managing the care for the patients with the 20 highest risk scores at each service center. The manner of outbound correspondence was determined by the CM procedures at each of the service centers and may have differed by center. The control group participants were not included in the lists of high-risk patients provided for the CM program. The CM was conducted in a “black box” proprietary program by the plan, as there were no logs of specific conversations with providers as part of the evaluation. Further, the number of times that each provider was contacted by health plan CM staff or the number of attempts of contact is not known. Finally, as the CM was implemented by the payer’s staff, with little visibility into the exact nature of the CM program, this study did not assess any follow-up action taken by the provider based on the outbound correspondence by the payer.
To evaluate the effectiveness of the algorithm and CM program, pre- and postimplementation periods were used to capture outcomes related to relapse. The pre-CM period utilized data captured prior to dissemination of the CM program. The window for the preimplementation period was November 1, 2015, through April 30, 2016. The postimplementation period utilized data captured after implementation of the CM program, with a window from December 1, 2016, through May 15, 2017. Both a cross-sectional and a longitudinal study design were employed, meaning that outcomes for the CM cohort were compared with those for the control cohort (cross-sectional) and the preimplementation period outcomes were compared with the postimplementation period outcomes within the same group (longitudinal) (Figure 2).
Outcome measures for each group included (1) the percentage of patients who had at least 1 inpatient hospitalization for schizophrenia, (2) the overall rate of inpatient admissions, (3) the percentage of patients who had at least 1 psychiatric ED visit, and (4) the rate of ED visits per month. As this was a pre—post design, all measures were calculated for both the pre- and postimplementation periods for each group. CM and control groups were compared on frequency measures using χ2 tests of proportions. Differences over time (pre- to postimplementation period) in frequency data were tested using McNemar’s χ2 test. The α level was set at P <.05. All analysis was conducted with SAS version 9.4 (SAS Institute; Cary, North Carolina).
The final sample consisted of 12,353 health plan members with a diagnosis of schizophrenia from the payer’s entire Medicaid population. Approximately 9% of the sample (n = 1044) had evidence of a relapse event and were placed in the relapse group. The remainder of the sample (n = 11,309) were placed into the nonrelapse group. The final regression model for the case-finding algorithm was as follows:
ln(odds) = −3.270 + 0.09 (CCI score)
+ 0.41 (medication switch)
+ 0.92 (PIE: schizophrenia inpatient visit)
+ 0.04 (PIE: psychiatric ED visit)
+ 0.28 (PIE: other psychiatric medication fill, excluding antipsychotics)
+ 0.25 (PIE: depression diagnosis)
+ 0.22 (PIE: bipolar diagnosis)
+ 0.48 (other nonorganic psychosis diagnosis)
All variables were significant, positive predictors of relapse in the model, indicating that higher scores on each are related to an increased likelihood of relapse. The strongest predictor was prior relapse, with a 2.5 times greater likelihood of future relapse for every prior relapse assessed. The medication switch flag was also a strong predictor, with switchers showing a 1.5 times greater likelihood of relapse compared with nonswitchers (Table 1). The overall model accounted for 30% of the variance in schizophrenia relapse (Nagelkerke R2 = 0.307) and had a PPP of 64.0%, NPP of 94.7%, sensitivity of 40.2%, and specificity of 97.9%.
Based on the regression algorithm, the CM group mean (SD) risk score was 0.9985 (0.0093) and the control group mean (SD) risk score was 0.9932 (0.0003). A comparison of the percentage of inpatient admissions and percentage of members with at least 1 inpatient admission in the preimplementation period revealed no differences between the CM and control groups. However, following the implementation period, the percentage of patients with at least 1 inpatient admission dropped by 10.0 percentage points in the CM group (pre- vs post, 23.3% vs 13.3%; P = .08) compared with a decrease of 3.3 percentage points in the control group. Moreover, there was a statistically significant difference in the number of patients with at least 1 inpatient admission following implementation between the CM and control groups (13.3% vs 20.0%; P <.05) (Table 2). Pre—post analysis of the CM group with inpatient admissions using McNemar’s χ2 test (Table 3) revealed that 7 of 8 (87.5%) patients with an inpatient admission during the preimplementation period were not readmitted during the postimplementation period (P = .08). The rate of inpatient admissions was slightly lower in the CM group (26.1%) than in the control group (34.2%) during the preimplementation period and was very similar between the groups during the postimplementation period (28.9% vs 29.4%, respectively); neither difference was statistically significant.
Regardless of assignment to the CM or control group, greater than 90% of patients had at least 1 ED visit for a psychiatric disorder prior to (91.7% for both groups) and following (93.3% for both groups) implementation. However, the rate of ED visits per month decreased 14.5% for the CM group, from 7.62 to 6.52, while increasing 21.5% for the control group, from 2.68 to 3.26 (P <.05) (Table 2).
Relapse in schizophrenia is associated with significant healthcare costs. Proactively identifying patients with schizophrenia who are at risk for relapse may result in fewer hospitalizations, better outcomes, and reduced costs.9 Clinical, patient-focused interventions can be time-consuming, and the long-term effectiveness of such interventions has not been well established.17 Previous models of intervention at the payer level have attempted to estimate the trajectory to relapse in schizophrenia using standalone claims-based indicators of patient behaviors (eg, antipsychotic medication adherence) or consequences (eg, prior psychiatric hospitalization). These standalone measures are generally ineffective at predicting relapse.15 In contrast, predictive models that allow for the combination of multiple claims-based proxy measures of instability appear to provide greater accuracy at predicting relapse than any single measure alone.16 The algorithm developed in this study had acceptable PPP, meaning that 64% of patients identified by the algorithm as being at risk were truly at risk. High PPP indicates that intervention resources will be used efficiently and not wasted on patients who are not in need. However, this model’s NPP and specificity were higher than its PPP and sensitivity, indicating that the model may be better at detecting a lack of relapse risk than a positive relapse risk.
Algorithm-generated relapse scores from 3 regional service centers ranged from 0.959 to 1.000, and patients were segmented into CM and control groups based on the magnitude of their total score. Following CM implementation, there was a decrease in the percentage of patients with 1 or more inpatient admissions in the CM group (23.3% to 13.3%). The rate of ED visits per month in the CM group decreased by nearly 15%, whereas increases in the control group were observed following implementation. Together these differences suggest that the PIE algorithm may be an effective case-finding tool for plans attempting to mitigate hospitalizations among high-risk patients with schizophrenia. Using the algorithm to identify appropriate patients for CM may be a more efficient way for health plans to target limited CM resources rather than relying on single-indicator methods of patient identification.
This study had several limitations. First, the sample selected for CM based on the algorithm was small, with 60 overall participants (20 at each of the 3 locations). The control group was similarly small (20 at each site ranked 21st-40th most at-risk) identified by the algorithm at each site. The small sample size may have contributed to the nonsignificant results, indicated by trends in a positive direction. Perhaps a larger sample size would have garnered a greater number of significant outcomes. Although the mean risk score was similar between groups (0.9985 in CM group vs 0.9932 in control group), the groups were not equivalent, because the group chosen for CM at each service center had the highest risk scores. Future research using a similar approach may benefit from alternating the patients chosen for CM and control groups or randomly assigning patients to each group to minimize any differences between the groups.
Second, although the algorithm was used uniformly to identify the patients most at risk for relapse, CM was implemented at 3 service centers within the payer system, and there may have been differences in implementation between sites. Details about the exact nature of the program can only be surmised, as the intervention was conducted in a “black box” without access to details or logs that would speak to intervention fidelity. The authors recommend that future studies use a single, well-developed intervention protocol to ensure that all details of the intervention are captured accurately and implemented across sites in the same manner. Third, as this was not a randomized controlled study, it is unknown whether interventions outside of the program being evaluated had a differential impact on the 2 groups’ inpatient admissions and ED visits.
Finally, although the communications went out to the providers managing the care for the 20 patients with the highest risk scores at each service center, it is not clear to what degree providers attended to them, nor was assessing this within the scope of this project. How providers process clinical information from payers is not clearly understood and could be the topic of further investigation. The authors recommend including provider actions in CM intervention for future research purposes, as well to further reduce variability in the results.
This study points to the usefulness of a claims-based case-finding algorithm to identify patients with schizophrenia at high risk of relapse within the payer system. Overall, these findings suggest that with refinement and more potent intervention, this algorithm might be a reasonable case-finding tool for a population of patients with schizophrenia whose use of high-cost services such as inpatient and ED care could be mitigated. The authors encourage additional research using this or similar case-finding algorithms in conjunction with more robust interventions to further assess their effectiveness. The supplementation of such programs with additional components may enhance effectiveness; for example, prodromal symptoms have been found to be reliable early warning signs for relapse.10 It is possible that the current version of the PIE or the disease management program may be made even more robust by the inclusion of patient-reported measures of symptoms that commonly precede relapse, including insomnia, social withdrawal, and increased irritability.12 The current study did not assess risk for hospitalizations for reasons other than schizophrenia; however, patients with schizophrenia are hospitalized for a variety of reasons (eg, substance abuse, chronic medical conditions), and inclusion of these factors into the algorithm and CM efforts may in fact be as beneficial as simply attempting to prevent schizophrenia-related admissions. Inclusion of other indicators of patient stability, such as medication adherence, clinical symptom improvement, outpatient visits, and improvements in chronic medical conditions, may also be warranted. Further research should be conducted within high-risk schizophrenia samples, as patients may vary greatly by relapse indicators and relapse consistency. Effective disease management programs would prove useful because, by design, they require less of a time burden on the provider and patient, and in the long run could be economically valuable for health plans and payers.
The authors would like to thank Manticore Consultancy for editorial and formatting support.Author Affiliations: Otsuka Pharmaceutical Development & Commercialization, Inc (HCW), Princeton, NJ; Health Analytics, LLC (CR, JT), Columbia, MD.
Source of Funding: The research was supported by Otsuka Pharmaceutical Development & Commercialization, Inc, and Lundbeck, LLC.
Author Disclosures: Dr Waters is employed by Otsuka Pharmaceutical Development & Commercialization, Inc, which manufactures 3 treatments for schizophrenia. Dr Ruetsch and Mr Tkacz were employed by Health Analytics, which functioned as a paid consultant of Otsuka at the time this work was completed.
Authorship Information: Concept and design (HCW, CR); acquisition of data (JT); analysis and interpretation of data (HCW, CR, JT); drafting of the manuscript (HCW, CR, JT); critical revision of the manuscript for important intellectual content (HCW, CR, JT); statistical analysis (CR, JT); obtaining funding (HCW); and supervision (HCW).
Address Correspondence to: Heidi C. Waters, PhD, MBA, Otsuka Pharmaceutical Development & Commercialization, Inc, 508 Carnegie Center Dr, Princeton, NJ 08540. Email: Heidi.firstname.lastname@example.org.REFERENCES
1. Chapter 4: behavioral health in the Medicaid program—people, use, and expenditures. Medicaid and CHIP Payment and Access Commission website. macpac.gov/wp-content/uploads/2015/06/Behavioral-Health-in-the-Medicaid-Program—People-Use-and-Expenditures.pdf. Published June 2015. Accessed November 16, 2018.
2. Regier DA, Narrow WE, Rae DS, Manderscheid RW, Locke BZ, Goodwin FK. The de facto US mental and addictive disorders service system: epidemiologic catchment area prospective 1-year prevalence rates of disorders and services. Arch Gen Psychiatry. 1993;50(2):85-94. doi: 10.1001/archpsyc.1993.01820140007001.
3. Cloutier M, Aigbogun MS, Guerin A, et al. The economic burden of schizophrenia in the United States in 2013. J Clin Psychiatry. 2016;77(6):764-771. doi: 10.4088/JCP.15m10278.
4. Ascher-Svanum H, Zhu B, Faries DE, et al. The cost of relapse and the predictors of relapse in the treatment of schizophrenia. BMC Psychiatry. 2010;10:2. doi: 10.1186/1471-244X-10-2.
5. Rajagopalan K, O’Day K, Meyer K, Pikalov A, Loebel A. Annual cost of relapses and relapse-related hospitalizations in adults with schizophrenia: results from a 12-month, double-blind, comparative study of lurasidone vs quetiapine extended-release. J Med Econ. 2013;16(8):987-996. doi: 10.3111/13696998.2013.809353.
6. Almond S, Knapp M, Francois C, Toumi M, Brugha T. Relapse in schizophrenia: costs, clinical outcomes and quality of life. Br J Psychiatry. 2004;184:346-351. doi: 10.1192/bjp.184.4.346.
7. Fitzgerald P, de Castella A, Arya D, et al. The cost of relapse in schizophrenia and schizoaffective disorder. Australas Psychiatry. 2009;17(4):265-272. doi: 10.1080/10398560903002998.
8. Weiden PJ, Olfson M. Cost of relapse in schizophrenia. Schizophr Bull. 1995;21(3):419-429. doi: 10.1093/schbul/21.3.419.
9. Cañas F, Alptekin K, Azorin JM, et al. Improving treatment adherence in your patients with schizophrenia: the STAY initiative. Clin Drug Investig. 2013;33(2):97-107. doi: 10.1007/s40261-012-0047-8.
10. Emsley R, Chiliza B, Asmal L, Harvey BH. The nature of relapse in schizophrenia. BMC Psychiatry. 2013;13:50. doi: 10.1186/1471-244X-13-50.
11. Olivares JM, Sermon J, Hemels M, Schreiner A. Definitions and drivers of relapse in patients with schizophrenia: a systematic literature review. Ann Gen Psychiatry. 2013;12(1):32. doi: 10.1186/1744-859X-12-32.
12. Velligan DI, Sajatovic M, Hatch A, Kramata P, Docherty JP. Why do psychiatric patients stop antipsychotic medication? a systematic review of reasons for nonadherence to medication in patients with serious mental illness. Patient Prefer Adherence. 2017;11:449-468. doi: 10.2147/PPA.S124658.
13. Carpenter WT Jr. Schizophrenia: disease, syndrome, or dimensions? Fam Process. 2007;46(2):199-206. doi: 10.1111/j.1545-5300.2007.00204.x.
14. Kirkpatrick B, Buchanan RW, Ross DE, Carpenter WT Jr. A separate disease within the syndrome of schizophrenia. Arch Gen Psychiatry. 2001;58(2):165-171. doi: 10.1001/archpsyc.58.2.165.
15. Gaebel W, Riesbeck M. Revisiting the relapse predictive validity of prodromal symptoms in schizophrenia. Schizophr Res. 2007;95(1-3):19-29. doi: 10.1016/j.schres.2007.06.016.
16. Ruetsch C, Un H, Waters HC. Claims-based proxies of patient instability among commercially insured adults with schizophrenia. Clinicoecon Outcomes Res. 2018;10:259-267. doi: 10.2147/CEOR.S149519.
17. Chien WT, Leung SF, Yeung FK, Wong WK. Current approaches to treatments for schizophrenia spectrum disorders, part II: psychosocial interventions and patient-focused perspectives in psychiatric care. Neuropsychiatr Dis Treat. 2013;9:1463-1481. doi: 10.2147/NDT.S49263.
18. Zygmunt A, Olfson M, Boyer CA, Mechanic D. Interventions to improve medication adherence in schizophrenia. Am J Psychiatry. 2002;159(10):1653-1664. doi: 10.1176/appi.ajp.159.10.1653.