Use of Administrative Data to Identify Health Plan Members With Unrecognized Bipolar Disorder: A Retrospective Cohort Study

Objective: This retrospective cohort study used an algorithmiccase-finding system on claims data from nationwide commercialhealth plans to validate previously identified predictors of unrecognizedbipolar disorder among adults.

Study Design: Retrospective cohort design.

Methods: Using logistic regression, 2 claims data sets wereevaluated to explore potential predictors; the first included claimsfor all healthcare encounters (all-encounters data set); the secondexcluded mental health provider claims (carve-out data set). A totalof 280 244 members aged 18 to 64 years were included from 2commercial health plans.

Results: Claims related to attention deficit-hyperactivity disorder,depression, depression treated with antipsychotics, use of 3 (of5) classes of psychotherapeutic drugs, younger age, and sex wereall significant predictors of a subsequent diagnosis of bipolar disorder.In the all-encounters data set, a predicted value of 5% orgreater yielded a sensitivity of 9.8% and a specificity of 99.9%; apredicted threshold of 3% increased sensitivity to 20.7%; areaunder the receiver operating characteristic curve (AUC) was 0.82.Performance of the model was acceptable in the carve-out data set,with AUC 0.69.

Conclusions: The case-finding system described here, whichcompares favorably with other screening tests used in primary care,may have significant value in helping physicians to identify patientswith unrecognized bipolar disorder.

(Am J Manag Care. 2005;11:578-584)

From the Outcomes Management Department, Active Health Management, Inc, NewYork, NY (IJ); the Outcomes Research Department, Eli Lilly and Company, Indianapolis, Ind(MS); Brauer Biomed, LLC, Minneapolis, Minn (LB); and the Mental Health Department, VAMedical Center, Minneapolis, Minn (PT).

Funding for this study was provided by Eli Lilly and Company, Indianapolis, Ind.

Address correspondence to: Iver Juster, MD, 45 Rodeo Avenue, No. 2, Sausalito, CA94965. E-mail: iverjuster@aol.com.

Bipolar disorder is a chronic, disabling mental illnessthat can have devastating consequences forthe patient and poses a substantial burden to thehealthcare system. Although bipolar disorder is relativelyuncommon, patients with the disorder utilize disproportionatemedical and psychiatric healthcareresources.^1,2Proper treatment of bipolar disorder canprevent psychosocial morbidity and relapses, althoughpatients must be followed closely and treatment regimensmay require ongoing adjustment.^3-5

The prevalence of bipolar disorder is difficult todetermine, in large part because of controversy and lackof clarity regarding diagnostic criteria. The diagnosis ofbipolar I, characterized by a frank manic episode, is lessambiguous than the diagnosis of bipolar II, whichdepends on accurate characterization of a hypomanicepisode. Whereas estimates of the lifetime prevalence ofbipolar disorder range from 1%⁶to 11%⁷across studies,prevalence estimates for clearly defined bipolar disorderI and II generally fall in the range of 1% to 4%, with bipolarI clustering around 1%.

Bipolar illness is frequently misdiagnosed or unrecognized.In 1 survey, 70% of patients with bipolar disorderreported being misdiagnosed at some point,⁸oftenwith major depression. Another study found that onaverage 8 years elapsed between initial presentationand diagnosis.⁹Such diagnostic and treatment delayscan lead to poor outcomes.¹⁰Inappropriate treatmentfor (misdiagnosed) unipolar depression may exacerbatesymptoms and induce cycling to mania or hypomania.^7,11Furthermore, retrospective analyses indicatethat healthcare costs may be higher for bipolar patientswhose diagnosis is delayed.^12,13

Because earlier identification of bipolar disordercould lead to improved outcomes and lower costs, innovativemethods to facilitate its early identification areneeded. One recently developed tactic is case-findingsystems using standardized claims, prescription fills,and laboratory results data with clinical algorithms toidentify patients whose medical care appears to deviatefrom accepted standards. Such a system can be used toscreen large numbers of claims for various treatmentpatterns that may indicate a missed bipolar disorderdiagnosis, prompting contact with the healthcareprovider for secondary screening with a brief questionnaire.Several such questionnaires have been validatedto screen for bipolar disorder in primary care settings,including the Mood Disorders Questionnaire¹⁴and theHypomanic Personality Scale.¹⁵These instruments haveproven useful for identifying patients who may havebipolar disorder, but first the physician must considerthe diagnosis. A reasonably effective claims-basedscreening tool could help primary care physicians identifythe up to 30% of patients with affective disorderswho may have bipolar disorder.¹⁶

In a previous exploratory case-control study (unpublished),we tested the ability of several claims-codablecharacteristics (developed by a panel of bipolar disorderexperts) to discriminate healthplan members with bipolardisorder from members with unipolar depression orneither diagnosis. We found that diagnoses or medicationsfor unipolar depression, psychotic depression,attention deficit-hyperactivity disorder (ADHD), conductor impulse control disorders, and prescriptions formultiple classes of psychotropic medications were significantdiscriminators.

The purpose of this study was to replicate our previousfindings using a retrospective cohort design on 2nationwide commercial health plan claims data sets: 1including all encounters and 1 with a mental health"carve-out" (missing encounters with the identifiedmental healthcare system, but not prescriptions). These2 databases reflected 2 common managed care scenariosand allowed us to replicate the predictive accuracywith mental health services "carved in" and extend thefindings to the mental health "carved out" scenario.This study assessed the accuracy of a predictive modelof bipolar disorder based on claims clues.

METHODS

To construct and validate the predictive model, weutilized "claims clues" developed by a bipolar disorderexpert consensus panel and tested in the previous case-controlstudy. The current study extended the previousfindings in additional data sets, using a retrospectivecohort design.

Data

all-encounters

data set

mental health carve-out data

set.

The present study utilized the variables from thedevelopment study in 2 different data sets representing2 common mental health payment scenarios: an and a The carve-out data set excluded claims from mentalhealth providers; both included all filled prescriptions.Both data sets contained only members of commerciallyinsured nationwide health maintenance organizationsor preferred provider organizations aged 18 to 64years as of January 1, 2001, who were continuouslyenrolled from January 1, 2000, through September 30,2003. The all-encounters set contained all claims for40 244 members who met the age and enrollment criteria.The carve-out set, representing a different healthplan, contained all noncarved-out claims and all prescriptionfills for 240 000 members who met the age andenrollment criteria.

Procedures

antecedents period

recognition period

For each data set, we defined an as containing claims with dates of service in 2000, anda as containing claims with dates ofservice from January 1, 2001, through September 30,2003. The algorithmic case-finding system was designedto search for "bipolar clues" in claims from the year2000, from the standpoint of January 1, 2001.

International Classification of

Diseases, 9th Revision, Clinical Modification [ICD-9-

CM]

For the all-encounters set, which included mentalhealth providers' claims, we defined the term "bipolardisorder" based on the presence of at least 2 claims forbipolar disorder (codes 296.1, 296.4-296.8). For the carve-out set, wedefined bipolar disorder as the presence of at least 2claims for bipolar disorder, at least 2 claims for lithium,or at least 2 claims for valproic acid derivatives unlessthe patient had a claim for any of the following diagnoses:migraine (346.xx); epilepsy (345.xx); convulsions(not otherwise specified) (780.39); undersocialized conductdisorder, aggressive type (312.34); or isolatedexplosive disorder (312.35). This medication-inclusivedefinition of bipolar disorder was used to capture individualswith bipolar disorder in the carve-out set whomight not have claims with a bipolar disorder diagnosisfrom providers outside the mental health carve-out.

To increase the diagnostic specificity, several typesof individuals were excluded from the analysis: individualswith preexisting bipolar disorder, a single bipolardiagnosis during the antecedents period, or only a singlebipolar diagnosis during the recognition period.Figures 1 and 2 describe the categorization based onthese restrictions. In the all-encounters data set weconsidered individuals to have bipolar disorder onlywhen their diagnoses were made by a mental healthprovider, based on the following criteria: (1) at least 1hospitalization with a diagnosis in the mental healthICD cluster or (2) at least 2 claims with a CPT codebetween 90801 and 90899.

In both data sets the base-rates of newly diagnosedbipolar disorder were low. For the all-encounters dataset 82 (0.20%) members were identified as having bipolardisorder during the recognition period. For the carve-outset, 1081 (0.45%) members were identified ashaving bipolar disorder during the recognition period.

Construction of the Predictive Model

ICD-9-CM

A production algorithmic case-finding system (theCareEngine^SMSystem, Active Health Management, Inc,New York) was used to identify possibly unrecognizedcases of bipolar disorder. The system applied clinicalrules to healthcare encounter claims (codes), prescription fills (National Drug Codes), age,and sex. All members in each data set who did nothave a bipolar disorder diagnosis during theantecedents period were loaded into the case-findingsystem and evaluated from the standpoint of January 1,2001. The system evaluated each member using claimswith dates of service from January 1, 2000, throughDecember 31, 2000 (the antecedents period), searchingfor the presence of predictor variables.

For each data set, we also used logistic regression¹⁷(SPSS V.11, Chicago, Ill) to predict new bipolar diagnosisduring the recognitionperiod using all the predictorvariables as well as age andsex. For the all-encountersdata set 2 variables (impulsecontrol and conduct controldisorders) were removedbecause they were not positivefor any of the 82 individualswith a new diagnosis ofbipolar disorder; thereforethe equation could not be fitwith them. Because all of thepredictors had been selectedbased on expert opinion andpreviously substantiated, weconstructed an all-predictorsmodel (regardless of statisticalsignificance). In eachdata set the model performancewas further characterizedas area under the curve(AUC) of the receiver operatingcharacteristic (ROC)curve.¹⁸

RESULTS

all-encounters data

set,

The predictor variablesentered into the case-findingsystem are listed in Table 1.In the no individuals with bipolardisorder diagnosed duringthe recognition period exhibitedthe impulse control, conductdisorder, or substanceabuse variables. In multivariatelogistic regression enteringall remaining variables, sex, age, use of 3 (of 5)psychotherapeutic medications, and depression werefound to be independently predictive of subsequentrecognition of bipolar disorder. Gender was found tohave opposite associations in the data sets with subsequentrecognition of bipolar disorder, as shown in Table2: negative for the "carve-out" but positive in the "allencounters." The other significant associationsappeared in the same direction in both data sets.

carve-out data set,

Table 2 shows the regression equation fitted from thevariables representing those triggered by any patientwith bipolar disorder in the all-encounters set, plus sexand age. The overall performance of the model on theall-encounters set can be expressed by the AUC of theROC curve. The AUC for themodel on the all-encountersset was 0.82 (95% confidenceinterval [CI], 0.76-0.87). In the largerrecognitionperiod patients withbipolar disorder triggered allcandidate variables.Although this model did notfit as well to the carve-outset, performance was significantlybetter than chance,with an AUC of 0.69 (95%CI, 0.67-0.70).

For the all-encountersdata set, a threshold of 5%predicted probability ofbipolar disorder found 8 ofthe 82 individuals with bipolardisorder (ie, a sensitivityof 9.8%) and correctly identified39 940 of the 39 996individuals without bipolardisorder (for a specificity of99.9%). Reducing the detectionthreshold to 3%increased the sensitivity to20.7%, with 99.4% specificity;however, the number ofpositive tests needed toidentify a case increasedfrom 8 using the 5% thresholdto 15.6 with the 3%threshold.

DISCUSSION

all-encounters data set

carve-out data

set

The purpose of this studywas to create a practical predictivemodel to support identification of unrecognizedbipolar disorder using claims data from commerciallyinsured adults and an algorithmic case-finding system.Using claims for adult members (continuously enrolledfor 3.75 years) of 2 commercial health plans, we developedand validated a predictive model based on claims-codedrules for identification of unrecognized bipolardisorder developed by a bipolar expert panel and substantiatedin a previous case-control study. The goal ofthe model was to predict subsequent recognition ofbipolar disorder in individuals with no evidence of thedisorder during the first year. We validated the model in2 common scenarios relating to how health plans payfor mental health services. An contained claims for all encounters; a excluded claims from mental health providers; bothincluded all prescriptions.

The model demonstrated that in both data sets, sex,age, history of ADHD, psychotic depression, history ofuse of multiple categories of psychotherapeutic medication,and depression were significantly associatedwith the subsequent recognition of bipolar disorder. Anunexpected finding was that sex was a negative associationfor "carve-out" but positive for "all-encounters."We postulate that this perplexing finding may relate tothe differences in our operational definitions for bipolar disorder, the sequestration of claims in the carved-outmental health networks, or differences in the populations.In the carve-out data set we used a lessstringent definition (based on any bipolar diagnoses orspecific patterns medication use; see Methods section)that yielded a newly diagnosed bipolar disorder base-rateof 0.45%, whereas in the all-encounters data set weused a more stringent definition of 2 bipolar claims bymental health providers that yielded a base-rate of only0.20%.

Performance of the algorithmic case-finding systemwas acceptable for a primary screening tool with low riskof misidentification of false positives, as shown by a sensitivityof 9.8% and a specificity of 99.9% using a predicteddiagnosis threshold of 5%, and a sensitivity andspecificity of 20.7% and 99.4%, respectively, using a 3%threshold in the carve-out set. Sensitivities in the allencountersset were slightly higher. A lower sensitivitywas expected in the carve-out set, in which some cases ofbipolar disorder were identified by relatively (but notcompletely) specific mood stabilizer therapy. Neverthelessthe area under the ROC curve, a test performancemeasure relating the calculated probability to each individual'sactual state, was highly significantly better thanchance. The AUC indicated that 82% of the time themodel would accurately discriminate a randomly selectedindividual with bipolar disorder from a randomlyselected individual without bipolar disorder. Althoughindividuals in clinical practice who screen positive forbipolar disorder based on this predictive model will infact have it much less frequently (because of the condition'slow prevalence), the positive predictive values atthe 5% and 3% prediction thresholds in our study comparefavorably with those of many commonly advised primarycare screening tests.^19-21Further improvement ofthe model's performance with a prospective study couldreduce the number needed to secondary screen and capturemore individuals with bipolar disorder.

This study looked for subsequent recognition of bipolardisorder during a relatively short prediction interval, 2.75years. Although this strategy was practical given theturnover commonly observed in health plans, this amountof time may be less than is commonly needed to make anaccurate diagnosis,⁹and therefore the possibility existsthat more diagnoses of bipolar disorder might have beenmade with a longer recognition period. Nevertheless, theprevalence of bipolar diagnosis in the mental health carve-outdata set was approximately 1%, similar to the prevalence cited in epidemiological studies for bipolar I.⁶Furthermore, the time during which claims were examinedwas sufficient to yield impressive sensitivity andspecificity estimates in identifying people who were laterrecognized as having bipolar disorder.

It may be impractical to execute a claims-based studyon a longer-term continuous health plan enrollment dataset. A prospective study would address the often-citeddiagnostic delay associated with bipolar disorder, as wellas the issue of error in ICD coding by physicians.^22-24Discrepancies across claims-based studies may also berelated to differences in criteria for determining who hadbipolar disorder. For example, in some claims-basedstudies, bipolar disorder is defined as a single ICD codeand sometimes as a single prescription for a mood stabilizer,without exclusionary diagnoses in the case ofvalproic acid derivatives. In this study, more rigorouscriteria were used to define bipolar disorder, includingthe absence of exclusionary diagnoses.

Determining a "diagnosis" of ADHD or depressionduring the antecedents period using medications commonlyprescribed for these conditions might yield false-positiveADHD or depression "diagnoses," as thesemedications may be used to treat other conditions (eg,bupropion for smoking cessation). However, the accuracyof the claims-based ADHD and depression diagnosesis not of primary importance for the predictive model;it is the ability of these claims-based ADHD anddepression "diagnoses" to accurately predict missedbipolar disorder that is crucial.

With any screening test, it is important to considerthe potential burden of screening results on the physicianand the healthcare system. At a predictive thresholdof 5% probability (from the regression equation), of1000 patients identified from the predictive model, 125would have bipolar disorder; of 1000 patients with bipolardisorder, 98 would be found and 902 missed; thenumber needed to subject to secondary screening(NNS) to identify 1 case of bipolar disorder would be 8.At the 3% threshold, of 1000 identified from the predictivemodel, 64 would have bipolar disorder; 207 of 1000with bipolar disorder would be accurately identified;and NNS would be 15.6. Thus the predictive "score" orthreshold could be set to find a reasonable proportion ofcases without undue burden, with the knowledge of thelikely proportion of missed cases.

Early identification and proper treatment of bipolardisorder can reduce healthcare cost and work-loss, andimprove psychosocial function. A shorter diagnosticdelay means less opportunity for inappropriate treatment(eg, antidepressant monotherapy, which can hastenthe switch to mania). Further, delayed treatment isassociated with worse outcome.¹⁰Early identification isalso important from the health plan perspective. In theUnited States alone, the total lifetime cost of care forindividuals with bipolar disorder with onset of illness in1998 was $24 billion.²⁵During a 1-year period, patientswith bipolar disorder were found to cost nearly 4 timesmore than age- and sex-matched individuals withoutthe illness ($7663 vs $19 622). The situation is exacerbatedfor patients with unrecognized bipolar disorder,who have been shown to have higher rates of hospitaluse and attempted suicide compared with patients withrecognized bipolar disorder. Thus, it is reasonable toexpect that care providers and health plans could substantiallybenefit from the use of a predictive model orcase-finding algorithm.

Given the routine underrecognition of bipolar disorder,its devastating consequences for patients, and itssignificant cost to health plans, a case-finding algorithmthat could be used to identify patients with risk factorsearly in the disease course would be expected to contributesubstantially to the management of bipolar disorder.Such a system, based on readily availableadministrative data, has real-world practicality and canbe used to screen millions of claims in a day. Indeed,such systems are beginning to see widespread implementationfor other conditions.

We propose that such a system could be used to sortindividuals identified into 2 levels of intervention basedon their predicted likelihood of having bipolar disorder.For example, physicians of patients predicted at greaterthan 5% risk by regression equation could receive a validatedbrief screening tool (such as the Mood DisordersQuestionnaire); physicians of individuals who triggereda 3% risk by regression equation might receive a recommendationto consider the diagnosis and might beurged to use the screening tool if the physician considersthe diagnosis a possibility. We hypothesize thatsuch a system could considerably reduce the biopsychosocialand financial costs of unrecognized bipolardisorder. Prospective studies with a large number ofclaims and clinical follow-up of identified patients willbe needed to determine the actual effect of a case-findingsystem.

Acknowledgments

We thank Thom Shannon, Jr, MAS, at Health Data ManagementSolutions, Inc, for HIPAA-compliant preparation of the data sets.

Funding for this study was provided by Eli Lilly and Company, Indianapolis, Ind.

Address correspondence to: Iver Juster, MD, 45 Rodeo Avenue, No. 2, Sausalito, CA94965. E-mail: iverjuster@aol.com.

Am J Psychiatry.

1. Peele PB, Xu Y, Kupfer DJ. Insurance expenditures on bipolar disorder: clinicaland parity implications. 2003;160:1286-1290.

Bipolar Disord.

2. Bryant-Comstock L, Stender M, Devercelli G. Health care utilization and costsamong privately insured patients with bipolar I disorder. 2002;4:398-405.

Arch Gen Psychiatry.

3. Tohen M, Waternaux CM, Tsuang MT. Outcome in mania. A 4-year prospectivefollow-up of 75 patients utilizing survival analysis. 1990;47:1106-1111.

J Clin Psychiatry.

4. Bowden CL. Efficacy of lithium in mania and maintenance therapy in bipolardisorder. 2000;61(suppl 9):35-40.

Am J Psychiatry.

5. American Psychiatric Association. Practice guideline for the treatment ofpatients with bipolar disorder (revision). 2002;159(4 suppl):1-50.

Arch Gen Psychiatry.

6. Kessler RC, McGonagle KA, Zhao S, et al. Lifetime and 12-month prevalenceof DSM-III-R psychiatric disorders in the United States. Results from the NationalComorbidity Survey. 1994;51:8-19.

World Psychiatry.

7. Angst J, Gamma A, Lewinsohn P. The evolving epidemiology of bipolar disorder.2002;1:146-148.

J Affect Disord.

8. Lish JD, Dime-Meenan S, Whybrow PC, Price RA, Hirschfeld RM. TheNational Depressive and Manic-depressive Association (DMDA) survey of bipolarmembers. 1994;31:281-294.

Am J Psychiatry.

9. Baldessarini RJ, Tondo L, Hennen J. Treatment delays in bipolar disorders [letter].1999;156:811-812.

Am J Psychiatry.

10. Swann AC, Bowden CL, Calabrese JR, Dilsaver SC, Morris DD. Differentialeffect of number of previous episodes of affective disorder on response to lithiumor divalproex in acute mania. 1999;156:1264-1266.

Am J Psychiatry.

11. Wehr TA, Sack DA, Rosenthal NE, Cowdry RW. Rapid cycling affective disorder:contributing factors and treatment responses in 51 patients. 1988;145:179-184.

J Affect Disord.

12. Li J, McCombs JS, Stimmel GL. Cost of treating bipolar disorder in theCalifornia Medicaid (Medi-Cal) program. 2002;71:131-139.

J Clin Psychiatry.

13. Birnbaum HG, Shi L, Dial E, Oster EF, Greenberg PE, Mallett DA. Economicconsequences of not recognizing bipolar disorder patients: a cross-sectionaldescriptive analysis. 2003;64:1201-1209.

Am J Psychiatry.

14. Hirschfeld RM, Williams JB, Spitzer RL, et al. Development and validation ofa screening instrument for bipolar spectrum disorder: the Mood DisorderQuestionnaire. 2000;157:1873-1875.

J Abnorm Psychol.

15. Eckblad M, Chapman LJ. Development and validation of a scale for hypomanicpersonality. 1986;95:214-222.

Compr Psychiatry.

16. Manning JS, Haykal RF, Connor PD, Akiskal HS. On the nature of depressiveand anxious states in a family practice setting: the high prevalence of bipolar IIand related disorders in a cohort followed longitudinally. 1997;38:102-108.

Applied Logistic Regression.

17. Hosmer D, Lemeshow S. 2nd ed. New York, NY:John Wiley & Sons; 2000.

Radiology.

18. Hanley JA, McNeil BJ. The meaning and use of the area under a receiveroperating characteristic (ROC) curve. 1982;143:29-36.

Ann Intern Med.

19. Coley CM, Barry MJ, Fleming C, Mulley AG. Early detection of prostate cancer.Part I: prior probability and effectiveness of tests. The American College ofPhysicians. 1997;126:394-406.

Am Coll Cardiol.

20. Califf RM, Armstrong PW, Carver JR, D'Agostino RB, Strauss WE. 27thBethesda Conference: matching the intensity of risk factor management with thehazard for coronary disease events. Task Force 5. Stratification of patients intohigh, medium, and low risk subgroups for purposes of risk factor management. 1996;27:1007-1019.

JAMA.

21. Kerlikowske K, Grady D, Barclay J, Sickles EA, Eaton A, Ernster V. Positivepredictive value of screening mammography by age and family history of breastcancer. 1993;270:2444-2450.

J Fam Pract.

22. Chao J, Gillanders WG, Flocke SA, Goodwin MA, Kikano GE, Stange KC.Billing for physician services: a comparison of actual billing with CPT codesassigned by direct observation. 1998;47:28-32.

Ambul Care Manage.

23. Fowles JB, Fowler EJ, Craft C. Validation of claims diagnoses and self-reportedconditions compared with medical records for selected chronic diseases. 1998;21(1):24-34.

J Am Board Fam Pract.

24. King MS, Sharp L, Lipsky MS. Accuracy of CPT evaluation and managementcoding by family physicians. 2001;14:184-192.

PharmacoEconomics.

25. Begeley CE, Annegers JF, Swann AC, et al. The lifetime cost of bipolar disorderin the US: an estimate for new cases in 1998. 2001;19:483-495.