Currently Viewing:
The American Journal of Managed Care November 2013
Opioid Analgesic Treated Chronic Pain Patients at Risk for Problematic Use
Joseph Tkacz, MS; Jacqueline Pesa, PhD, MPH; Lien Vo, PharmD, MPH; Peter G. Kardel, MA; Hyong Un, MD; Joseph R. Volpicelli, MD, PhD; and Charles Ruetsch, PhD
Safety and Effectiveness of Mail Order Pharmacy Use in Diabetes
Julie A. Schmittdiel, PhD; Andrew J. Karter, PhD; Wendy T. Dyer, MS; James Chan, PharmD, PhD; and O. Kenrik Duru, MD, MSHS
Depression Self-Management Assistance Using Automated Telephonic Assessments and Social Support
John D. Piette, MSc, PhD; James E. Aikens, PhD; Ranak Trivedi, PhD; Diana Parrish, MSW; Connie Standiford, MD; Nicolle S. Marinec, MPH; Dana Striplin, MHSA; and Steven J. Bernstein, MD, MPH
Creating Peer Groups for Assessing and Comparing Nursing Home Performance
Margaret M. Byrne, PhD; Christina Daw, PhD; Ken Pietz, PhD; Brian Reis, BE; and Laura A. Petersen, MD, MPH
Upcoding Emergency Admissions for Non-Life-Threatening Injuries to Children
Zachary Pruitt, MHA; and Etienne Pracht, PhD
Variations in the Service Quality of Medical Practices
Dan P. Ly, MD, MPP; and Sherry A. Glied, PhD
Using Health Outcomes to Validate Access Quality Measures
Julia C. Prentice, PhD; Michael L. Davies, MD; and Steven D. Pizer, PhD
Currently Reading
Collecting Mortality Data to Drive Real-Time Improvement in Suicide Prevention
Brian K. Ahmedani, PhD; M. Justin Coffey, MD; and C. Edward Coffey, MD

Collecting Mortality Data to Drive Real-Time Improvement in Suicide Prevention

Brian K. Ahmedani, PhD; M. Justin Coffey, MD; and C. Edward Coffey, MD
The Perfect Depression Care initiative serves as an example of how suicide prevention programs can collect real-time mortality data internally to drive rapid quality improvement.
Objectives: To evaluate the use of government mortality records compared with internally collected data to drive quality improvement in suicide prevention programs using suicide mortality data from the Perfect Depression Care initiative.

Methods: Perfect Depression Care (PDC) is a quality improvement suicide prevention initiative within the Behavioral Health Services (BHS) division of the Henry Ford Health System. Eligible subjects were all patients who received services from BHS, were members of the health maintenance organization, and had a medical group physician during the 11-year study period. Mortality data were collected internally and from government-collected death records, and were linked to treatment utilization data from the medical record.

Results: The mean suicide rate was 96.6 per 100,000 during the 2-year baseline period (1999- 2000) and declined to 19.1 per 100,000 during the initiative (2002-2009) using both sources of data combined. We observed a similar statistically significant (P <.001) reduction in the suicide death rate using both the internal and government data sources. There were no significant differences between the 2 sources of data in the mean suicide rates for the baseline and intervention periods (P >.05). The data sources did differ in the capture of unique suicide deaths.

Conclusions: Internally collected data were an effective measure of suicide deaths in the PDC initiative. A combination of internal and government-collected records may be most effective for future suicide prevention programs.

Am J Manag Care. 2013;19(11):e386-e390
Health systems trying to implement suicide prevention programs are limited by their inability to access real-time suicide mortality data to inform rapid-cycle quality improvement.
  • As demonstrated by the Perfect Depression Care initiative, suicide data can be collected internally to drive rapid quality improvement.

  • Internal suicide data should be compared with official government records after their release 2 years later.
Suicide is a major public health concern and it remains a leading cause of death in the United States.1-7 Population-level estimates of suicide have not improved over time,2,8,9 leading the Institute of Medicine to declare suicide prevention “a national imperative.”10 While some suicide prevention strategies exist,11 they are difficult to evaluate due to the rarity of suicide deaths and the inefficiency in the current methods for counting and measuring suicide deaths.

One suicide prevention initiative that has received much attention, including receipt of the 2006 Gold Achievement Award from the American Psychiatric Association and the 2006 Earnest Amory Codman Award from the Joint Commission, is the Perfect Depression Care (PDC) initiative12,13 at Henry Ford Health System (HFHS). Perfect Depression Care is a health system quality improvement initiative associated with reductions in suicide deaths among patients in its health maintenance organization (HMO) network.14

As described previously, the PDC initiative is unique in its use of internally collected suicide data for rolling program evaluation rather than using government records,13 which are traditionally used for population mortality research.15 This internal surveillance method  of suicide ascertainment was chosen to drive real-time process improvement, such as to inform protocol as well as provider- and system-level changes. Such rapid-cycle improvement would not be feasible with traditional government records, because it can take up to 2 years before they are available. The aim of this study was to compare the agreement between our internal collected data on suicide and the data obtained from official government mortality records.


Setting and Population

The Henry Ford Health System is an integrated healthcare organization serving southeastern Michigan. It consists of multiple hospitals, a large medical group, and a large HMO. The Behavioral Health Services (BHS) division offers specialty mental health services within HFHS. Eligible subjects included all patients who made any service contact at BHS between January 1, 1999, and December 31, 2009, and were enrolled in the HMO network during the year in which they received care. The total sample size of all eligible individuals varied each year, ranging from 6168 to 10,831 patients (Figure). The HMO population, from which the sample for this study was derived, is broadly representative of southeast Michigan with respect to age, sex, race/ethnicity, and socioeconomic status. This project was approved by institutional review boards at Henry Ford Hospital (IRB# 6821) and the state of Michigan (IRB# 953-PHALHAS).

Perfect Depression Care Initiative

The concept and components of the PDC initiative are described in more detail elsewhere.12,13 The PDC initiative began in response to the Institute of Medicine report titled Crossing the Quality Chasm: A New Health System for the 21st Century, which identified depression as a key condition of importance.16 We named our initiative Perfect Depression Care to reflect the Institute of Medicine’s audacious call to pursue a system of perfect healthcare. In such a system, persons receiving treatment for mental disorders would not die from suicide—thus, we adopted the elimination of suicide as one of PDC’s primary aims. To pursue this aim, we developed a bundle of core interventions organized into 3 main categories, which have been described elsewhere.12 Briefly, these categories include (1) a distinct effort to establish a “just culture” in which departmental operations and clinical care are aligned with the aims of the Institute of Medicine report, (2) comprehensive evaluations for every patient using a multilevel suicide risk assessment, and (3) a redesigned care delivery system based on the Chronic Care Model.17

Mortality Data

Date and cause of death were determined from 2 sources of data: internally from a clinical suicide surveillance system and externally from the state of Michigan (SoM) mortality records. For the remainder of this study, the term “suicide” refers to death by suicide. Internal data were collected as they were reported to BHS from any source, generally by a family member, care provider, or HMO representative. In the latter scenario, BHS collected information from the HMO on the reason for death when an individual was no longer enrolled in the health plan and was listed as deceased. De-enrollment from the health plan occurred immediately during the month of death. Finally, sources of suicide information also included information from media outlets or any other means of identification in which a provider or the department identified the suicide victim as being a patient. Internally collected suicides were investigated using TapRooT, a root cause analysis system allowing the PDC team to investigate each incident by reviewing prior treatment history, involvement in care, access to means, and system-level errors.

External suicide data came from the SoM mortality records,18 where cause of death is described by a medical examiner and coded by government officials using the International Classification of Diseases, 10th Revision (ICD-10). After initial coding by the state, the preliminary mortality information is sent to the Centers for Disease Control and Prevention’s (CDC's) National Center for Health Statistics. There, the National Vital Statistics System processes and corrects all mortality records before sending the final corrected version back to each state and region. Approximately 2 years after the end of each calendar year, HFHS receives the final corrected SoM data set.

In the current study, all deaths coded as X60-X84 or Y87.0 were identified as suicides. This coding scheme is recommended by the CDC.19 These data were linked to HFHS records using a 2-step process. First, social security numbers were matched between the SoM data set and the health system’s administrative data from the medical record. Then name, date of birth, address, and sex were used to identify additional matches. The matching process also complies with the CDC protocol for mortality death linkage.20


Both data sources were collected independently and matched to medical record data to develop a single data set using SAS version 9.2.21 A trained psychiatrist confirmed the accuracy of the data by conducting medical chart reviews of each case. The research team then reviewed each case to ensure agreement. Using the total number of suicides each year divided by the total number of eligible subjects receiving care that year, we calculated 4 distinct suicide rates: (1) internal data only, (2) SoM data only, (3) either internal or SoM data (“combined”), and (4) both internal and SoM data (“matched”). The suicide rates were adjusted to provide a rate per 100,000 persons for each year, as shown in the Figure.

Poisson regression models were used to compare the change in mean suicide rate with the PDC initiative (implemented in 2001) from baseline (1999-2000) to intervention (2002-2009). Paired-samples t tests were used to assess the degree of agreement between data sources. We then compared and contrasted the matched and mismatched cases from the 2 data sources using comprehensive medical chart review. The main analyses were conducted using Stata release 11.2.22


As shown in the Figure, throughout the observation period the mean suicide rate for patients in the HMO network dropped from the baseline period (1999-2000) to the intervention period (2002-2009). The data show a decline in the suicide rate of 82% (from 89 per 100,000 to 16 per 100,000; P <.001) using the SoM data and a similar decline by 86% (from 97 per 100,000 to 14 per 100,000; P <.001) using the internal data (Table 1). There were no statistical differences between the mean suicide rates for each observation period (baseline/intervention) as estimated by the 2 data sources (Table 2). Also, as shown in the Figure, for 5 of the 11 years

the suicide rates were identical. For the 6 years in which they differed, the SoM rates were slightly higher than the internal rate in 3 years and slightly lower in the other 3 years.

Although they yielded similar rates, the 2 data sources differed in actual cases captured (Table 3). The combined 31 cases of suicide included 23 matched cases (74%). For each of these cases, the 2 data sources also agreed on the manner of suicide. Of the 8 mismatched cases, the SoM data identified 4 cases that were missed by internal data. In each case, these patients were lost to follow-up and the medical record contained no reference to their death. Similarly, the SoM data missed 4 cases identified by the internal data: 2 cases were nonresidents of Michigan and therefore not captured by SoM records (1 Florida resident, 1 Canada resident) and the 2 other cases had their cause of death listed as unintentional self-poisoning, even though our internal system clearly identified them as suicides.

Overall, both data sources missed cases of suicide and, as such, appeared to underestimate the actual rate. Thus, the combined total (n = 31) appeared to be a better estimate of  the suicide rate. Using the combined data, the mean suicide rate was 97 per 100,000 for the baseline years and 19 per 100,000 for the intervention period (P <.001) (Table 1).


We found that our internally collected data on suicides agreed closely with official government-collected mortality data, although neither source captured all identified cases. This agreement in suicide rates between the 2 data sources suggests that internally collected mortality data are sufficient for driving real-time quality improvement work in suicide prevention. Official US mortality data from the National Center for Health Statistics are considered the gold standard for research,23 a consideration that may be founded on the grounds that no better alternative exists.15,24 On one hand, the findings from this study seem to affirm this conclusion. Both the internal and external methods identified—and missed—the same number of suicides. This finding suggests that while both methods are imperfect, neither is decidedly better than the other. On the other hand, quality improvement interventions are driven by small, rapid cycles of change that rely on realtime measurement of outcomes. Because official mortality records may not be readily available, they are less useful for driving quality improvement. Thus, an effective overall strategy for suicide prevention efforts might  be to use an internal real-time surveillance system for rapid cycles of learning while also using government-collected data, once they are available, to confirm or enhance that learning.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up