A Multistakeholder Effort in Pennsylvania to Improve the Accuracy of Reporting Fatal Drug Overdoses

ABSTRACT

The accuracy of information on death certificates, such as the underlying causes of death, the contributing factors, and the decedent’s sociodemographic background, has important and direct impacts on mortality data, policy making, and setting priorities in public health. A significant proportion of death certificates contain errors that affected the estimation and understanding of key population health indicators. In the context of the opioid epidemic and the COVID-19 pandemic, death certifiers faced unique challenges that were difficult to overcome. An interdisciplinary team developed a novel training program based on the Extension for Community Health Outcomes (ECHO) model for death certifiers in Pennsylvania. The contents of the training focused specifically on some of the main challenges in the death certification process and resulted in a useful model that might be adopted by other states or municipalities.

Am J Accountable Care. 2022;10(4):16-22. https://doi.org/10.37765/ajac.2022.89284

The role of mortality data in developing relevant and timely policy to address public health issues is paramount, even more so in the context of the ongoing opioid crisis and COVID-19 pandemic.¹ The accuracy and reliability of these data^2-4 are particularly concerning to both researchers and policy makers, who note the complexity of the death certification process⁵ in the United States and the diversity of the professionals involved.⁶ Similar to other policy regimes at local-state-federal intersections,⁷ the process of death certification involves different and sometimes competing stakeholder interests and demands.

Death certifiers include coroners, medical examiners, physicians, nurses, nurse practitioners, physician assistants, and others who may have limited medical training. The specific requirements for death certifiers vary across states. For example, 28 states elect their coroners locally, but only 4 require coroners to be physicians and 16 require them to receive initial and continuous education.⁸ Also, resources for investigations into causes of death are not equal across jurisdictions, inhibiting uniform and precise reporting on death certificates.⁹ Although the main sections of death certificates are standardized in the United States, the actual process of reporting—the way that death certifiers collect, interpret, and communicate information—can differ significantly.¹⁰ Because of the highly variable contexts of deaths, such as location, timing, information available (eg, the decedent’s medical history and sociodemographic background), and the different qualifications and experiences of the certifiers, efforts to standardize the certification process have met significant challenges.⁵ As a result, the information reported on death certificates has been persistently subject to questioning,^11,12 and more prevalent trainings and active outreach have been advocated to better engage the death certifier community and improve the quality of the death certification process.¹³

Based on firsthand experience from a newly developed program in Pennsylvania, we describe a novel training model for death certifiers and provide important reflections on the status of the death certification process, including some of the main challenges, and possible opportunities to address challenges. We discuss findings from the training participants’ needs assessment and feedback along with several program implementation considerations for other states or municipalities that may seek to develop and implement similar programs for their death certifier community. We conclude with thoughts on future directions.

Background

Findings of studies in various contexts have consistently shown that a significant proportion of death certificates contain errors that affect the estimation and understanding of key public health indicators.^14-16 The ongoing COVID-19 pandemic started in the United States in early 2020 and posed unique challenges to the death certification process.^17-19 On one hand, the urgency to trace COVID-19 mortality rates and understand key risk factors and comorbidities in a timely manner required careful, thorough, and prompt death reporting. On the other hand, the highly contagious nature of the virus, the concerns for the safety of death certifiers, and the potential further spread of the disease, especially at the early stage of the outbreak, made thorough postmortem investigations impractical in many COVID-19–related deaths.²⁰ Because COVID-19 was a relatively new condition at the time, there was no standardized case definition for more than 4 months after the first confirmed case, and as a result, death certifiers, including many physicians, felt utterly overwhelmed by the workload.^21,22 The pandemic also intensified the opioid crisis,²³ with its magnitude being significantly underestimated due to the large number of unidentified causes of deaths and unspecified overdose deaths reported.^15,24-27 For many certifiers, correctly identifying drug-related causes of death could be a daunting task because of limited information and resources available to them at the time of performing the task.^5,15,28-30

One way to improve death reporting and thus the accuracy and availability of the resulting data is the use of electronic death registration systems (EDRSs).³¹ With implementation as early as 2002, most states and other registration jurisdictions now have EDRSs,³² a technological advancement parallel to the adoption and use of electronic health records (EHRs) by health care providers. However, when adjusting practices based on EDRSs, death certifiers have encountered difficulties such as inconsistency in interfaces and data standards, limited access due to operating platforms and browsers, and obsolete information systems design.³³ In order for EDRSs to become more effective and user-friendly, improvement and standardization of these systems are needed. In particular, interoperability and information sharing between EDRSs and EHRs can provide quick and easy access to the decedent’s medical history and recent health care encounters and greatly improve the efficiency and accuracy of death reporting.³⁴ Achieving this goal also requires focused training of death certifiers at the local level.

A Novel Training Model for Death Certifiers in Pennsylvania: Motivation and Design

Pennsylvania has a decentralized system for death certification, with coroners elected and medical examiners appointed at the county level. Upon initial election or appointment, coroners and medical examiners are required by Pennsylvania state law to complete the Basic Education Course from the Coroners’ Education Board with a minimum of 32 hours of instruction and pass a written exam.³⁵ This required course includes topics on crime scene investigation, toxicology, and autopsy. There is also a requirement of at least 8 hours of annual continuing education, with additional optional courses on various topics, such as forensic anthropology and human osteology, available at a cost.³⁶ The importance and effectiveness of training programs for death certifiers were documented in previous studies.^37-39 In particular, interactive trainings were shown to be more effective than simply distributing educational materials.⁴⁰ The development of a new training model for death certifiers was initiated through a request from the Pennsylvania Department of Health (PADOH). Administrators of PADOH and death certifiers in Pennsylvania both recognized issues of accuracy and availability of existing data on substance-related factors contributing to deaths throughout Pennsylvania.^15,41 These issues prompted the exploration of opportunities to address them through a training program.

An interdisciplinary team from The Pennsylvania State University, in collaboration with the PADOH, implementation experts from the National Nurse-Led Care Consortium, and subject matter experts, developed a model for addressing the needs of the death certifier community in Pennsylvania (Figure). This model was informed by an initial review of the literature on effective modalities for training this community.^15,35,36,41 This team adopted an engagement strategy that included PADOH, the Pennsylvania State Coroners Association, and the death certifier community as key stakeholders to understand the needs of death certifiers in Pennsylvania. Accordingly, we designed a customized model focusing on accurate reporting of drug overdose deaths, with specific topics on improving the collective knowledge of and adherence to death reporting standards. Our training program can be modified for use in other contexts.

Implementation

To implement the model, as a first step, our team hosted several meetings with PADOH and the Pennsylvania State Coroners Association to learn about the complexities of death certification and drug overdose in Pennsylvania. We then developed and fielded a survey to assess the needs of the death certifier community in Pennsylvania. The main part of the questionnaire asked how confident they were in performing a set of important tasks during the death certification process. The Table summarizes the responses to the survey, completed by 40 coroners and 1 medical examiner (n = 41). Across the different tasks, the confidence level was relatively low in utilizing public health tools (eg, prescription drug monitoring programs [PDMPs], interactive data reports). When asked if they ever encountered a situation in their experience where the cause of death was difficult to identify or determine, the majority of the respondents (65.9%) reported they did. The greatest challenges in determining the cause of death included decomposed bodies, toxicology and autopsy reports coming back with no cause of death found, missing medical history, and lack of narratives for decedents’ final days. There were additional challenges if the decedent was hospitalized, as initial blood samples were often discarded after a few days. In their responses to an open-ended question on what would help death certifiers feel supported in their work in Pennsylvania, several common themes emerged, including additional education and training, the creation of a training reference manual that could be followed, and increased availability and resources for autopsy services. Finally, respondents highlighted the need for the EDRS to operate more efficiently with a mobile app to support the users and the importance of support and open communication from PADOH.

Based on this assessment, our team designed a training program for death certifiers in Pennsylvania with 5 topical areas addressing the identified needs (Table), delivered in 3 different components (modalities). All 5 topical areas were included in all 3 components of the training series, which were made available to all death certifiers in Pennsylvania at no cost. Participants in any of these training components can receive continuing education credits as needed.

As a first component, the team developed and delivered 5 synchronous online training sessions using the Extension for Community Health Outcomes (ECHO) model,⁴² which was pioneered at the University of New Mexico and has since been used globally to address major public health issues.⁴³ Launched at The Pennsylvania State University in 2018, Project ECHO has become a model for rapidly transferring knowledge and increasing capacity to deliver best-practice care to underserved populations in Pennsylvania. The core of ECHO, a guided model aimed at practice improvement, is its hub-and-spoke knowledge-sharing networks, led by expert teams (the “hub”) to mentor community health care providers (the “spokes”) via teleconferencing. ECHO participants retain responsibility for patients and operate with increasing independence as skills, confidence, and self-efficacy grow.^42,43

The second component was asynchronous online sessions, using the same content as the synchronous online component, which were made available through TRAIN PA, Pennsylvania’s learning platform. Participants can access them at any time at no cost, and the contents can be updated as necessary.

The third and final component was a series of daylong in-person trainings geographically dispersed throughout Pennsylvania, aimed at strengthening the key messages of the other 2 online training components. These training sessions were organized jointly by all involved stakeholders, conducted by regional members of the death certifier community, and hosted at sites affiliated with The Pennsylvania State University. In addition to delivering the content, the in-person trainings were also intended to increase local participation, improve community-university partnerships, and reduce training overhead costs.

Participants’ Feedback and Evaluation

An important part of our training model is to incorporate feedback from training participants and other stakeholders on early sessions to inform later sessions. Additionally, our team conducted a final evaluation that could be used to revise and improve the design of similar training programs in the future. The in-person trainings included 2 sets of evaluative feedback. First, at the end of each content module in the training, there was a listening session to gather feedback from the participating death certifiers. Second, a printed evaluation form was used to assess the overall participant satisfaction. The online synchronous training sessions incorporated a similar mechanism for participants’ feedback, collected electronically.

Feedback from the both the synchronous online training and in-person sessions indicated positive experiences, with participants reporting “a lot of helpful information,” “good discussion!,” and that “it was nice to talk to others in the death investigation field and compare common issues.” Meanwhile, some participants noted that “for coroners [who] have been in office for a long time, this was too basic,” indicating the need for further development of the content. Based on the majority of the responses, there was at least some value for training on all the covered topics, and the training was relevant to practices and increased the participants’ knowledge. However, at the same time, only a small proportion of responses indicated that there would be changes in the participants’ practices in the future. Similarly, only a small number of participants felt that professional isolation decreased due to the training. When asked to self-assess their knowledge before and after the training, participants reported improvement in all areas covered in the sessions, noting in particular the usefulness of discussions on EDRSs and the data from the PDMP. Finally, the feedback indicated the participants’ appreciation for the connection with PADOH via the training.

We utilized the aggregate feedback to inform some of the specific contents (eg, sample cases used in both the ECHO and in-person trainings) for training sessions to be conducted at a later time. In addition, the feedback from the ECHO sessions (eg, demand for information based on deep and varied field experience, as well as case study examples) provided useful insights for selecting individuals to conduct the in-person trainings, and the feedback from the in-person training motivated the creation of fact sheets to be included as companions to the online trainings for quick reference. This process of evaluation also increased partnerships between state and local stakeholders (eg, collaboration on creating the contents and format of the fact sheets).

Conclusions and Future Directions

Inaccuracies in death certification affect policy making in public health. In the context of the COVID-19 pandemic and the opioid crisis, death certifiers faced unique challenges that may be mitigated by tailored outreach and trainings. Our team developed a training model with multiple modalities for death certifiers in Pennsylvania. An important part of our implementation was to engage different key stakeholders and use their feedback to inform the design and delivery of training sessions. The results from the program evaluation indicated the need for further training and support for death certification process. An important limitation of the program is that the training participants have been almost exclusively coroners, possibly because most counties in Pennsylvania (64 of 67) use coroners rather than medical examiners. Another potential factor might be that the current contents in the training are less informative for medical examiners and some other death certifiers (eg, hospital-based physicians) due to their more extensive clinical background. Based on our experience, we identify 4 aspects to be improved in implementing this training model in the future: (1) broader inclusion of the death certifier community (professionals other than coroners) in the training to expand the reach of the information; (2) evaluating impact of the training on the accuracy of the information provided in the participants’ death certificates; (3) exploring opportunities to expand the use of case studies and simulations during the training sessions; and (4) developing networks for knowledge sharing among peers and across locations.

Author Affiliations: The Pennsylvania State University (YS, GS, KAW, LM, HE, IR, EA), University Park, PA; MRO Corporation (VD), Norristown, PA; National Nurse-Led Care Consortium (KG), Philadelphia, PA.

Source of Funding: This study was funded by the Pennsylvania Department of Health through the Coroner Education Project.

Author Disclosures: Drs Shi and Sterner report receiving funding for this study from the Pennsylvania Department of Health. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (YS, GS, KAW, LM, KG); acquisition of data (YS, GS, LM, VD, HE, KG, EA); analysis and interpretation of data (YS, GS, KAW, VD, HE, EA); drafting of the manuscript (YS, GS, KAW, LM, IR, KG, EA); critical revision of the manuscript for important intellectual content (YS, GS, KAW, VD, IR, KG, EA); statistical analysis (YS, GS, VD, EA); provision of study materials or patients (GS, KAW, HE); obtaining funding (YS, GS); administrative, technical, or logistic support (GS, KAW, IR, KG, EA); and supervision (YS, GS).

Send Correspondence to: Yunfeng Shi, PhD, The Pennsylvania State University, 504 Ford Bldg, University Park, PA 16802. Email: yus16@psu.edu.

REFERENCES

1. Ling SM, Warner M, Anderson RN. Opportunities to enhance the utility of cause of death information from death certificates. Am J Public Health. 2022;112(suppl 1):S42-S44. doi:10.2105/AJPH.2022.306722

2. Anderson BA, Silver BD. Issues of data quality in assessing mortality trends and levels in the new independent states. In: National Research Council. Premature Death in the New Independent States. The National Academies Press; 1997:120-155.

3. McGivern L, Shulman L, Carney JK, Shapiro S, Bundock E. Death certification errors and the effect on mortality statistics. Public Health Rep. 2017;132(6):669-675. doi:10.1177/0033354917736514

4. Nam YH, Shea DG, Shi Y, Moran JR. State prescription drug monitoring programs and fatal drug overdoses. Am J Manag Care. 2017;23(5):297-303.

5. Brooks EG, Reed KD. Principles and pitfalls: a guide to death certification. Clin Med Res. 2015;13(2):74-84. doi:10.3121/cmr.2015.1276

6. Aiken SS. Death certification in the United States. Am J Public Health. 2021;111(suppl 2):S55-S56. doi:10.2105/AJPH.2021.306443

7. Sterner G. A comparative case study of the Main Street Program in the United States. In: Vittuari M, Devlin J, Pagani M, Johnson TG, eds. The Routledge Handbook of Comparative Rural Policy. Routledge; 2019.

8. Coroner training requirements. CDC. January 15, 2015. Accessed March 30, 2022. https://www.cdc.gov/phlp/publications/coroner/training.html

9. Stokes AC, Lundberg DJ, Bor J, Bibbins-Domingo K. Excess deaths during the COVID-19 pandemic: implications for US death investigation systems. Am J Public Health. 2021;111(suppl 2):S53-S54. doi:10.2105/AJPH.2021.306331

10. National Research Council Committee on National Statistics. The U.S. Vital Statistics System: a national perspective. In: Vital Statistics: Summary of a Workshop. The National Academies Press; 2009:87-110.

11. Pritt BS, Hardin NJ, Richmond JA, Shapiro SL. Death certification errors at an academic institution. Arch Pathol Lab Med. 2005;129(11):1476-1479. doi:10.5858/2005-129-1476-DCEAAA

12. Krywanczyk A, Amoresano E, Tatsumi K, Mount S. Autopsy service death certificate review. Arch Pathol Lab Med. 2020;144(9):1092-1096. doi:10.5858/arpa.2019-0452-OA

13. Hart JD, Sorchik R, Bo KS, et al. Improving medical certification of cause of death: effective strategies and approaches based on experiences from the Data for Health Initiative. BMC Med. 2020;18(1):74. doi:10.1186/s12916-020-01519-8

14. Nielsen GP, Björnsson J, Jonasson JG. The accuracy of death certificates: implications for health statistics. Virchows Arch A Pathol Anat Histopathol. 1991;419(2):143-146. doi:10.1007/BF01600228

15. Buchanich JM, Balmert LC, Williams KE, Burke DS. The effect of incomplete death certificates on estimates of unintentional opioid-related overdose deaths in the United States, 1999-2015. Public Health Rep. 2018;133(4):423-431. doi:10.1177/0033354918774330

16. Schuppener LM, Olson K, Brooks EG. Death certification: errors and interventions. Clin Med Res. 2020;18(1):21-26. doi:10.3121/cmr.2019.1496

17. Gill JR, DeJoseph ME. The importance of proper death certification during the COVID-19 pandemic. JAMA. 2020;324(1):27-28. doi:10.1001/jama.2020.9536

18. Weinberger DM, Chen J, Cohen T, et al. Estimation of excess deaths associated with the COVID-19 pandemic in the United States, March to May 2020. JAMA Intern Med. 2020;180(10):1336-1344. doi:10.1001/jamainternmed.2020.3391

19. Whittaker C, Walker PGT, Alhaffar M, et al. Under-reporting of deaths limits our understanding of true burden of COVID-19. BMJ. 2021;375:n2239. doi:10.1136/bmj.n2239

20. Hooper JE, Padera RF, Dolhnikoff M, et al. A postmortem portrait of the coronavirus disease 2019 (COVID-19) pandemic: a large multi-institutional autopsy survey study. Arch Pathol Lab Med. 2021;145(5):529-535. doi:10.5858/arpa.2020-0786-SA

21. Moretti M, Malhotra A, Visonà SD, Finley SJ, Osculati AMM, Javan GT. The roles of medical examiners in the COVID-19 era: a comparison between the United States and Italy. Forensic Sci Med Pathol. 2021;17(2):262-270. doi:10.1007/s12024-021-00358-0

22. Kiang MV, Irizarry RA, Buckee CO, Balsari S. Every body counts: measuring mortality from the COVID-19 pandemic. Ann Intern Med. 2020;173(12):1004-1007. doi:10.7326/M20-3100

23. Opioid crisis statistics. HHS. Updated February 12, 2021. Accessed March 30, 2022. https://www.hhs.gov/opioids/about-the-epidemic/opioid-crisis-statistics/index.html

24. Slavova S, Delcher C, Buchanich JM, Bunn TL, Goldberger BA, Costich JF. Methodological complexities in quantifying rates of fatal opioid-related overdose. Curr Epidemiol Rep. 2019;6(2):263-274. doi:10.1007/s40471-019-00201-9

25. Slavova S, O’Brien DB, Creppage K, et al. Drug overdose deaths: let’s get specific. Public Health Rep. 2015;130(4):339-342. doi:10.1177/003335491513000411

26. Mertz KJ, Janssen JK, Williams KE. Underrepresentation of heroin involvement in unintentional drug overdose deaths in Allegheny County, PA. J Forensic Sci. 2014;59(6):1583-1585. doi:10.1111/1556-4029.12541

27. Boslett AJ, Denham A, Hill EL. Using contributing causes of death improves prediction of opioid involvement in unclassified drug overdoses in US death records. Addiction. 2020;115(7):1308-1317. doi:10.1111/add.14943

28. Smith Sehdev AE, Hutchins GM. Problems with proper completion and accuracy of the cause-of-death statement. Arch Intern Med. 2001;161(2):277-284. doi:10.1001/archinte.161.2.277

29. Adeyinka A, Bailey K. Death certification. In: StatPearls. StatPearls Publishing; 2022. Accessed June 15, 2022. https://www.ncbi.nlm.nih.gov/books/NBK526015/

30. Davis GG, Cadwallader AB, Fligner CL, et al. Position paper: recommendations for the investigation, diagnosis, and certification of deaths related to opioid and other drugs. Am J Forensic Med Pathol. 2020;41(3):152-159. doi:10.1097/PAF.0000000000000550

31. Rosenbaum JE, Stillo M, Graves N, Rivera R. Timeliness of provisional United States mortality data releases during the COVID-19 pandemic: delays associated with electronic death registration system and weekly mortality. J Public Health Policy. 2021;42(4):536-549. doi:10.1057/s41271-021-00309-7

32. Ventura SJ. The U.S. National Vital Statistics System: transitioning into the 21st century, 1990-2017. Vital Health Stat 1. 2018;(62):1-84.

33. Next generation electronic death registration system. Public Health Informatics Institute. March 16, 2016. Accessed March 30, 2022. https://phii.org/resources/next-generation-electronic-death-registration-system/

34. Nicholson N, Perego A. Interoperability of population-based patient registries. J Biomed Inform X. 2020;6:100074. doi:10.1016/j.yjbinx.2020.100074

35. Pennsylvania: coroner/medical examiner laws. CDC. January 15, 2015. Accessed March 30, 2022. https://www.cdc.gov/phlp/publications/coroner/pennsylvania.html

36. Upcoming coroners education events. Pennsylvania Office of Attorney General. Accessed March 30, 2022. https://www.attorneygeneral.gov/pennsylvania-state-coroners-education-board/coroner/

37. Messite J, Stellman SD. Accuracy of death certificate completion: the need for formalized physician training. JAMA. 1996;275(10):794-796. doi:10.1001/jama.1996.03530340058030

38. Villar J, Pérez-Méndez L. Evaluating an educational intervention to improve the accuracy of death certification among trainees from various specialties. BMC Health Serv Res. 2007;7:183. doi:10.1186/1472-6963-7-183

39. Degani AT, Patel RM, Smith BE, Grimsley E. The effect of student training on accuracy of completion of death certificates. Med Educ Online. 2009;14:17. doi:10.3885/meo.2009.Res00315

40. Lakkireddy D, Basarakodu KR, Vacek JL, et al. Improving death certificate completion: a trial of two training interventions. J Gen Intern Med. 2007;22(4):544-548. doi:10.1007/s11606-006-0071-6

41. Ruhm CJ. Geographic variation in opioid and heroin involved drug poisoning mortality rates. Am J Prev Med. 2017;53(6):745-753. doi:10.1016/j.amepre.2017.06.009

42. Arora S, Geppert CMA, Kalishman S, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Acad Med. 2007;82(2):154-160. doi:10.1097/ACM.0b013e31802d8f68

43. Faherty LJ, Rose AJ, Chappel A, Taplin C, Martineau M, Fischer SH. Assessing and expanding the evidence base for Project ECHO and ECHO-like models: findings of a technical expert panel. J Gen Intern Med. 2020;35(3):899-902. doi:10.1007/s11606-019-05599-y