Nimah Haq, MPH

 To describe the scale and scope of inpatient electronic health record (EHR) maintenance following initial implementation.

 A retrospective study reviewing EHR change documentation within an integrated healthcare delivery system that has 21 hospitals.

 Between 2010 and 2015, we identified and categorized all significant changes made to the inpatient EHR, as documented within monthly EHR communication updates. We categorized EHR changes as updates to existing functionality or upgrades to new functionality. We grouped changes within larger functional domains as orders, alerts and customization, surgical and emergency department (ED), data review, reports and health information management, and other. We also identified the clinical areas and user roles targeted by these changes.

 Over a 6-year period, 5551 unique changes were made to the inpatient EHR, with a median of 72 changes per month. Changes most frequently targeted orders (44.7% of 2190 change documents) and order sets (29.9% of documents). In total, changes affected 135 EHR functions. A total of 151 unique user roles were affected by these changes, with the most frequent roles including nurses (30.6%), physicians (26.6%), and other clinical staff (22.7%). The clinical areas most targeted by changes included surgical areas and the ED.

 Over 6 years, EHR maintenance for clinical functionality was substantial and varied with pervasive impacts, requiring persistent attention, diverse expertise, and interdisciplinary collaboration.

We evaluated clinically oriented changes needed to maintain an inpatient electronic health record (EHR) in a single health system. Over 6 years, 5551 unique changes were made that together had an impact on more than 130 EHR tools and 150 user roles.

Although much attention has focused on initial EHR implementation, ongoing maintenance needs are substantial, diverse, and pervasive.

The most frequently updated EHR elements targeted order sets, surgical and emergency department areas, and users (nurses, physicians, and pharmacists).

Given our focus only on inpatient EHR clinical changes, our findings are likely to be a significant underestimate of ongoing EHR resource needs.

Electronic health record (EHR) implementation within US acute care hospitals increased rapidly between 2010 and 2015 as a result of the Health Information Technology for Economic and Clinical Health Act of 2009.

 Among hospitals with and without an EHR system in 2009, some of the major barriers to EHR adoption were perceived to include the cost of purchase, uncertain return on investment, clinician resistance, and inadequate information technology staff.

 Many of these barriers have been persistently cited as concerns during the subsequent period of rapid EHR uptake.

Ongoing maintenance has also been cited as a key barrier to EHR implementation and use. However, few studies have described the subsequent improvements needed to maintain and optimize EHR functionality over time.

 Given the substantial costs resulting from initial implementation, far less attention has been devoted to the maintenance needs that follow for optimal EHR performance. In this study, our goal was to describe the scale and scope of changes made to an inpatient EHR system after initial implementation within the 21 hospitals of Kaiser Permanente Northern California (KPNC).

This study was deemed exempt by the Kaiser Permanente Institutional Review Board. KPNC is an integrated healthcare delivery system serving 4.1 million members at 21 hospitals. Implementation of an inpatient EHR system (Epic; Verona, Wisconsin), known internally as KP HealthConnect (KPHC), was completed in 2010; outpatient KPHC completed implementation in 2008.

We evaluated significant changes made to the inpatient EHR system based on documentation within monthly “KPHC communication” reports between 2010 and 2015. These reports were used to inform end users about upcoming KPHC updates (ie, changes to existing functionality) and/or upgrades (ie, new functionality). Multiple unique changes affecting the same EHR functionality were included within a single change document, which described the overall changes to that functionality.

The changes arose from diverse sources, including clinician requests; feedback from clinical technology leads; safety, risk, or reporting needs; and/or forthcoming system upgrades. Although several governance committees exist to oversee EHR changes, the modifications described here were shepherded through the change management process by a core regional team of physician and clinician informaticists with significant experience in EHR clinical functionality; working alongside this core team was informatics staff experienced in building EHR tools. The changes documented within these reports represented only a subset of all requested changes, as not all requests were ultimately fulfilled. Thus, the reports included only those changes that were implemented as prioritized by clinical value, safety, regulations, resource availability, and end-user satisfaction. Documented changes also varied in scope and scale, with some being implemented with relative ease and others requiring months of preparation.

After identifying all changes grouped within the documents, we further categorized them within broad functional domains based on their type, including orders (single, grouped, and templated orders), alerts and customization (user-customized tools and decision support), surgical and emergency department (ED) (perioperative- and ED-specific tools), data review (clinician-facing tools for review of patient charts, results, and imaging), reports and health information management (HIM) (unit- and group-based reporting tools), and other (patient tools and other change types). For each change, we also identified and grouped the type of user affected, as well as the clinical or functional area affected, based on report documentation.

Data are reported as median (interquartile range [IQR]) and number (percent). Analyses were conducted using STATA 14.1/SE (StataCorp; College Station, Texas).

Between 2010 and 2015, 5551 unique changes were made to the inpatient EHR (

), with a median of 72 (IQR, 35-112) changes per month. Most unique changes (n = 3191 [57.5%]) were updates to existing functionality, with 95.7% affecting all 21 hospitals. Individual changes were aggregated within 2190 update communication documents.

Upgrades related to EHR orders contributed to the largest proportion of all significant changes (44.7% of documents) (

). In total, changes to templated order sets comprised 29.9% of all documents. Other EHR functional domains that accounted for a significant proportion of all changes included clinical data review (15.7%), surgical and ED-specific tools (13.5%), alerts and customization (11.4%), grouped reports and HIM (8.3%), and patient tools and other (6.3%). Overall, changes affected 135 EHR functions.

In total, 151 specific types of users were affected by changes (eg, bed controller, cardiologist, certified nurse midwife, support site specialist), with an impact on all KPHC users 10.2% of the time. Targeted changes most frequently affected nurses (30.6%), physicians (26.6%), and other clinical staff (22.7%), such as pharmacists, therapists, and dietitians. The specific clinical areas most commonly affected by changes included surgical specialties (7.9%), ED (7.1%), mother—baby (6.9%), and pharmacy (6.4%) (

Over a 6-year period, the changes required to maintain and improve an inpatient EHR system were substantial and diverse, with a pervasive impact. On average, 2.5 significant EHR changes occurred each day, together affecting more than 130 specific tools and 150 unique user roles across the 21 hospitals. Key areas that were frequently targeted by updates included specific EHR tools (order sets), clinical domains (surgical and ED), and end users (nurses, physicians, and pharmacists).

Widespread implementation of inpatient EHR systems has occurred rapidly over the past decade; however, few studies have comprehensively detailed the maintenance required to optimize their use.

 Most studies evaluating EHR implementation have focused on quantifying the costs and barriers related to initial implementation, the outcomes associated with EHR uptake, and the impact of EHR use on clinicians and patients.

 Nevertheless, emerging evidence suggests that the benefits of EHR use accrue gradually over time and are likely attributable to the ongoing addition of new functionality attained via continual updates and upgrades.

 Thus, although much attention is focused on the initial “go live” of the system, the true benefits of EHR adoption may only emerge with persistent attention to enhancing the EHR-based workflows and tools that drive improvements in care. In particular, the customization and usability of EHR functions to meet end-user needs have been identified as key measures that portend likely EHR benefit and can also mitigate potential harm arising from usability or workflow challenges.

The primary limitation of this study is that our findings were based on a single healthcare system and a specific EHR product, which may limit the generalizability of our findings. Our study focused on the clinical aspects of inpatient EHR maintenance recorded within monthly change communication reports. However, our findings almost certainly represent a significant underestimate of the true scale and scope of ongoing EHR changes across our system. Numerous daily changes are made to EHR functions that do not rise to the significance level that would trigger their inclusion within communication reports. Our findings also do not account for simultaneous outpatient EHR and information technology infrastructure support, which contributes heavily to ongoing maintenance needs. Finally, the resources needed to implement each change could vary significantly, in terms of both time and cost.

EHR maintenance needs were prevalent and diverse, affecting 150 unique user roles and contributing to an average of more than 2.5 significant changes per day. Our findings highlight the need for significant resources, expertise, and collaboration to maximize EHR clinical utility and benefit. They also demonstrate that an EHR system represents a dynamic network of evolving tools that requires ongoing investment well after initial implementation.

 Kaiser Permanente Division of Research (VXL), Oakland, CA; Master of Public Health Program, University of Southern California (NH), Los Angeles, CA; The Permanente Medical Group (VXL, ICC, BH), Oakland, CA.

 The Permanente Medical Group, National Institutes of Health/National Institute of General Medical Sciences 

K23 GM112018, and NIH/NIGMS R35 GM128672.

 Dr Liu is employed with The Permanente Medical Group and received the NIH/NIGMS K23 GM112018 grant. Dr Chan and Dr Hoberman are employed with The Permanente Medical Group. Ms Haq reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (VXL, NH, ICC, BH); acquisition of data (VXL, ICC, BH); analysis and interpretation of data (VXL, NH, ICC, BH); drafting of the manuscript (VXL); critical revision of the manuscript for important intellectual content (VXL, NH, ICC, BH); statistical analysis (VXL, NH); provision of patients or study materials (ICC); obtaining funding (VXL); administrative, technical, or logistic support (VXL, ICC, BH); and supervision (VXL, ICC, BH).

 Vincent X. Liu, MD, MS, Kaiser Permanente Division of Research, 2000 Broadway, Oakland, CA 95070. Email: Vincent.x.liu@kp.org.

1. Non-federal acute care hospital electronic health record adoption. Office of the National Coordinator for Health Information Technology website. dashboard.healthit.gov/quickstats/pages/FIG-Hospital-EHR-Adoption.php. Updated September 2017. Accessed July 1, 2018.

. 2010;362(5):382-385. doi: 10.1056/NEJMp0912825.

3. Adler-Milstein J, Jha AK. HITECH Act drove large gains in hospital electronic health record adoption. 

. 2017;36(8):1416-1422. doi: 10.1377/hlthaff.2016.1651.

4. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. 

. 2009;360(16):1628-1638. doi: 10.1056/NEJMsa0900592.

5. Dorr DA, Cohen DJ, Adler-Milstein J. Data-driven diffusion of innovations: successes and challenges in 3 large-scale innovative delivery models. 

. 2018;37(2):257-265. doi: 10.1377/hlthaff.2017.1133.

6. Fleming NS, Culler SD, McCorkle R, Becker ER, Ballard DJ. The financial and nonfinancial costs of implementing electronic health records in primary care practices. 

. 2011;30(3):481-489. doi: 10.1377/hlthaff.2010.0768.

7. Wang SJ, Middleton B, Prosser LA, et al. A cost-benefit analysis of electronic medical records in primary care. 

. 2003;114(5):397-403. doi: 10.1016/S0002-9343(03)00057-3.

8. Evans DC, Nichol WP, Perlin JB. Effect of the implementation of an enterprise-wide electronic health record on productivity in the Veterans Health Administration. 

. 2006;1(2):163-169. doi: 10.1017/S1744133105001210.

9. Gagnon MP, Nsangou ÉR, Payne-Gagnon J, Grenier S, Sicotte C. Barriers and facilitators to implementing electronic prescription: a systematic review of user groups’ perceptions. 

. 2014;21(3):535-541. doi: 10.1136/amiajnl-2013-002203.

10. Kruse CS, Kothman K, Anerobi K, Abanaka L. Adoption factors of the electronic health record: a systematic review. 

. 2016;4(2):e19. doi: 10.2196/medinform.5525.

11. Kruse CS, Kristof C, Jones B, Mitchell E, Martinez A. Barriers to electronic health record adoption: a systematic literature review. 

. 2016;40(12):252. doi: 10.1007/s10916-016-0628-9.

12. McGinn CA, Grenier S, Duplantie J, et al. Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. 

. 2011;9:46. doi: 10.1186/1741-7015-9-46.

13. Zandieh SO, Yoon-Flannery K, Kuperman GJ, Langsam DJ, Hyman D, Kaushal R. Challenges to EHR implementation in electronic- versus paper-based office practices. 

. 2008;23(6):755-761. doi: 10.1007/s11606-008-0573-5.

14. Burton LC, Anderson GF, Kues IW. Using electronic health records to help coordinate care. 

. 2004;82(3):457-481, table of contents. doi: 10.1111/j.0887-378X.2004.00318.x.

15. Babbott S, Manwell LB, Brown R, et al. Electronic medical records and physician stress in primary care: results from the MEMO Study. 

. 2014;21(e1):e100-e106. doi: 10.1136/amiajnl-2013-001875.

16. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. 

. 2010;10:231. doi: 10.1186/1472-6963-10-231.

17. Howley MJ, Chou EY, Hansen N, Dalrymple PW. The long-term financial impact of electronic health record implementation. 

. 2015;22(2):443-452. doi: 10.1136/amiajnl-2014-002686.

18. Adler-Milstein J, DesRoches CM, Kralovec P, et al. Electronic health record adoption in US hospitals: progress continues, but challenges persist. 

. 2015;34(12):2174-2180. doi: 10.1377/hlthaff.2015.0992.

Connected for Health: Using Electronic Health Records to Transform Care Delivery

Articles

In the 6 years following inpatient electronic health record (EHR) implementation, an average of 2.5 significant EHR changes per day were made for maintenance and improvement.

Nimah Haq, MPH

Articles

Inpatient Electronic Health Record Maintenance From 2010 to 2015