Currently Viewing:
The American Journal of Managed Care February 2013
Are Benefits From Diabetes Self-Management Education Sustained?
JoAnn Sperl-Hillen, MD; Sarah Beaton, PhD; Omar Fernandes, MPH; Ann Von Worley, RN, BSHS, CCRP; Gabriela Vazquez-Benitez, PhD, MSc; Ann Hanson, BS; Jodi Lavin-Tompkins, RN, CNP, CDE, BC-ADM; William Parsons, MS; Kenneth Adams, PhD; and C. Victor Spain, DVM, PhD
Impact of Oral Nutritional Supplementation on Hospital Outcomes
Tomas J. Philipson, PhD; Julia Thornton Snider, PhD; Darius N. Lakdawalla, PhD; Benoit Stryckman, MA; and Dana P. Goldman, PhD
Comparative Effectiveness Research and Formulary Placement: The Case of Diabetes
Michael E. Chernew, PhD; Rick McKellar, BS; Wade Aubry, MD; Roy Beck, MD, PhD; Joshua Benner, PharmD, ScD; Jan E. Berger, MD, MJ; A. Mark Fendrick, MD; Felicia Forma, BSc; Dana Goldman, PhD; Anne Peters, MD; Rebecca Killion, MA; Darius Lakdawalla, PhD; Douglas K. Owens, MD; and Joe Stahl, MA
Oral Nutritional Supplementation
Gordon L. Jensen, MD, PhD
Medical Homes Require More Than an EMR and Aligned Incentives
Samantha L. Solimeo, PhD, MPH; Michael Hein, MD, MS; Monica Paez, BA; Sarah Ono, PhD; Michelle Lampman, MA; and Greg L. Stewart, PhD
Currently Reading
Do Electronic Medical Records Improve Diabetes Quality in Physician Practices?
Jeffrey S. McCullough, PhD; Jon Christianson, PhD; and Borwornsom Leerapan, MD, PhD
The Cost of Implementing Inpatient Bar Code Medication Administration
Julie Ann Sakowski, PhD; and Alana Ketchel, MPP, MPH
Spending and Mortality in US Acute Care Hospitals
John A. Romley, PhD; Anupam B. Jena, MD, PhD; June F. O'Leary, PhD; and Dana P. Goldman, PhD
Cost-Effectiveness of Medicare Drug Plans in Schizophrenia and Bipolar Disorder
Kenneth J. Smith, MD, MS; Seo Hyon Baik, PhD; Charles F. Reynolds III, MD; Bruce L. Rollman, MD, MPH; and Yuting Zhang, PhD
Introducing Forensic Health Services Research
Laurence F. McMahon Jr, MD, MPH; and Vineet Chopra, MD, MSc
Guidance for Structuring Team-Based Incentives in Healthcare
Daniel M. Blumenthal, MD, MBA; Zirui Song, PhD; Anupam B. Jena, MD, PhD; and Timothy G. Ferris, MD, MPH

Do Electronic Medical Records Improve Diabetes Quality in Physician Practices?

Jeffrey S. McCullough, PhD; Jon Christianson, PhD; and Borwornsom Leerapan, MD, PhD
Policy makers should not expect public sector electronic medical record investments to yield substantial short-term improvements in publicly reported measures.
Objectives: To measure the effect of electronic medical records (EMRs) on a publicly reported composite measure indicating optimal diabetes care (ODC) rates in ambulatory settings.

Study Design: Data from Minnesota Community Measurement on 557 clinics were used, including information on ODC, EMR adoption, and clinic characteristics.

Methods: A difference-in-differences strategy was used to estimate the impact of EMR adoption on patient outcomes while controlling for observed and unobserved clinic characteristics. Results were compared with a cross-sectional analysis of the same data.

Results: EMRs had no observable effect on ODC for the average clinic during the first 2 years postadoption. EMRs may, however, generate modest (+4 percentage point) ODC increases for clinics in large, multisite practices. Cross-sectional analysis likely overestimates the effect of EMRs on quality.

Conclusions: There is little evidence that EMR adoption improves diabetes care during the first 2 years postadoption. This is notable as diabetes is a condition for which information technology has the potential to improve care management. The results suggest that policy makers should not expect public sector EMR investments to yield significant short-term improvements in publicly reported measures.

(Am J Manag Care. 2013;19(2):144-149)
Data from Minnesota Community Measurement on 557 clinics were used to estimate the effect of electronic medical record (EMR) adoption on measures of diabetes care quality in the initial postadoption years.

  •  Although EMRs may generate modest improvements in diabetes quality indicators for clinics in relatively large, multisite, group practices, there is little evidence they improve short-term diabetes care.

  • Estimates of the impact of EMRs on quality using cross-sectional data should be viewed cautiously as they may overestimate the positive impact of EMRs.
In 2009, Congress passed the Health Information Technology for Economic and Clinical Health (HITECH) Act, authorizing an estimated $30 billion in payments to healthcare organizations that purchase and implement electronic medical records (EMRs).1 This unprecedented action was based on the observation that the healthcare sector lagged in the adoption of information technology (IT), along with the expectation that EMRs would improve quality while reducing costs. With respect to quality improvement, it was thought that EMRs would improve care coordination, promote treatment guideline adherence, and simplify tracking of treatments and outcomes, reducing patients’ exposure to risk and unnecessary care.

Recent literature reviews generally have found evidence to support the beneficial effects of EMRs. However, the majority of studies have been carried out in inpatient settings, and studies often focused on a small set of technical functionalities.2 Evidence regarding the impact of EMRs implemented in physician practices is less extensive. There have been reports of strong favorable impacts of EMRs on quality,3,4 but some studies and experts argue that benefits are small and disputable.5-7 In particular, a recent large-scale analysis of physician survey data8 reported a positive relationship between EMRs and only 1 of 20 quality indicators. These authors state that their findings “raise concerns about the ability of health information technology to fundamentally alter outpatient care quality.” A recent study using medical-records data from a single community found that EMR use was correlated with large diabetes outcome improvements.9 However, the authors of this study also recognize that their cross-sectional empirical strategy may be subject to selection bias.

We add to the relatively limited number of quantitative analyses that address the relationship between EMRs and ambulatory care quality. Because past studies typically have used cross-sectional data and/ or have not controlled for unobserved differences between clinics that adopted EMRs and those that have not (selection effects), their ability to draw inferences about the relationship between EMR adoption and publicly reported quality measures has been limited. In this article, we examine whether changes in physician practice quality measures are linked to EMR adoption using data from public reports of diabetes care. We contrast these results with findings from cross-sectional analyses of the same data. To help ensure the robustness of our approach, we also explore whether EMR adoption leads to changes in the measurement of quality metrics by examining missing data rates.


Data Sources

We used data on the quality of diabetes care provided by physician clinics in Minnesota from 2008 to 2010. These data were publicly reported by Minnesota Community Measurement (MNCM), a collaboration among a wide range of community stakeholders.10

Diabetes care is an appropriate focus for the analysis as there are widely accepted treatment guidelines that can be incorporated in EMRs. Treatment requires coordination of tests, prescriptions, and patient behavior, as well as management across time. EMRs should facilitate each of these tasks.

Data are reported annually by clinics on a voluntary basis through a process called MNCM Direct Data Submission. Required data elements are assembled from medical records by clinic abstractors. After completing quality checks and addressing Health Insurance Portability and Accountability Act requirements, clinics submit data to MNCM, which conducts quality checks and performs on-site audits to ensure data quality.11

Optimal diabetes care (ODC) scores were calculated for each submitting clinic. These scores measure the percentage of patients with diabetes (type I and type II) aged 18 to 75 years who reach 5 treatment goals: (1) glycated hemoglobin (A1C) less than 8%; (2) blood pressure less than 130/80 mm Hg; 3) low-density lipoprotein cholesterol less than 100 mg/ dL; (4) daily aspirin use unless contraindicated (ages 41-75 years only); and (5) documented tobacco-free status. MNCM altered the goal for A1C control in 2010, using <8% to replace the prior standard of <7%.11

Study Population

The study population was 557 clinics in Minnesota and neighboring states. These clinics included both stand-alone facilities and those that were members of multiclinic group practices. The number of clinics rose from 309 to 527 during the study period, while the number of groups grew from 58 to 123. Note that because a small number of clinics discontinued reporting, the total number of clinics exceeded the maximum number of clinics per year. Consequently, our data form an unbalanced panel with the number of clinics growing overtime. Average sample characteristics from 2008 to 2010 are shown in Table 1.

Study Variables

The MNCM data documented clinic-level performance for ODC and its 5 component measures (Table 1), as well as the rates at which component measures were missing. These missing data rates reflect the clinic’s ability to document and report clinical information.

MNCM tracked whether each clinic’s data were drawn from a paper-based system (no EMR), a hybrid of paper and electronic documentation (partial EMR), or an entirely electronic system (EMR). An important limitation of these data is that they did not allow us to capture the systems’ functional capabilities, such as decision-support systems; however, they did allow us to measure an average effect of EMR adoption. EMR utilization rates were quite high in the study clinics; 54% of clinics used EMR systems, an additional 24% used partial EMR systems, and only 22% used no EMR. These rates are substantially higher than the national average.12,13 We observed 124 adoption events in our sample.

The data set contained additional information regarding the clinics and their patient populations. On average, clinics treated 427 diabetic patients and submitted a sample of 273 records. Most clinics were members of larger group practices composed of, on average, 15 clinics (Table 1). Clinic characteristics were notably different across EMR adoption levels. For instance, clinics with EMRs had a 25% larger diabetic patient population than clinics with no EMRs.

As a first step in our analysis, we used linear regression to measure average quality differences between clinics with no EMRs, partial EMRs, and EMRs (Table 2, model 1). To account for potential selection effects, we used a differencein- differences strategy. We implemented this strategy using linear regression with clinic and time-fixed effects, using xtreq commands in Stata version 11 (StataCorp, College Station, Texas). We also included observable, time-varying, clinic controls such as the number of diabetes patients, the number of clinics within the group, and whether data were drawn from a sample or a census (model 3). Standard errors were corrected to reflect the fact we used multiple observations for each clinic.14


During 2010, the average clinic with no EMR achieved ODC for 17% of patients, and clinics with partial EMR utilization achieved ODC for 18%. In contrast, clinics with EMR achieved ODC for 26% of patients.

Although these results were statistically significant, there were differences in the characteristics of clinics with and without EMRs. Clinics that had adopted EMRs prior to 2008 were members of larger group practices and treated more diabetes patients per clinic than clinics that eventually adopted EMRs. Similarly, clinics that eventually adopted EMRs were larger than those that never adopted EMRs. When we incorporated these clinic characteristics into the analyses (model 2), partial EMR use became uncorrelated with ODC. Also, the effect of EMR utilization was substantially dampened, but remained statistically significant. After incorporation of clinic characteristics, EMR utilization was associated with only a 3 percentage point difference in ODC compared with clinics with no EMR.

This finding does not necessarily mean that having EMRs is the determining factor in achieving higher quality. Higher-quality clinics might, for example, be more likely to adopt EMRs. Conversely, lower-performing clinics might adopt EMRs to address quality problems. We examined the differences in individual clinic quality before and after EMR adoption relative to quality differences in clinics that did not change their EMR systems over the same period. If unobserved aspects of clinic quality (eg, clinic culture) were relatively stanble across time, then taking differences eliminates this bias. Similarly, analyzing differences across adopting and nonadopting clinics eliminated bias from changes across time in factors that were common to clinics (eg, changes in reporting rules).

Using this approach, we found no statistically significant relationship between EMR use and ODC rates. The lack of a significant relationship between EMRs and rates of ODC was not likely due to imprecise measurement. The parameter was nearly zero (-0.004) and the estimate was fairly precise; we rejected a 2.4 percentage point improvement with 95% certainty. These findings suggest that, although EMRs and ODC are correlated, having an EMR was not a significant factor leading to higher ODC during the time period covered in our data set. Instead, the findings imply that clinics adopting EMRs performed better on MNCM metrics before EMR adoption. There are a number of potential explanations for our finding that EMR adoption did not improve ODC rates. First, ODC rates combine performance on 5 different measures, and the effect of EMRs could vary across measures. To address this possibility, we estimated separate models for each component measure. We found that EMR adoption did not improve any individual measure.

Second, EMR systems may change the way medical care is documented independent of their effect on actual care processes. Documentation changes could obscure the effect of EMRs on quality. We found no relationship between EMR adoption and missing data rates. (Results from the component quality and missing data analyses are available on request from the authors.)

Third, the benefits of EMR adoption may not be realized immediately. Although we only observed 3 years of data, we tested for a 1-year lagged effect of EMRs on ODC controlling for clinic characteristics and allowing for unobserved clinic and time effects (model 4, Table 3). We found no relationship between lagged EMR adoption and ODC. Naturally, a longer time series is needed to better test alternative lag structures.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up