Elisha M. Friesema, BA, CCRP

To evaluate the effects of a multifaceted quality improvement intervention during 2 time periods on 4 coronary artery disease [CAD] measures in 4 primary care practices. During the first phase, electronic reminders prompted physicians to order indicated medications or record contraindications and refusals (exceptions). In the second phase, physicians also received reports about their performance (including lists of patients not satisfying these measures), and financial incentives were announced.

Adult CAD patients seen within the preceding 18 months were included. The primary outcome was the performance on each measure (proportion of eligible patients satisfying each measure after removing those with exceptions). Secondary outcomes were the proportion with the medication on their medication list, and the proportion with exceptions.

Median performance at baseline was 78.8% for antiplatelet treatment, 85.1% for statin treatment, 77.0% for beta-blocker after myocardial infarction (MI), and 67.1% for angiotensinconverting enzyme inhibitor or angiotensin receptor blocker after MI. Performance improved slightly for 3 measures during the first phase and improved more substantially for all 4 measures during the second phase. For 3 of 4 measures, however, documentation of exceptions increased but not medication prescribing. Most exceptions were judged to be appropriate by peer review.

Physicians responded more to the combination of feedback and financial incentives than they had to electronic reminders alone. High performance was only achieved for 1 of 4 measures and recording of exceptions rather than increases in medication prescribing accounted for most of the observed improvements.

We examined the sequential implementation of commonly used quality improvement techniques to improve outpatient coronary heart disease care in 4 practices.

Physician response to electronic reminders alone to address outpatient coronary heart disease measures was small.

Physicians responded more when the combination of feedback reports and financial incentives was added to reminders.

For 3 of 4 measures, documentation of exceptions increased, but not medication prescribing.

Most physician-recorded exceptions to quality measures were judged to be appropriate by peer review.

Quality improvement techniques that leverage an electronic health record (EHR) have been shown to improve care in many cases.

However, EHR-based quality improvement has not been universally successful, and even in many instances where study results were positive, the magnitude of the improvement was small.

Furthermore, observational data do not suggest that simply having an EHR improves quality in outpatient settings.

 In contrast, we have shown in the UPQUAL study (Utilizing Precision Performance Measurement for Focused Quality Improvement) that interconnected EHR-based tools can improve quality for multiple process of care measures in a large urban, single-site, university-affiliated practice.

 This intervention was designed to improve quality measurement (including capture of contraindications and patient refusals), make point-of-care reminders more accurate, and provide more valid and responsive feedback to clinicians (including lists of patients not receiving essential medications).

In this current study, we applied these principles—improve quality measurement in order to enable more accurate point-of-care reminders and feedback&mdash;to coronary artery disease (CAD) care in 4 suburban primary care group practices (2 family medicine and 2 internal medicine) that belong to the same health system and use the same EHR. We selected CAD care as our study objective because it is a common and important chronic disease and because implementing CAD measures in this setting was more feasible than several other candidate chronic disease and prevention topics. In this health system, point-of-care reminders were implemented first in July 2008 (Phase 1). Starting September to November 2009 (Phase 2), feedback was given to physicians and the medical group publicized to physicians that financial incentives would be tied to performance measures (including the 4 measures studied here). Both the reminders and physician feedback portions of the interventions were planned prior to Phase 1 by the study team. The financial incentives were initiated independently by leadership in the organization that was not directly associated with this study. This sequential implementation provides an opportunity to observe the additional effects of adding the combination of feedback and announcing financial incentives to electronic reminders on measured performance.

We performed this study at 4 primary care practices in the northern suburbs of Chicago, Illinois, that use the same commercial EHR (EpicCare, Epic Systems Corporation, Verona, Wisconsin). Northwestern University’s and Northshore University HealthSystem’s institutional review boards approved the study. All patients eligible for 1 or more quality measures cared for by 33 attending physicians (10 family medicine and 23 internal medicine) and 15 family medicine resident physicians were included. The practices had used the EHR for 5 years before the start of the first intervention examined in the study.

Sequential Implementation of Quality Improvement Techniques

We selected for consideration 4 measures of CAD care quality that were based on national measures: antiplatelet drug and lipid-lowering drug treatment in all patients with CAD, beta-blocker use in patients with prior myocardial infarction (MI), and angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) for diabetes or left ventricular systolic dysfunction.

 Health system clinicians, including cardiologists, discussed and modified the measures for local use, changing the lipid-lowering drug measure to statin treatment in CAD and changing the ACE inhibitor or ARB measure to apply to patients with prior MI only.

Electronic Clinical Decision Support (Phase 1)

Prior to these interventions, there were no other clinical decision support tools in use that addressed these topics. We added electronic point-of-care reminders that appeared during patient encounters when an apparently eligible patient did not have an indicated medication on their current medication list and had no exception recorded. These alerts were minimally intrusive (they did not interrupt clinicians’ work flow and the alert was indicated only by a single yellow highlighted tab that appeared on the left side of the screen when any clinical reminder criteria were present, and physicians had to select this tab to see the individual reminders). These alerts were displayed using existing EpicCare functionality. These electronic reminders included standardized ways to capture patient reasons (eg, refusals) or medical reasons that were exceptions for individual reminders within the reminder system of the EHR. These reminders were implemented in July 2008. We sent physicians educational e-mails with brief training materials to introduce the new alerts and to show how to record patient or medical exceptions.

Implementation of Feedback Reports and Announcement of Incentives (Phase 2)

Starting in September 2009, on a monthly basis, we gave physicians printed reports indicating their overall performance on each of the 4 measures for all their eligible patients and lists of individual patients who appeared to be eligible for an indicated medication but were not receiving it and had no exception recorded.

In October and November of 2009, the medical group leadership announced to physicians that a small portion of their compensation (1.5% of total compensation, which constituted 25% of the incentive-based compensation) would be tied to their performance on quality metrics, including the 4 metrics covered in this study.

We retrospectively calculated the performance for the 4 CAD measures for each month from September 2007 through March 2011. At each time point, patients were eligible for a measure if they had an office visit with a physician from 1 of the 4 included practices during the preceding 18 months and had a qualifying

 International Classification of Diseases, Ninth Revision, Clinical Modification 

code used on their active problem list, past medical history, or as an encounter diagnosis. We used Structured Query Language to retrieve data from an enterprise data warehouse that contains data copied daily from the EHR. For each time point, all patients were classified for each measure for which they were eligible as: a) satisfied, b) did not satisfy but had an exception, or c) did not satisfy and had no documented exception. The primary outcome for each measure was calculated as the number of patients who satisfied the measure divided by the total number of eligible patients excluding those with an exception. As an equation, the primary outcome = number satisfied / [number eligible — number not satisfied with an exception]. We also analyzed separately for each measure the proportion of eligible patients who satisfied the measure (were given the medication) and the proportion of all eligible patients who did not satisfy the measure and had exceptions.

We performed peer review beginning in September 2009 of medical exceptions recorded in the EHR and continued the review process for exceptions entered within the first 10 months of the intervention. One physician reviewed medical records to collect the reason for the exception and additional clinical information needed to judge the validity of the exception. When the clinical reasoning was unclear, the peer reviewer would request clarification from the treating clinician. Two board-certified internists and 1 board-certified family medicine physician met regularly to review the exceptions and judged them as appropriate, inappropriate (including when no contraindication to the medication was evident on physician chart review or cases where clarifying information was requested from the primary care physician but was not provided), or of uncertain appropriateness by consensus. When a consensus was not reached or the appropriateness was uncertain, 1 physician reviewed the medical literature, requested advice from specialists when needed, and the group discussed the case again until consensus was reached. Practice physicians received e-mail or telephone feedback for medical exceptions that were judged to be inappropriate.

We used interrupted time series analysis to examine changes in the primary and secondary outcomes over time for 2 different interventions: before and after July 2008 (the transition between baseline and Phase 1), and before and after September 2009 (the transition between Phase 1 and Phase 2). We calculated the primary and secondary outcomes for each of the performance measures for each month from September 2007 through August 2010. A linear model was fit to each series including a continuous time variable, a dichotomous indicator of the intervention, and the interaction term of time and intervention as covariates. The individual data points used for each time period are depicted in the figures. Next, we determined the autoregressive order of the model residuals by minimizing Akaike’s information criterion.

 Finally, we fit a linear regression model with autoregressive errors (using the appropriate number of autoregressive parameters, if any were necessary) to each series. These fitted models were used to test statistical significance.

 To ensure model validity, we examined several residual diagnostics, the Jarque-Bera and the Shapiro-Wilk tests for normality of residuals, and normal Q-Q and autocorrelation plots.

 Analyses used SAS version 9.2 (SAS Institute Inc, Cary, North Carolina) and R software package version 2.13.1 (R Foundation for Statistical Computing, Vienna, Austria).

The number of patients eligible for the coronary disease measures and their characteristics are provided in 

. The number of eligible patients increased over time from 779 CAD and 218 MI patients in October 2007 to 1099 CAD and 332 MI patients by October 2010. Their characteristics changed little during the 3 years we examined (Table 1).

Median performance during the baseline period was 78.9% for antiplatelet treatment, 85.3% for statin treatment, 77.0% for beta-blocker after MI, and 67.2% for ACE inhibitor or ARB after MI (

). Performance on the antiplatelet measure was increasing significantly during the baseline period. Performance on the other 3 measures did not change significantly during the baseline period (

During Phase 1, overall performance continued to increase for the antiplatelet measure at a rate that was similar to the rate of improvement observed during the baseline period. There were statistically significant, but very small, increases in overall performance for the statin and ACE inhibitor/ARB measures (Table 2 and Figure 1). However, there were no significant increases in the rates of patients given medication for these 2 measures, and there was a decrease in patients given beta-blockers during Phase 1 compared with baseline (Table 2 and Figure 2). Physicians recorded exceptions to these measures during Phase 1 for small percentages of eligible patients (

There were significant increases in measured performance for all 4 measures during Phase 2 compared with Phase 1 (Table 2 and Figure 1). For 3 of the 4 measures, there was an increase in the documentation of exceptions (Table 2 and Figure 3) but not medication prescribing (Figure 2). For the antiplatelet measure there was a significant increase in both medication prescribing and exception documentation (Table 2, Figures 2 and 3).

By March 2011, overall performance was 99.7%, 91.1%, 86.8%, and 78.9% for antiplatelet treatment, statin treatment, beta-blocker after MI, and ACE inhibitor or ARB after MI, respectively.

We performed peer review for 179 medical exceptions. For 13 of these, a physician recorded an exception even though the patient was given the medication (the physician both recorded an exception and ordered the medication). Exceptions of this kind would not be included in the performance calculation, since the numerator criteria were met. Of the remaining 166 exceptions, 145 (87.3%) were judged appropriate on peer review,19 (11.4%) had an inappropriate reason or had no reason found for not prescribing on peer review, and 2 (1.2%) were of uncertain appropriateness (

We used time series analysis to examine the effects of implementing electronic clinician reminders directed at 4 measures of outpatient coronary artery disease care followed later by physician audit and feedback, lists of individual patients not meeting the measures, and the announcement of financial incentives. We made several observations that warrant discussion.

Introducing electronic reminders alone during Phase 1 had very little impact on measured quality. Physicians did interact with the clinical decision support (CDS) system to some degree, as evidenced by recording of exceptions using the CDS system. However, the extent of this usage was modest, and led to very small changes in measured quality. These findings are consistent with older studies of CDS aimed at changing outpatient provider prescribing behavior for coronary artery disease that found small effects, or no effect, from electronic clinician reminders.

After the start of individualized feedback to physicians with lists of patients not satisfying the measures and the announcement that there would be financial incentives based on performance (Phase 2), there were much more prominent changes in performance. During this second intervention period, there was a sharp increase in physicians’ use of exception recording for all 4 measures as well as an increase in patients with antiplatelet medication recorded on their medication lists. As a result of these changes, there were significant increases in measured performance for all 4 measures. This observed change suggests that linking CDS tools to local accountability systems like incentives or providing feedback on performance can lead to greater physician engagement with EHR quality improvement tools than would occur with CDS alone. Prior studies have shown a positive effect of providing audit and feedback to physicians,

 and inconsistent effects of financial incentives to improve quality.

Because performance feedback and the announcement of incentives occurred at nearly the same time in this healthcare system, we cannot distinguish the relative contributions of each to the observed changes. Future studies should examine whether adding financial incentives to performance feedback produces a greater effect on quality than feedback alone. This distinction is important since the long-term sustainability of financial incentives requires that the organization commit some amount of financial resources to prioritizing quality over other behaviors such as clinical volume.

Articles

We examined the impact of electronic reminders followed by performance reports and financial incentives. Physicians responded more to reports and incentives than to reminders alone.

Elisha M. Friesema, BA, CCRP

Articles

Implementation of EHR-Based Strategies to Improve Outpatient CAD Care