The Comparative Effectiveness of 2 Electronic Prescribing Systems | Page 2
Published Online: December 16, 2011
Rainu Kaushal, MD, MPH; Yolanda Barron, MS; and Erika L. Abramson, MD, MS
There was no significant difference between provider groups (Table 1). All physicians were board certified in internal or family medicine. The mean number of prescriptions written per provider and patients seen per provider were similar at baseline (0.65 and 0.26, respectively). At 1 year, integrated adopters saw significantly more patients and wrote significantly more prescriptions during the study period (0.002 and 0.002, respectively).
For stand-alone adopters, 1273 unique patients seen at baseline and 1598 unique patients seen at 1 year received prescriptions (Table 2). For integrated adopters, 481 unique patients seen at baseline and 368 unique patients seen at 1 year received prescriptions. At baseline, patients of stand-alone adopters were significantly older (56 vs 53 years, P <.001) and at 1 year more were female (60% vs 54%, P = .03).
Baseline Error Rates
We reviewed 2550 prescriptions at baseline, 1901 of which were written by stand-alone adopters and 649 of which were from integrated adopters (Tables 3A and 3B).
Rates of baseline prescribing errors for both groups were high, with planned adopters of integrated e-prescribing 1.6- fold less likely to have prescribing errors (P = .03). Rule violations were also high at baseline for both groups, but with no statistical differences. Near misses were detected infrequently and without statistical differences.
Rates of Errors for Stand-alone E-prescribing Adopters at Baseline and 1 Year
We reviewed 2305 prescriptions at 1 year for stand-alone adopters. Stand-alone e-prescribing reduced prescribing error rates by 6.7-fold (P <.001). Rule violations also decreased significantly (P <.001). Near misses remained low at 1 year and unchanged from baseline.
Rates of Errors for Integrated E-prescribing Adopters at Baseline and 1 Year
We reviewed 536 prescriptions at 1 year for integrated e-prescribing adopters. Use of an integrated system reduced prescribing error rates by 1.6-fold, suggesting a trend for decreasing prescribing errors but without achieving statistical significance. Rule violations significantly decreased (P <.001). Rates of near misses remained low and unchanged.
Rates of Errors for Stand-alone Versus Integrated E-prescribing Adopters at 1 Year
Stand-alone adopters were 2.45 times less likely to have prescribing errors than integrated adopters at 1 year (P <.001). After accounting for differences in baseline error rates, standalone users had a 4-fold lower rate of errors at 1 year (P <.001). There was no significant difference in rates of rule violations (P = .16) or near misses (P = .94) at 1 year.
Types of Prescribing Errors at Baseline and 1 Year
Use of the stand-alone system reduced all types of prescribing errors, while use of the integrated system led to a decrease in only some error types (Tables 4A and 4B).
In our small comparative effectiveness study, we found that use of a commercially available stand-alone e-prescribing system with more sophisticated CDS and more rigorous technical support led to a significantly greater reduction in prescribing errors compared with an e-prescribing system integrated within an EHR. To our knowledge, this study is the first to quantitatively compare the effectiveness of 2 different e-prescribing systems on medication safety in a single community using uniform methodology. The results are in contrast to our original hypothesis and highlight the need to approach HIT evaluation in a framework that goes beyond simply presence or absence of HIT and instead considers other factors, such as implementation processes and system functionalities. This study also highlights a novel application of CER to directly compare the effectiveness of HIT applications, an important method of healthcare delivery, in actual use.
Theoretically, an e-prescribing application within an EHR should perform better than a stand-alone application in reducing ambulatory prescribing errors. Safety benefits from e-prescribing are largely a function of incorporated CDS.13 A stand-alone application is more limited in the types and extent of available CDS. For example, a stand-alone application is unable to incorporate laboratory values and therefore perform drug-laboratory checks. However, this study demonstrates just the opposite.
These findings may be the result of the fact that the CDS was not fully configured, and was therefore relatively immature in the integrated application. Shortly after the study’s completion, with the system fully configured, more sophisticated CDS was introduced. Another explanation may be the different levels of training and support for the 2 groups. Although a for-profit HSP supported all providers, those using the integrated system were trained earlier and received less intensive support initially. Previous research has demonstrated the importance of training and support for successful adoption and use of EHRs and e-prescribing.14,15 For example, a study evaluating e-prescribing implementation found that challenges with implementation and insufficient technical support can lead physicians to abandon e-prescribing or delegate e-prescribing to support staff, reducing the potential of CDS to positively impact point of care prescribing decisions.14
A third explanation may be that this study was powered to measure prescribing errors rather than near misses or preventable ADEs. An integrated system may be the most effective at targeting these more serious errors due to the incorporation of more sophisticated CDS. There may also be other unmeasured safety benefits from the integrated system that our study design would not capture. For example, an alert based on a laboratory value that led to a lower drug dose or alternate drug would not be captured, although the possibility of patient harm would be reduced.
Nevertheless, this small study highlights the importance of CER research of EHRs and associated HIT applications. Our results also highlight the need to evaluate and compare HIT applications and their quality and safety effects in a framework that includes more than simply presence or absence of that technology,16 addressing factors such as implementation and technical support resources and the heterogeneity of the applications themselves. Large national investments are being made to promote widespread EHR adoption.17 Although EHRs are being certified and incentives to providers are dependent on the demonstration of “meaningful use,”2 little work is under way comparing which functionalities, implemented and supported in which ways, and used by whom, are most effective in practice.
Additionally, CDS is one of the most effective tools to deliver evidence-guided clinical recommendations to providers in actionable ways. Research suggests that providers have varying levels of satisfaction with CDS which may impact system use.18 Therefore, in order to assure that providers receive these recommendations in clinically useful ways, CER on CDS structure, content, and delivery will be critical.
Two of the most exciting and novel models of health systems delivery, the patient-centered medical home and accountable care organizations, include EHRs.19,20 CER of these models, and variation within these models, is another important application.
As stated by Blumenthal, “Information is the lifeblood of medicine.”21 Investments are being made in EHRs to provide evidence-based clinical guidelines and patient information to providers at the point of care to improve quality and promote a “learning health system” through the availability of electronic medical information.22 As such, electronic information will greatly facilitate and enable clinical research, including CER.
Our study has several limitations. We studied only 21 providers using a non-randomized design. Our study was conducted in 1 geographic region among community-based, solo, and small-group practitioners, limiting generalizability. The providers have different baseline error rates, which may be a result of our sample size or due to true differences in the types of providers who chose to adopt one type of e-prescribing system over another. We have controlled for the baseline difference in error rates in our analysis, but future studies evaluating this issue should be conducted. Because prescribers were aware that we were reviewing their prescriptions, our error rates may be underestimates due to the Hawthorne effect. We were unable in our analysis to do patient case mix adjustment and thus cannot determine how the benefits and harms associated with particular mistakes by condition may differ. We studied only 2 e-prescribing systems and focused solely on prescribing errors. However, we reviewed several thousand prescriptions and the e-prescribing systems incorporated many features recommended by an expert review panel.23 In addition, the implementation support was markedly different between the 2 groups and may have led to lower error rates among the standalone adopters who received more support.
PDF is available on the last page.