IT-Enabled Systems Engineering Approach to Monitoring and Reducing ADEs

Published Online: March 19, 2012
Ranjit Singh, MD; Diana Anderson, EdM; Elizabeth McLean-Plunkett, MA; Ron Brooks, BS; Angela Wisniewski, PharmD; Nikhil Satchidanand, PhD; and Gurdev Singh, PhD
Objectives: To develop and pilot-test a Web-based implementation of a team resource management (TRM) intervention to improve medication safety in primary care.


Study Design: Randomized controlled trial.


Methods: Eight practices were randomized to either the Web-based TRM or usual practice (4 practices in each group). Primary outcome was adverse drug events (ADEs) in older adults, ascertained using a trigger tool chart review at two 12-month periods (before and after the intervention). The prospective TRM approach, designed to inculcate ownership and empowerment, facilitates systematic appraisal of risk and error reduction. This approach uses the highly adaptable and transferable Safety Enhancement and Monitoring Instrument that is Patient Centered.


Results: The rate of ADEs decreased from 25.8 to 18.3 per 100 patients per year in the intervention group. The rate was virtually unchanged in the control group (24.3 vs 24.8). In an analysis of covariance at the practice level, being in the intervention group was associated with a lower rate of ADEs. The interaction between time (preintervention vs postintervention) and group (intervention vs control) was not signifi cant (P = .104) but showed a trend toward a decrease in the intervention group compared with the control group over time.


Conclusions: The Web-based TRM intervention proved feasible and demonstrated potential for effectiveness in various ambulatory settings. This pilot study was limited by small size and short follow-up period. Future studies should test the intervention on a larger scale over a longer period of time and should explore methods for overcoming common barriers to change.


(Am J Manag Care. 2012;18(3):169-175)
This randomized controlled trial pilot-tested a Web-based implementation of a self-empowering team resource management (TRM) intervention aimed at improving medication safety in primary care settings.

  •  The intervention sites showed a significant reduction in adverse drug events.

  •  Interaction between time (pre- vs postintervention) and the 2 arms of the study was not statistically significant.

  •  This study suggests that the Web-based TRM has the potential to improve medication safety in busy primary care offices.
Medication use is recognized to be a high-risk activity across all settings. A 2007 Institute of Medicine (IOM) report on this subject acknowledges that the rates and impact of medication errors are huge but are poorly understood.1 In ambulatory settings, medication errors and adverse drug events (ADEs) are important safety issues. Gurwitz and colleagues have estimated (by extrapolation) that Medicare enrollees alone suffer approximately 500,000 preventable ADEs per year.2

Lack of awareness of the type, incidence, and consequences of errors in any setting is one of the most important barriers to reducing these errors as well as improving safety and quality of care. The most commonly used method for estimating vulnerabilities in healthcare is to retrospectively collect and count errors through voluntary reporting systems (often referred to as incident reports). These are fraught with difficulty due to various issues, including underreporting; according to IOM’s 2000

report, only 5% of known errors are typically reported.3 Error reporting often does not promote understanding of the organizational structure and processes of care. Instead it tends to be associated with blame and shame, and frequently results in antagonism between team members that undermines mutual respect, trust, and cooperation. Bates and colleagues have described diffi culties involved in defi ning and quantifying errors; they report that even direct observational studies, which are highly labor intensive, often miss errors.4

An alternative approach that is prospective rather than retrospective, and encourages involvement of all team members in identifying and prioritizing safety and quality problems, invokes failure modes and effects analysis. This type of analysis has been widely used in other high-risk industries and has been advocated by the IOM as a means of analyzing a system to identify its weaknesses (failure modes) and possible consequences of failure (effects), and to prioritize areas for improvement.3 We adapted and tailored this methodology to allow for the levels of resources and expertise available in ambulatory settings and developed an instrument that has been well received by staff in a variety of settings. The details of the concepts, rationale, and processes behind this highly transferable instrument, termed Safety Enhancement and Monitoring Instrument that is Patient Centered (SEMI-P), are described elsewhere.5-7

The objectives of this study were to develop and pilot-test an information technology (IT)–based team resource management (TRM) system, based on SEMI-P, aimed at improving medication safety in primary care. We examined (1) the ability of this intervention to reduce selected ADEs among geriatric patients, (2) the ability of this intervention to improve monitoring of geriatric patients taking selected chronic medications, and (3) how offi ce staff used and applied this IT-based TRM tool for improving geriatric medication safety.

Study Design

This was a randomized controlled trial of a Web-based TRM intervention to reduce ADEs in primary care. Randomization was at the site level; 4 sites were assigned to the intervention and 4 to a control state (usual practice). In all 8 sites, ADEs were ascertained using a previously published trigger tool methodology.2,8

All practices in the current IT-facilitated study were part of the Upstate New York Practice Based Research Network and had electronic medical records in place for at least 12 months prior to the start of the study. Both groups contained a variety of practice types including safety net practices. Urban, suburban, and rural practices of various sizes and with various ownership structures were represented. All staff at the above sites—including physicians, physician extenders, nurses, medical assistants, administrative staff (secretarial and management), and all others (eg, dieticians and social workers if present)—were invited to participate in surveys and team discussions.

The project addressed the IOM priority areas of frailty associated with old age and medication management. The study protocol was approved by the Social and Behavioral Sciences Institutional Review Board of the State University of New York at Buffalo.

METHODS

The intervention involved implementation of the Webbased TRM system in the 4 practices that were randomly assigned to this group. The system uses a cyclical safety improvement process facilitated by use of 2 anonymous online staff surveys. The fi rst survey instrument is SEMI-P, which is a failure modes and effects analysis tool designed for the ambulatory setting that focuses on medication management processes.5-7 The second instrument is the safety attitudes questionnaire– ambulatory version (SAQ-A), which provides measures of the safety climate.9,10 Among the published safety climate surveys available at the time of this study, the SAQ was determined by Colla and colleagues to have the best psychometric properties.11 For each survey, online instructional clips and automated analysis with visual presentation of results were developed and implemented as part of the IT system.

Figure 1 is a screen shot of the first of 12 pages of the online SEMI-P survey. Each member of every intervention practice (including all job categories) was invited to complete this survey anonymously as part of a staff meeting. The same procedure was followed for the SAQ-A.

At each site, after administration of each survey, the practice teams immediately regrouped and reviewed the results and commenced discussion. Figure 2 is a screen shot showing the compiled results of the SEMI-P, generated by the system in real time. Figure 3 shows example results from the SAQA in a visual format that was developed especially for this project to highlight strengths and weaknesses in the safety culture.

These steps were followed by a series of staff meetings, held during times when no patient care was being delivered (generally at lunchtime), which all available staff attended. At these meetings, the survey results were reviewed and discussed, leading to consensus-based prioritization of medication safety issues. This consensus was also aided by visualized displays of individual (anonymous) team members’ opinions. Examples of prioritized areas included poor patient education about medications, high no-show rate, poor medication tracking, and poor coordination/teamwork with respect to handling of medication refi ll requests.

In subsequent staff meetings, teams worked together to address the chosen priorities by developing feasible system changes to improve medication safety. Examples include consistent use of patient education materials for high-risk medications, inclusion of diagnosis on prescriptions, patient reminders regarding follow-up, patient-carried medication lists, redesign of medication refi ll work fl ow, better training and follow-up for new personnel, and better employee performance feedback. The figures in the eAppendix (available at www.ajmc.com) illustrate the visual aid to consensus forming and the overall view of the whole methodology.

The initiatives tool within the software was used by staff to defi ne their goals and objectives for safety improvement, identify and assign specific work steps to individual team members, track progress, coordinate meetings, and remind staff of their commitments. The indicators tool was used to define specific measurable outcomes related to each initiative and to graphically track these over time in order to determine whether the stated objectives were being met.

Outcome Ascertainment

The primary outcome at the 4 intervention and 4 control sites was the rate of ADEs (measured using a trigger tool methodology). A secondary outcome was compliance with Healthcare Effectiveness Data and Information Set (HEDIS) guidelines for laboratory monitoring for patients who were prescribed certain medications chronically (meaning that they were prescribed the medication for 6 or more months out of a 12-month period). Both of these outcomes were for older adults (aged >65 years) since these patients are known to be at higher risk of adverse events.

A Web-based data capture tool was developed and implemented as part of the IT system for both the trigger tool and the HEDIS measure.

These 2 outcomes were ascertained for a baseline period defi ned as the 12 months before the start of the intervention and an end point period defi ned as the 12 months after the start of the intervention. For each period (baseline and end point), research assistants reviewed an independent sample of 100 charts of patients aged >65 years at each site. The HEDIS laboratory monitoring measurement was completed by these research assistants. For the trigger tool, the research assistants conducted the fi rst of 2 steps, known as the screening step. Charts identifi ed in this fi rst step as having triggers underwent secondary review (the review step) by a physician or pharmacist who reviewed each trigger to determine whether an adverse drug event occurred, and if so, its severity and preventability.

Analysis

PDF is available on the last page.

Feature

Recommended Reading