Rajiv Jain, MD

To assess the early impact of implementation of the electronic consults (e-consults) initiative by the Veterans Health Administration (VHA), designed to improve specialty care access.

Observational cohort study exploiting a natural experiment begun in May 2011 at 12 VHA medical centers and expanded to 122 medical centers by December 2013.

The following were assessed: 1) growth of e-consults by VHA regional networks, medical centers, and specialty; 2) location of patient’s primary care provider (medical center vs community-based outpatient clinic [CBOC]); 3) potential patient miles needed to travel for a specialty care face-to-face consult in place of the observed e-consults using estimated geodesic distance; 4) use of specialty care subsequent to the e-consult.

Of 11,270,638 consults completed in 13 clinics of interest, 217,014 were e-consults (adjusted rate, 1.93 e-consults per 100 consults). The e-consult rate was highest in endocrinology (5.0 per 100), hematology (3.0 per 100), and gastroenterology (3.0 per 100). The percentage of e-consult patients with CBOC-based primary care grew from 28.5% to 44.4% in the first year of implementation and to 45.6% at year 3. Of those e-consult patients from community clinics, the average potential miles needed to travel was 72.1 miles per patient (SD = 72.6; median = 54.6; interquartile range = 17.1-108), translating to a potential savings of 6,875,631 total miles and travel reimbursement costs of $2,853,387.

E-consult volume increased significantly since inception within many medical and surgical specialties. For patients receiving primary care at one of more than 800 CBOCs, e-consults may decrease travel burden and direct travel costs for patients.

This article discusses a broad-scale implementation of outpatient specialty care e-consults within a patient-centered medical home model for a large integrated health system. 

This can be a patient-centered model that saves patients travel time and money. 

E-consults may obviate the need for some specialty care visits. 

Policy to financially support this model may be prudent. 

Increased uptake may occur in specific medical specialties.

The Veterans Health Administration (VHA) is the largest integrated healthcare delivery system in the United States. Like an accountable care organization (ACO), the VHA provides care to a defined group of enrollees with the goal of delivering coordinated, high-quality, patient-centered care.

 Approximately 50% of the 8 million veterans cared for in the VHA receive specialty care. Efforts to bring primary care closer to veterans’ homes have led to more than 800 community-based outpatient clinics (CBOCs). In contrast, specialty care remains largely concentrated in urban medical centers, although 41% of veterans enrolled in VHA live in rural communities. These and other barriers to access present challenges for the delivery of specialty care.

To improve access, efficiency, and coordination between specialty care and primary care, the VHA’s Specialty Care Transformation Program Office implemented electronic consults (e-consults) in 2011. Primary care clinicians request clinical guidance with the expectation of an expeditious, albeit asynchronous, response. E-consults and responses are integrated into the VHA’s electronic health record (EHR). Through e-consults, primary care clinicians can incorporate specialty care advice into veteran care and better prepare patients for specialty care visits if needed. Although e-consults have been implemented on smaller scales in other settings, the implementation of a national e-consult program has not been previously described.

The study’s objective was to describe the spread and impact of the VHA national e-consult program. We assessed the growth of e-consults by VHA regional networks, medical centers, and clinical specialty, and evaluated the use of e-consults based on the location of the patient’s primary care provider (PCP) (medical center vs CBOC). Finally, we estimated the number of miles patients may have potentially traveled for an in-person specialty care visit in the absence of an e-consult program. E-consultation implementation across the VHA system can serve as a model for technology-based interventions to improve access to specialist consultation and provide patient-centered specialty care.

The VHA has 152 medical centers and over 800 CBOCs within 21 regional networks (Veterans Integrated Service Networks [VISNs]). Each regional VISN has 4 to 8 medical centers and generally covers 1 to 4 states. In addition to medical centers, each VISN has CBOCs located 10 to 300 miles from a medical center, providing primary care to patients closer to their homes. To support veterans living far from medical centers (41% of the veteran population lives in rural areas), patients are frequently eligible for financial reimbursement for miles traveled for care. In general, veterans are assigned to a PCP at a CBOC or medical center closest to them.

At the discretion of the PCP, e-consults can be entered into the EHR for any specialty in which an e-consult mechanism has been implemented. E-consults about specific questions are generally entered as free text or into templates with pre-specified questions. Responding specialty clinicians have access to the medical records, including progress notes, laboratory data, radiology tests, and medications. E-consults generally focus on narrow questions that help address questions of diagnosis, testing, or management of a condition. Either specialist or PCP can convert an e-consult into a request for a face-to-face consultation. Follow-up communication with patients about recommendations, if any, is the responsibility of the PCP, unless otherwise identified in the EHR.

In May 2011, the e-consult program for certain specialties began at 12 VHA medical centers in 7 VISNs. Through a competitive process, 12 pilot-site medical centers received limited funding and national guidance for business rules. Oncology, diabetes, endocrinology, dementia, neurosurgery, cardiology, hematology, liver transplant, pain medicine, and rheumatology were the included specialties. Following initial rollout at pilot sites, Specialty Care Transformation worked with VISN leadership to expand the use of e-consults nationally. Further funding was provided until September 30, 2014, to all VISNs; local facilities could implement e-consults in other clinical specialties based on facility needs. Subsequently, e-consults became a national priority and use of e-consults was tracked for each medical center as a component of VHA goals for patient-centered care delivered virtually.

We focused on the following clinics of interest to Specialty Care Transformation: endocrinology, hematology, neurology, pulmonary, nephrology, urology, cardiology, gastroenterology, oncology, pain medicine, geriatric, dementia, and diabetes. Consults were identified in administrative outpatient visit data that included specialty clinic category (ie, clinic stop codes) and a designated Current Procedural Terminology code to distinguish e-consults. We focused on medical centers in the continental United States and patients with assigned PCPs.

E-consult volume adjusted for clinic volume. 

Many medical centers implemented e-consults in multiple specialties. For every specialty/site combination, we calculated daily rolling rates of e-consult volume adjusted for total clinic volume. For a given 30-day window, we summed the number of e-consults and divided by the number of total consults.

 Travel distance was estimated by the geodesic (“as the crow flies”) distance from patients’ residence to their “home station” (ie, facility where they received primary care). The adjusted distance was considered the difference between the distance from the patient’s residence to the specialty care facility and the distance from the patient’s residence to their primary care facility. Distances were considered erroneous and set to missing if the patient distance to primary care exceeded 150 miles or the adjusted distance was lower than —25 miles. (Negative values may occur if the patient’s primary care location is farther away than the specialty clinic completing the e-consult.) Primary care visits outside the specialty care administrative region didn’t contribute to home station identification; patients receiving primary care outside that region were considered to have missing home stations. Patients with home locations outside the contiguous 48 states were excluded from all analyses.

We aggregated the number of completed e-consults by specialty, regional network, and location where patients received primary care, either at a CBOC or medical center. We reported the quarterly frequency of e-consults from May 2011 to December 2013, stratified by VISN and specialty. Quarterly e-consults were adjusted by total consults and expressed as a rate. We calculated the e-consult rate to facilitate comparisons across VISNs, facilities, and clinic specialties.

We calculated the proportion of patient consults receiving primary care at a CBOC for each yearly quarter, stratified by type of consult (e-consult or not). We used χ

 tests to compare binary outcomes (patient receiving primary care at a CBOC; subsequent face-to-face consult after the first consult; and primary care visit in the 3 months following the first consult) between e-consults and face-to-face consults. Among patients receiving primary care at a CBOC, we used Wilcoxon rank-sum tests to compare the distribution of miles needed to travel between e-consults and face-to-face consults.

Statistical tests were performed using R software, version 3.1.0, (R Foundation for Statistical Computing, Vienna, Austria). Initial data aggregation was performed with SAS version 9.3 (SAS Institute, Cary, North Carolina).

We identified 14,182,136 specialty care visits between May 1, 2011, and December 31, 2013, of which 262,143 were e-consults. We excluded visits outside of the continental United States (n = 275,611), visits occurring outside of VHA medical centers (n = 1,265,169), visits by patients without an assigned primary care clinic (n = 692,726), visits by patients whose assigned primary care clinic was not a CBOC or VHA medical center (eg, residents of long-term care facilities) (n = 986,759), sites that had less than 1 e-consult per 10,000 visits (n = 1,127,251), and sites with less than 1000 total visits across the time period (n = 87,973). These categories had some overlap, and 2,911,498 visits in total were excluded.

There were 217,014 completed e-consults that met the inclusion criteria in the 13 clinics of interest in all 21 VISNs. During the same time period, there were 11,270,638 total consults that met the inclusion criteria, giving an adjusted rate of 1.93 e-consults per 100 consults. At the VISN level, the rate of e-consults per 100 total consults varied (mean = 2.0; SD 

0.8; median = 1.9; interquartile range [IQR] = 1.4-2.4). Although there has been some variability in uptake of e-consults across VISNs, it has generally increased with time (

). At the individual medical centers, the range of e-consults has varied from 0.02 to 26 per 100 consults.

E-consults were completed most frequently in cardiology (n = 44,322), gastroenterology (n = 29,043) and endocrinology (n = 23,972). Adjusting for total consult volume, the rate of e-consults were highest in endocrinology (5.0 per 100 total consults), hematology (3.0 per 100 total consults), and gastroenterology (3.0 per 100 total consults) (

Comparing e-consults to face-to-face consults, there were differences in both: 1) the percent of patients coming from outside the specialty care medical center, and 2) the distance from their primary care center to the specialty care center. During the study period, comparing the patients participating in e-consults with those participating in face-to-face consults, the e-consult population had a higher percentage of patients who received primary care at CBOCs (45.3% vs 36%; 

 <.0001). In the first year of implementation, the percent of e-consult patients with CBOC-based primary care grew from 28.5% to 44.4%. This increase slowed in the subsequent 2 years, growing from 44.4% to only 45.6%. In the same 3-year period, the percent of face-to-face consult patients with CBOC-based primary care grew slightly, from 34.7% to 37% (see 

). Among the patients receiving primary care at CBOCs outside of a medical center, the distance from their PCP to the medical center was farther for patients receiving e-consults (38 vs 32 miles; 

 <.0001) compared with patients receiving face-to-face consults. These results suggest that compared with face-to-face consults, e-consults are used more frequently for patients with lower spatial access to specialty care.

Geographic distributions of “hub” medical centers and “spoke” community clinics demonstrate wide variation in number of spoke connections for each hub, as well as number of consults between each hub and spoke combination (

). The Figure 3 map highlights the geographic reach of e-consults to patients with potentially less access to specialty care.

In the 3 months following a specialty care consult, patients who had received e-consults were less likely to have subsequent face-to-face visits with the same specialty compared with patients with a face-to-face consult (18.6% vs 43.2%; 

 <.001). In contrast, e-consult patients were more likely to have a subsequent primary care visit compared with patients with a face-to-face consult (78.8% vs 67.4%; 

 <.001), suggesting that patients receiving an e-consult have a higher probability of follow-up with their PCP for subsequent care compared with patients with a face-to-face specialty care visit.

Finally, we calculated the potential miles needed to travel to receive a face-to-face consult in lieu of an e-consult. Among the 217,014 e-consults, 93% (201,735) had acceptable distance calculations. Among those patients, 52.7% (n = 106,310) received primary care from the same facility; thus there were no potential mileage-traveled differences between an in-person consult and an e-consult. For the 47.3% (n = 95,425) of e-consults for patients from CBOCs, the average potential miles needed to travel was 72.1 miles per patient (SD = 72.6; median = 54.6; IQR = 17.1-108) (see 

). This distance translates into 6,875,631 total potential miles needed to travel. During the time period, the VHA reimbursed patient mileage at a rate of 41.5 cents per mile; thus, the total potential miles needed to travel would translate into potential direct costs of $2,853,387. These mileage estimates are conservative because they are based on “as the crow flies” rather than true distance necessary to be driven.

Our study describes the implementation, spread, and impact of the VHA national e-consult program. Since VHA implementation in 2011, nearly 2% of all consults were e-consults without evidence for a plateau in their use. In addition, receipt of e-consults for patients with primary care at CBOCs was associated with reductions in the total potential miles traveled and travel costs. These findings have important implications for the VHA for addressing both spatial and nonspatial barriers to access for specialty care services, especially for veterans who live in rural communities. Furthermore, our results provide potential approaches to specialty care access for ACOs and other integrated healthcare delivery systems.

 Like other healthcare institutions, VHA has adopted the “triple aim” framework for optimizing healthcare delivery to improve the patient healthcare experience and the health of the patient population while balancing per capita cost.

 The implementation of e-consults is a potential mechanism by which VHA can strive to achieve these aims.

E-consults may improve the overall patient experience of care by reducing inconvenience and waiting times for specialist consultation/input. In addition, patients have the opportunity to avoid long travel distances to see specialists and to obtain specialty advice in a timely manner compared with an in-person visit. For patients who either prefer or require an in-person specialist visit, the initial e-consult may also provide an opportunity to be more prepared for the visit. For example, by arranging for diagnostic or laboratory tests in advance, some visits to the specialist could be avoided and time to needed services potentially decreased. We found that for the majority of consults, an in-person visit did not occur following the e-consult. This finding reinforces the study of Keely et al, who found that specialists were able to answer the e-consult without needing further information 89% of the time.

E-consults have the potential to improve quality of care for both individuals and the general patient population. First, e-consults improve patient access to specialist expertise, especially for the large number of veterans who live in rural communities. We found higher utilization of e-consults for those in CBOCs and more follow-up appointments with primary care after e-consults than for those patients not receiving e-consults. Second, e-consults promote the use of a standardized referral process with iterative communication that can lead to increased effectiveness of care delivered. In fact, preliminary results of interviews with clinicians have found that both PCPs and specialists value the benefits e-consults have for patients. In addition, e-consults provide an opportunity to improve care coordination for patients as a consequence of better PCP-specialist communication. Third, when combined with the capabilities of an EHR, it is possible to identify populations who would benefit from additional specialty care expertise. “Consultation” may be pre-emptive, thereby avoiding preventable morbidity.

 This population management approach will need to be evaluated prospectively.

E-consults may reduce the per capita cost of healthcare from the healthcare system perspective. Reductions may be accomplished by improved efficiency of use of high-cost specialists. On the other hand, increased access to specialists may result in increased testing and consequent expense; healthcare value may improve, albeit without reducing costs.

 However, we believe that the overall costs will be less with use of e-consults over time due to primary care coordination. VHA cost structure also has some unique aspects. The VHA outsources specialty care—particularly for rural veterans&mdash;because of difficulties for these veterans to access specialists. This so-called “fee-basis” care consumes funds that are used for outside care and do not return to the system. In addition, specific populations of veterans are reimbursed for their travel to clinic visits; generally, this reimbursement is provided to veterans with at least a 30% service-connected disability. The elimination of unnecessary visits subsequently reduces travel reimbursement costs. Finally, efficiency may be improved by better coordination of care and less fragmentation of care.

 We are not suggesting that e-consults will obviate the need for face-to-face visits, but this technology-supported platform can optimize efficient use of specialist expertise, which may be limited in differing geographic locations and health systems.

E-consult programs have been established in a variety of healthcare systems. However, in order for e-consults to be more widely adopted, changes to the payment systems are needed. The VHA’s model of salaried physicians and capitated reimbursement provides a more favorable environment for e-consult implementation. In contrast, under the fee-for-service model, specialists must physically see the patient and bill for a separate visit in order to receive payment. Thus, although the reach of technology expands and allows the extension of care delivery into areas where few specialists reside, payment models lag behind and may hinder the potential spread of such programs. It will be important to assess how the different models for reimbursement and shared savings in ACOs under the Affordable Care Act evolve; their effects on different types of specialist care delivery and coordination of care remain to be seen.

Our study has several limitations. First, the rate of increase of e-consults may reflect changes in documentation; formalized coding of e-consults was part of the initiative because of its relevance to tracking patient care and the necessity to capture the work of specialty care providers. Second, the e-consult initiative was conducted within an integrated healthcare delivery system and a single EHR enabling ready access to information for both PCPs and specialists. Nevertheless, trends in use of EHRs and efforts to achieve interoperability may render implementation of e-consults in other settings less difficult in the future. The logistical issues for implementing such a system in the fee-for-service sector are likely to be different. This may include recognition that some healthcare systems may need to contract out for specialist care; contracted specialists may not be interested in supporting large-scale e-consult initiatives. However, as the country moves toward ACOs, the lessons learned in implementing this initiative in the VHA may be applicable to ACOs. Finally, although the number of e-consults has increased, we do not know if these e-consults have resulted in improved patient outcomes. Future studies must be done to address any potential unintended consequences for patients, clinicians, and healthcare systems, as well as other unanswered questions. Last, the patients’ perspectives need to be considered and to that end, we will be conducting interviews with patients in the near future to obtain their feedback and experience with e-consults.

Since the e-consult initiative rolled out in 2011, there has been a steady increase in the use of e-consults across 21 networks and for a diverse group of specialties. Specifically, the number of patients with primary care at CBOCs—which are relatively remote from sites where specialists are located&mdash;receiving an e-consult has increased. Use of e-consults has been associated with reductions in potentially unnecessary driving, as evidenced by potential miles saved. These findings have important implications for the VHA in terms of increasing access to specialty care in general and to veterans who live in rural communities in particular, and the findings may apply to other healthcare systems.

The authors thank the Evaluation Center Quantitative Analysis Group: Jackie Szarka, PhD; Jeffrey A. Todd-Stenberg, BA; Christian D. Helfrich, PhD, MPH; Anne C. Lambert-Kerzner, PhD, MSPH; Catherine T. Battaglia, PhD, MS; Thomas J. Glorioso, MS; Kent Davis; Katherine M. Fagan Williams, MPH. All authors and the Evaluation Center Quantitative Group had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

 Office of Specialty Care Services (SK, OC, GG), Patient Care Services (RJ), Veterans Health Administration, Washington, DC; Louis Stokes Cleveland VA Medical Center (SK, DCA), Cleveland, OH; Case Western Reserve University School of Medicine (SK, DCA), Cleveland, OH; Eastern Colorado VA Medical Center (EC, PMH), Denver, CO; Puget Sound Health Services Research & Development Center of Innovation, Department of Veterans Affairs (DHA), Seattle, WA; Department of Medicine, University of Washington (DHA), Seattle, WA; New Jersey Veterans Health Care System, East Orange Campus (C-LT, HF), East Orange, NJ

 This material is based upon work supported by the US Department of Veterans Affairs, the Office of Specialty Care Transformation and the Office of Research and Development Quality Enhancement Research Initiative.

 The authors are employed by the VA and this work was funded, in part, by the VA Office of Specialty Care Transformation, the office overseeing the e-consult initiative. The views expressed here do not necessarily reflect the position or policy of the Department of Veterans Affairs. The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (OC, HF, DHA, DCA, GG, RJ, PMH, EC, SK); acquisition of data (DHA, GG, EC, SK); analysis and interpretation of data (OC, HF, DHA, C-LT, DCA, GG, RJ, EC, SK); drafting of the manuscript (DHA, C-LT, DCA, EC, SK); critical revision of the manuscript for important intellectual content (OC, DHA, DCA, GG, RJ, PMH, EC, SK); statistical analysis (HF, C-LT, EC, SK); provision of patients or study materials (GG, SK); obtaining funding (OC, DHA, DCA, SK); administrative, technical, or logistic support (DHA, GG, PMH, SK); and supervision (DHA, GG, RJ, SK).

 Susan Kirsh, MD, MPH, Case Western Reserve University School of Medicine, 10701 East Blvd, Cleveland, OH 44106. E-mail: susan.kirsh@va.gov.

1. Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. 

2. Kizer KW, Dudley RA. Extreme makeover: transformation of the Veterans Health Care System. 

3. Kizer KW. Veterans and the Affordable Care Act. 

4. Chen AH, Yee HF Jr. Improving the primary care-specialty care interface: getting from here to there. 

5. Mehrotra A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. 

Articles

E-consult implementation grew from 12 to 122 VHA sites with multiple specialties. The adjusted e-consult rate of 1.93/100 consults saved significant patient travel miles and costs.

Impact of a National Specialty E-Consultation Implementation Project on Access

To assess the role of a Toyota production system (TPS) quality improvement (QI) intervention on appropriateness of perioperative antibiotic therapy and in length of hospital stay (LOS) among surgical patients.

Pre-post quasi-experimental study using local and national retrospective cohorts.

We used TPS methods to implement a multifaceted intervention to reduce nosocomial methicillin-resistant Staphylococcus aureus infections on a Veterans Affairs surgical unit, which led to a QI intervention targeting appropriate perioperative antibiotic prophylaxis. Appropriate perioperative antibiotic therapy was defined as selection of the recommended antibiotic agents for a duration not exceeding 24 hours from the time of the operation. The local computerized medical record system was used to identify patients undergoing the 25 most common surgical procedures and to examine changes in appropriate antibiotic therapy and LOS over time.

Overall, 2550 surgical admissions were identified from the local computerized medical records. The proportion of surgical admissions receiving appropriate perioperative antibiotics was significantly higher (

<.01) in 2004 after initiation of the TPS intervention (44.0%) compared with the previous 4 years (range, 23.4%-29.8%) primarily because of improvements in compliance with antibiotic therapy duration rather than appropriate antibiotic selection. There was no statistically significant decrease in LOS over time.

The use of TPS methods resulted in a QI intervention that was associated with an increase in appropriate perioperative antibiotic therapy among surgical patients, without affecting LOS.

We found that the use of Toyota production system methods as part of a nosocomial methicillin-resistant Staphylococcus aureus prevention initiative on a surgical unit resulted in

interventions to improve the quality of care in other areas.

Specifically, we found increased compliance with recommendations regarding appropriate prophylactic perioperative antibiotic use.

Changes were primarily driven by improved compliance with appropriate antibiotic therapy duration not exceeding 24 hours.

The Toyota production system (TPS) industrial engineering approach to improve manufacturing quality originated in the automobile industry and is being applied to quality improvement (QI) in healthcare delivery systems. In the TPS model, frontline work groups identify problems, experiment with possible solutions, measure the results, and implement strategies to improve quality, resulting in a “ground-up” rather than “top-down” approach to solving system problems.

Although the first article describing the use of the TPS in healthcare was published more than 20 years ago,

 numerous recent publications describe the use of the TPS model to improve quality or efficiency in the healthcare setting.

 Toyota production system methods are associated with improvements in (1) diagnostic accuracy of thyroid gland fine-needle aspiration and Papanicolaou tests,

 (3) elimination of dead stock in radiology,

 and increased medical record completion in the emergency department.

Beginning in 2001, a Veterans Affairs medical center (VAMC) instituted TPS methods to reduce methicillin-resistant Staphylococcus aureus (MRSA) infections on a general surgical floor. In addition to using the TPS process to facilitate implementation of MRSA prevention,

 the intervention evolved to address other areas for improvement in healthcare quality and efficiency on the surgical unit, such as developing and implementing an intervention to increase appropriate prophylactic perioperative antibiotic therapy, redesigning the supply cabinet and wheelchair distribution system, and improving the bar code scanning system for dispensing medications.

The objectives of the present study were to determine (1) whether the QI intervention for perioperative antibiotic therapy was associated with improvements in selection and duration of prophylactic therapy and (2) whether the overall MRSA prevention initiative was associated with a reduction in length of hospital stay (LOS) as a secondary outcome. We hypothesized that implementation of the QI intervention for perioperative antibiotic therapy would increase the proportion of admissions receiving appropriate prophylactic antibiotic selection and duration and that the overall MRSA initiative would be associated with a decreased LOS.

We used a quasi-experimental design to assess the association between implementation of a perioperative antibiotic prophylaxis QI intervention and appropriate perioperative antibiotic therapy over time. We also assessed the effect of the MRSA prevention initiative on LOS as a secondary outcome reflecting the overall efficiency of hospital care. This study received exempt human subject approval from the local institutional review board.

We used clinical and administrative databases to identify 2 local cohorts of surgical patients from the intervention VAMC and a national cohort of similar patients from all other VAMCs nationwide. Using the Veterans Health Information Systems and Technology Architecture (VistA) database, we identified a local cohort of patients who were admitted to the local surgical unit between fiscal years 2000 and 2004 with a primary International Classification of Diseases, Ninth Revision, Clinical Modification (

) surgical procedure code. To maximize the probability of patients’ spending time on the surgical intervention unit, we identified patients who spent their entire hospital stay on the unit and further refined the sample by including only those patients who had 1 of the 25 most common surgical procedures. This was necessary to ensure adequate numbers of patients within each surgical procedure type and to facilitate operationalization of appropriate perioperative antibiotic therapy. Once the top 25 procedures were identified for patients who spent 100.0% of their time on the intervention unit, we expanded our selection to include anyone who spent at least some of his or her admission on the surgical unit (

). Information on antibiotic therapy was only available from VistA and was assessed only in this local cohort defined using VistA. For the LOS analyses, we excluded patients with an LOS longer than 90 days because they were more likely to have a severe surgical or medical complication. Therefore, data from this initial local VistA cohort were used to examine perioperative antibiotic selection and changes in LOS over time.

To examine differences in LOS between our local VAMC and the other VAMCs nationally, we used the Veterans Affairs (VA) National Patient Care Database (NPCD) to identify a local cohort and a national cohort of patients who had an 

 surgical procedure code at the intervention VAMC for 1 of the same top 25 procedures. Finally, to identify the national study cohort, we used the same inclusion and exclusion criteria but extended our patient identification to all 116 remaining VAMCs nationwide.

The TPS intervention was initially implemented on a 36-bed general and subspecialty surgical unit at the intervention VAMC, a 146-bed tertiary care hospital for veterans. On this unit, the attending VA physicians are faculty members of the local university school of medicine and provide clinical training for surgical residents.

In November 2001, the surgical unit initiated a TPS intervention aimed to reduce nosocomial MRSA. The TPS team leader, a staff nurse (EEM), worked with unit staff to identify obstacles to identification and isolation of patients colonized with MRSA and to engage staff in problem solving for better MRSA infection control. The first interventions, initiated in spring 2002, involved training attending physicians, house staff, nurses, and laboratory technicians on proper hand hygiene, as well as systematic culturing of all admissions and isolating patients colonized with MRSA.

 Although the primary goal of this initiative was to reduce nosocomial MRSA, efforts evolved to address additional processes of care that interfered with the new MRSA prevention procedures. With increased recognition that appropriate antibiotic use in the operating room and on hospital floors may also help prevent the development of MRSA and other postoperative bacterial infections, the staff undertook the task of improving appropriate selection and duration of perioperative antibiotic therapy. Staff on the unit identified the following 4 key barriers to this process of care: (1) reliance on a computerized bar code medication administration system that was frequently nonfunctional, (2) lack of standardization of perioperative antibiotic orders, (3) reliance on paper-based rather than computer-based medication records, and (4) monthly rotations of new surgical residents responsible for writing antibiotic orders. These barriers were identified and addressed in an iterative process as guided by TPS methods.

Problems experienced by nursing staff in using the bar code medication administration system were addressed in late 2002 by purchasing a backup battery pack to ensure constant computer availability. To address lack of medical provider standardization in prescribing antibiotics, beginning in 2003 TPS staff enlisted the assistance of hospital pharmacists to establish evidence-based prophylactic antibiotic selection criteria for all common surgical procedures using the most recently published guidelines for specific surgical procedure types.

 This list of recommended antibiotics was approved by the head of infectious diseases (RRM) and the chief of surgery (MAW) at the intervention VAMC. By late 2003, guidelines for surgical antibiotic prophylaxis were available in the computerized VA order entry system and were printed on pocket cards distributed to all VA surgical attending physicians and residents. To address issues regarding appropriate duration of antibiotic therapy, a computerized standing order automatically discontinued antibiotics 24 hours after surgery. In addition, education was provided to new surgical attending physicians and residents at regular intervals.

Patient demographic and clinical data were abstracted from VistA and NPCD from the time of the index admission. The demographic characteristics were age in years, race/ethnicity (white vs nonwhite), sex, and admission source (nonscheduled or scheduled outpatient, nursing home or domiciliary, other VA or non-VA hospital, or other). We also assessed the number of comorbid conditions using the Charlson Comorbidity Index

 and the type of surgical procedure using 

Our assessment focused on 2 processes of care, appropriateness of perioperative antibiotic therapy and LOS. The analytic sample for the assessment of antibiotic therapy was restricted to surgical procedures performed within 1 day of admission, with no subsequent surgical procedures within 2 days. These criteria were used to limit the sample to patients who were specifically admitted for the surgical procedure and did not have more than 1 operation necessitating different types of antibiotic therapy.

Appropriate antibiotic therapy was defined as prescription of the recommended antibiotic agents consistent with the locally developed surgical prophylaxis guideline recommendations for 5 classes of surgical procedures (ie, neurosurgery, vascular, gastrointestinal and biliary, genitourinary, and orthopedic) for a duration not exceeding 24 hours. For carotid endarterectomy, antibiotics were not recommended by our hospital guideline; therefore, no antibiotic prescription was considered compliant. Operations of the skin were not assessed because of the heterogeneity of surgical infections and a lack of consensus on appropriate antibiotics.

We defined LOS as the date of discharge minus the date of admission and defined postsurgical LOS as the date of discharge minus the date of surgery. For patients who died in the hospital, LOS was censored on the date of death.

We summarized categorical variables using frequencies and continuous variables using medians and interquartile ranges (IQRs) because of the skewed nature of the frequency distributions. To compare surgical patients at the intervention VAMC with those at the other 116 national VAMCs, we used 

 tests for categorical variables and Wilcoxon rank sum tests for continuous variables. We used a 2-sided 

 <.05 to define statistical significance. All comparisons were performed using SAS 8.2 (SAS Institute, Cary, NC).

We used logistic regression to analyze appropriateness of perioperative antibiotic prophylaxis before and after implementation of the intervention, while accounting for the correlation of more than 1 surgical procedure for the same person (STATA 9.0; StataCorp LP, College Station, TX). Because the intervention was not applied to appropriateness of antibiotic therapy until the end of 2003, we used 2004 to define the first intervention year for these outcomes. To distinguish the effects of the intervention from any secular changes in perioperative antibiotic prescribing, we included a time variable in the models (year since 2000) and tested the differences in antibiotic compliance after the intervention (2004) compared with before the intervention (2000-2003) with adjustment for this linear trend.

To identify potential confounders, the research team reviewed all demographic and clinical variables that were available within the clinical and administrative databases and identified those variables that they believed could potentially confound the study outcomes. These variables included type of surgical procedure, admission source, age, and number of comorbid conditions. All of these potential confounders were then entered and retained in our models as control variables. An odds ratio greater than 1.0 indicates a greater likelihood of compliance during 2004 than in 2000 to 2003.

Because of the nonnormality of the data and the correlation of admissions for the same person or admissions from the same site, we analyzed LOS and postsurgical LOS using discrete survival analysis in SUDAAN 9.0 (Research Triangle Institute, Research Triangle Park, NC). We modeled LOS using a proportional hazards model that included covariate adjustment for the surgical procedure performed, admission source, age, number of comorbid conditions, and separate baseline hazards for each year. Hazard ratios (HRs) exceeding 1.0 correspond to a shorter LOS. In-hospital deaths were censored at the time of death. Models were also run that included patients with an LOS exceeding 90 days. These methods were used with the national data from the NPCD to test if the LOS at the intervention VAMC was similar to that at other VAMCs, as well as to test whether change in LOS after the intervention on the surgical unit was similar to the secular trends in LOS observed at the other VAMCs. The national analyses accounted for the correlation of admissions at the same site and admissions for the same person.

In the local cohort defined using VistA, we tested whether LOS changed after implementation of the intervention. The estimate of the intervention effect quantifies the effect of the intervention over and above a linear time trend, as estimated by including a linear term for time in the analysis. The analyses in the local cohort account for the correlation of admissions by the same person. For all LOS analyses, 2003 was considered the first year of the intervention.

Using VistA, we identified 5328 surgical admissions to the unit between 2000 and 2004, of which 2550 admissions had 1 of the 25 most common surgical procedures and an LOS of 90 days or shorter. Using the NPCD, we identified a similar local cohort of 2958 admissions at the intervention VAMC and 184,907 admissions at the 116 other VAMCs nationwide (

Most patients in the local study cohort identified using VistA were white (88.4%) and male (96.9%) and had a median (IQR) age of 66 (57-75) years (Table 1). Overall, 82.4% spent their entire hospital stay on the intervention surgical floor, and 8.2% underwent more than 1 surgical procedure. The most common types of surgical procedures were orthopedic (28.5%) and vascular (27.5%). As expected, the local VAMC cohort identified using the NPCD had demographic and clinical characteristics almost identical to those of the local VAMC cohort identified using VistA.

Compared with the national study sample, the local VAMC cohort identified using the NPCD was older (median age, 66 vs 65 years), and a greater proportion were white (89.4% vs 80.8%) (Table 1). The VAMC cohort had significantly higher rates of diabetes mellitus (30.4% vs 24.9%) and chronic pulmonary disease (14.9% vs 13.0%). However, the VAMC cohort had a lower incidence of malignant neoplasm (12.4% vs 16.1%). The local VAMC cohort also had a higher frequency of vascular and orthopedic procedures and a lower frequency of genitourinary procedures.

Appropriateness of Perioperative Antibiotic Therapy For the analysis of appropriate perioperative antibiotic therapy, we excluded 660 admissions for which the surgical procedure occurred after the first day of admission to limit the sample to patients more likely to be admitted for surgery. We also excluded 111 surgical procedures involving the skin because of the heterogeneity of surgical skin infections and a lack of consensus on appropriate perioperative antibiotic therapy for these infections. Therefore, appropriateness of antibiotic therapy was assessed in 1779 surgical admissions to the intervention VAMC. This subset (69.8%) of the local cohort did not differ from the full local cohort with regard to age, race/ethnicity, sex, or number of comorbid conditions (data not shown).

, the proportion of surgical admissions receiving appropriate antibiotic therapy was significantly higher (

 <.01) in 2004 after TPS initiation (44.0%) compared with any of the previous 4 years (range, 23.4%-29.8%). The overall improvement across this interval was attributable to improvements in compliance with duration of antibiotic therapy rather than improvements in appropriate antibiotic selection (

Overall, appropriateness of prophylactic antibiotic therapy significantly improved for neurosurgical, vascular, and orthopedic procedures between 2000 and 2004, but not all surgical procedures demonstrated a similar pattern of improvement (Table 2). The proportion of admissions receiving the appropriate selection of antibiotic therapy was significantly higher (

 <.01) for neurosurgical procedures in 2004 compared with linear trends from 2000 to 2003; the proportion of admissions receiving appropriate antibiotic agents was significantly lower for gastrointestinal and biliary procedures. The proportion of admissions who received antibiotic therapy for the appropriate duration was significantly higher for gastrointestinal and biliary, genitourinary, and orthopedic procedures in 2004 compared with linear trends for the corresponding procedures from 2000 to 2003.

, the median (IQR) LOS for surgical admissions (2000-2004) was shorter at the intervention VAMC (5 [3-9] days) than at the other 116 VAMCs nationally (6 [3-12] days). Although the overall LOS was shorter for surgical admissions at the intervention VAMC than at the 116 VAMCs nationally (HR, 1.2; 95% confidence interval [CI], 1.12-1.28; 

 < .01), there was no statistically significant decrease in LOS following full implementation of the TPS in 2003 compared with national rates (HR, 0.99; 95% CI, 0.91-1.09; 

 = .90). Including patients with an LOS exceeding 90 days or using postsurgical LOS as the outcome had negligible effects on these analyses.

In the local intervention VAMC cohort identified using VistA (

), the median (IQR) LOS was significantly shorter (

<.01) in 2003 to 2004 (5 [3-8] days) compared with 2000 to 2002 (6 [3-9] days). However, this difference in LOS was no longer statistically significant in analyses that adjusted for the surgical procedure, admission source, age, and number of comorbidities for the full sample (HR, 0.91; 95% CI, 0.76-1.08) or among those patients who spent their entire hospital stay on the intervention unit (HR, 0.87; 95% CI, 0.72-1.06).

In this study of the effect of a QI intervention to improve perioperative prophylactic antibiotic therapy, the TPS-driven intervention was temporally associated with an increased proportion of patients receiving appropriate prophylactic perioperative antibiotic therapy. The observed improvements in the overall compliance with appropriate antibiotic therapy were primarily the result of changes in compliance with appropriate postoperative antibiotic duration. Although the proportion of surgical admissions who received antibiotics not exceeding 24 hours from the time of operation increased significantly for 3 of 5 surgical procedure types (gastrointestinal and biliary, genitourinary, and orthopedic), higher rates of compliance with guideline recommendation for the selection of agents were restricted to neurosurgical admissions.

Our findings regarding improvement in postsurgical antibiotic duration are likely due to implementation of a standing order to discontinue antibiotics within 24 hours of surgery. The effectiveness of such an automatic stop order for reducing the duration of postsurgical antibiotic therapy is consistent with prior studies demonstrating that standardized postoperative order sets are effective at decreasing the duration of postsurgical antibiotic therapy

 and are more effective than educational interventions alone.

However, to avoid a “one size fits all” approach to improving postsurgical antibiotic therapy, surgeons were able to edit the electronic stop date of 24 hours to reorder antibiotics if indicated by the clinical situation. The explanation for the lack of a significant decrease in antibiotic therapy duration for neurosurgical and vascular procedures is not entirely clear and may be due to discretionary reordering of antibiotics by these surgeons. Our finding of variability in practice patterns by type of surgeon is consistent with a prior QI study

using Six Sigma methods, which found that the timing of preoperative prophylactic antibiotics improved for orthopedic, colorectal, and gynecologic surgical procedures but not for vascular surgical procedures.

Another possible explanation for the lack of universal improvements across surgical procedure types is that the TPS team chose to focus their efforts on the surgical procedures and surgeons they encountered most often. For example, the most common surgical procedures were orthopedic; initial evaluation revealed high compliance for antibiotic selection (>80%) but low compliance for appropriate duration (<15%). Discussions conducted by the QI team with orthopedic residents revealed a belief that antibiotic therapy needed to be administered until all surgical drains were removed. Subsequently, the TPS team met with new residents each month to educate them about the appropriate duration of antibiotic therapy.

We did not find evidence of significant changes in the overall or postsurgical LOS as a result of implementation of the TPS MRSA prevention initiative on the surgical unit. Length of stay is a general measure of hospital efficiency and, in contrast to appropriateness of perioperative antibiotic therapy, was not directly targeted as part of the TPS methods. Although we identified no prior studies about the effect of QI interventions on LOS, we identified 2 studies

 that used TPS or Lean methods to decrease emergency department LOS by redesigning patient flow. Therefore, there is a lack of evidence that QI approaches targeting specific aspects of healthcare quality and efficiency result in a more generalized reduction in LOS.

Our study has several limitations. First, the intervention was conducted at one surgical unit in one VAMC and may not be generalizable to other hospitals or healthcare systems. Second, the intervention was intentionally designed to be an ongoing iterative process to address new problems as they were identified. Consequently, we defined the time frames (beginning of 2004 for antibiotic prophylaxis and 2003 for LOS) for a comparative analysis retrospectively based on notes and progress reports maintained by the team leader and project coordinator (EEM). Although useful for tracking program implementation, these data were not collected with the scientific rigor of a research study, and we did not have specific enough details on the exact time line of the implementation of all components of the perioperative antibiotic bundle or intervention to conduct the analyses across shorter intervals (ie, on a monthly or quarterly basis). Third, the study involves a before-andafter design without additional external controls for the perioperative antibiotic use end point, making it difficult to control for contemporaneous events or secular changes arising independent of the study intervention. Fourth, given the administrative nature of our data, the primary efficiency outcome was LOS rather than more robust outcome measures such as resource use for an episode of care or recuperation time.

This study adds to the growing body of literature demonstrating the effectiveness of a TPS approach to improve quality in a healthcare setting. Implementation of such an approach on a surgical unit was associated with higher rates of appropriate perioperative antibiotic therapy but not reductions in LOS. Although TPS strategies have the potential to improve quality and efficiency of healthcare, it is important when measuring the effectiveness of such an intervention to prospectively collect information about the  implementation process and to directly measure all outcomes that were targeted as part of the intervention.

Use of Toyota production system methods as part of a nosocomial MRSA prevention initiative on a surgical unit improved quality of care in other areas.


Rajiv Jain, MD

Articles

Impact of a National Specialty E-Consultation Implementation Project on Access

Toyota Production System Quality Improvement Initiative Improves Perioperative Antibiotic Therapy