Currently Viewing:
The American Journal of Managed Care October 2014
Quality of Care at Retail Clinics for 3 Common Conditions
William H. Shrank, MD, MSHS; Alexis A. Krumme, MS; Angela Y. Tong, MS; Claire M. Spettell, PhD; Olga S. Matlin, PhD; Andrew Sussman, MD; Troyen A. Brennan, MD, JD; and Niteesh K. Choudhry, MD, PhD
A Comprehensive Hospital-Based Intervention to Reduce Readmissions for Chronically Ill Patients: A Randomized Controlled Trial
Ariel Linden, DrPH; and Susan W. Butterworth, PhD
Physician Compensation Strategies and Quality of Care for Medicare Beneficiaries
Bruce E. Landon, MD, MBA; A. James O'Malley, PhD; M. Richard McKellar, BA; James D. Reschovsky, PhD; and Jack Hadley, PhD
Increasing Access to Specialty Care: Patient Discharges From a Gastroenterology Clinic
Delphine S. Tuot, MDCM, MAS; Justin L. Sewell, MD, MPH; Lukejohn Day, MD; Kiren Leeds, BA; and Alice Hm Chen, MD, MPH
Increasing Preventive Health Services via Tailored Health Communications
Kathleen T. Durant, PhD; Jack Newsom, ScD; Elizabeth Rubin, MPA; Jan Berger, MD, MJ; and Glenn Pomerantz, MD
The Duration of Office Visits in the United States, 1993 to 2010
Meredith K. Shaw; Scott A. Davis, MA; Alan B. Fleischer, Jr, MD; and Steven R. Feldman, MD, PhD
Evaluation of Collaborative Therapy Review to Improve Care of Heart Failure Patients
Harleen Singh, PharmD; Jessina C. McGregor, PhD; Sarah J. Nigro, PharmD; Amy Higginson, BS; and Greg C. Larsen, MD
Extending the 5Cs: The Health Plan Tobacco Cessation Index
Victor Olaolu Kolade, MD
Caregiver Presence and Patient Completion of a Transitional Care Intervention
Gary Epstein-Lubow, MD; Rosa R. Baier, MPH; Kristen Butterfield, MPH; Rebekah Gardner, MD; Elizabeth Babalola, BA; Eric A. Coleman, MD, MPH; and Stefan Gravenstein, MD, MPH
Ninety-Day Readmission Risks, Rates, and Costs After Common Vascular Surgeries
Eleftherios S. Xenos, MD, PhD; Jessica A. Lyden, BSc; Ryan L. Korosec, MBA, CPA; and Daniel L. Davenport, PhD
Currently Reading
Using Electronic Health Record Clinical Decision Support Is Associated With Improved Quality of Care
Rebecca G. Mishuris, MD, MS; Jeffrey A. Linder, MD, MPH; David W. Bates, MD, MSc; and Asaf Bitton, MD, MPH
Healthcare Utilization and Diabetes Management Programs: Indiana 2006-2010
Tilicia L. Mayo-Gamble, MA, MPH; and Hsien-Chang Lin, PhD
Predictors of High-Risk Prescribing Among Elderly Medicare Advantage Beneficiaries
Alicia L. Cooper, MPH, PhD; David D. Dore, PharmD, PhD; Lewis E. Kazis, ScD; Vincent Mor, PhD; and Amal N. Trivedi, MD, MPH

Using Electronic Health Record Clinical Decision Support Is Associated With Improved Quality of Care

Rebecca G. Mishuris, MD, MS; Jeffrey A. Linder, MD, MPH; David W. Bates, MD, MSc; and Asaf Bitton, MD, MPH
Using EHR clinical decision support is associated with improved quality of care. Most primary care practices are missing at least 1 "meaningful use" clinical decision support module.
To determine whether clinical decision support (CDS) is associated with improved quality indicators and whether disabling CDS negatively affects these.

Study Design/Methods
Using the 2006-2009 National Ambulatory and National Hospital Ambulatory Medical Care Surveys, we performed logistic regression to analyze adult primary care visits for the association between the use of CDS (problem lists, preventive care reminders, lab results, lab range notifications, and drug-drug interaction warnings) and quality measures (blood pressure control, cancer screening, health education, influenza vaccination, and visits related to adverse drug events).

There were an estimated 900 million outpatient primary care visits to clinics with EHRs from 2006-2009; 97% involved CDS, 77% were missing at least 1 CDS, and 15% had at least 1 CDS disabled. The presence of CDS was associated with improved blood pressure control (86% vs 82%; OR 1.3; 95% CI, 1.1-1.5) and more visits not related to adverse drug events (99.9% vs 99.8%; OR 3.0; 95% CI, 1.3-7.3); these associations were also present when comparing practices with CDS against practices that had disabled CDS. Electronic problem lists were associated with increased odds of having a visit with controlled blood pressure (86% vs 80%; OR 1.4; 95% CI, 1.3-1.6). Lab result notification was associated with increased odds of ordering cancer screening (15% vs 10%; OR 1.5; 95% CI, 1.03-2.2).

The use of CDS was associated with improvement in some quality indicators. Not having at least 1 CDS was common; disabling CDS was infrequent. This suggests that meaningful use standards may improve national quality indicators and health outcomes, once fully implemented.

Am J Manag Care. 2014;20(10):e445-e452
  • The use of some electronic health record clinical decision support (CDS) was associated with improved quality of care among national primary care outpatient visits from 2006 to 2009.
  • Most practices were missing at least 1 CDS module considered essential by meaningful use standards.
  • Meaningful use standards may have a significant impact on national quality of care and health outcomes once fully implemented.
The CMS Electronic Health Record Incentive Program, better known as “meaningful use,” provides initial monetary incentives for adoption of electronic health records (EHRs) and subsequently imposes monetary penalties if physician practices do not meet requirements. The meaningful use program has spurred a significant rise in the number of office-based physicians reporting EHR use, from 24% in 2005 to 48% in 2009 to 72% in 2012.1

The meaningful use goals and objectives are divided into 3 stages. Stage 1 requirements include using a certified EHR to capture and share patient data by 2015; stage 2, beginning in 2014, includes use of advanced care processes with clinical decision support (CDS); and stage 3, to begin in 2016, will require providers to demonstrate outcome improvements. The time frame for stages 2 and 3 was recently extended.2 As of March 2013, 73% of eligible providers had registered to participate in the meaningful use incentive program and 44% of eligible providers had received incentive payment for meeting meaningful use objectives.3

Although the meaningful use requirements have already been established, the evidence is inconsistent regarding improvement in healthcare processes or patient outcomes as a result of the implementation of general and individual EHR components. Prior studies have shown that EHR-based CDS is associated with improved prescribing safety,4 preventive care measures,5 and diabetes testing and control.6 CDS has also been associated with some improvements in quality indicators,7 but results have been variable.8-11 Despite having EHRs, many physicians report being unable to complete basic panel management activities,12 which affects their ability to deliver high-quality care for patients with chronic conditions.

Notwithstanding the mandate to use certified EHRs, use of an EHR in the outpatient setting has not yet been shown unequivocally to improve quality indicators across a broad range of practices and EHRs, especially because there is substantial variability among the 1600 certified EHR platforms. 13 In addition to the large number of EHRs, they also can be customized to each practice’s needs and resource availability, so results will vary between practices in terms of impact on care even with the use of a single vendor’s practice. Certain types of practices, such as smaller community-based practices, may be less able to fully implement a complete EHR with clinical decision support, which is the kind postulated to have the greatest potential to improve quality.14,15 Decisions about choices around the CDS functions implemented may affect the quality of care delivered by that practice. To further inform the potential effect of meaningful use stages 2 and 3, we examined the association between having CDS, not having CDS, and disabling CDS, and quality of care indicators using nationally representative data just prior to the start of the meaningful use incentive program.


We performed a retrospective, cross-sectional analysis of adult primary care ambulatory clinic visits in the National Ambulatory Medical Care Survey (NAMCS) and National Hospital Ambulatory Medical Care Survey (NHAMCS) outpatient department records from 2006-2009 in order to analyze data with little impact from current meaningful use incentive payments. We examined a number of patient and practice characteristics to determine any differences in access to practices with EHRs based on prior work showing associations between these characteristics and EHR availability.8,14,16 On the premise that most preventive care takes place through a primary care practice, we limited our sample to just those practices.

Data Source

The NAMCS and NHAMCS are conducted by the Ambulatory Care Statistics Branch of the National Center for Health Statistics (NCHS) of the CDC. These surveys focus on outpatient visits to non–federally funded US medical practices.

The NCHS uses a complex multistage probability sampling design based first on geography, then physician specialty, and finally patient visits to individual practices. Information is collected at ambulatory visits by participating practices or US Census Bureau representatives. The NCHS assigned weights to each visit record to allow for estimation of national statistics. The NCHS institutional review board approves the protocols for the NAMCS and NHAMCS, including a waiver of informed consent for patients.

The NAMCS collects information at a visit level from community-based physician practices; from 2006 to 2009, approximately 30,000 community-based office visits were recorded annually. The NHAMCS collects information at a visit level from hospital-based physician practices; from 2006 to 2009, approximately 34,000 hospital-based office visits were recorded annually.

In all years included in the analysis, both the NAMCS and NHAMCS included questions in their intake surveys that asked whether the practice had an electronic medical record (for other than billing purposes), a computerized patient problem list, computerized warnings of drug interactions or contraindications, a computerized system to view lab results, a computerized system that highlights out-of-range lab values, and a computerized system for reminders for guideline-based interventions and/or screening tests. Response variables indicated that a practice had a function (“yes”), did not have a function (“no”), had the function but disabled it (“turned off”), or was unsure if the practice had the function (“don’t know” or “blank”).

The NAMCS and NHAMCS record health information for each visit sampled. The respondent indicates the top 3 reasons for the visit, which are then coded by NCHS; the patient’s vital signs, including blood pressure; whether any health education was ordered or provided during the visit; and whether an influenza vaccination was ordered or administered during the visit. The respondent also indicates any screening services ordered or performed during the visit. The respondent can record mammography, Pap smears, and scope procedures (NCHS records and codifiers, free text scope procedures, 1 of which is sigmoidoscopy/ colonoscopy), among others.

The NAMCS and NHAMCS collect patient and practice demographic information: patient age, patient race, patient income category (imputed based on patient zip code by NCHS), primary reason for the visit, region of the country, practice ownership, and number of physicians in the practice (solo or non-solo). Race data are collected in the manner already in use by the practice. Due to high nonresponse rates for race (24% to 33% in NAMCS, and 12% to 15% in NHAMCS for the years included), the NCHS also provides a race variable imputed based first on the patient’s location, then on physician specialty and International Classification of Diseases, Ninth Revision, Clinical Modification code for primary diagnosis, and finally on a random basis.

Data Analysis

Our predictors of interest were EHR CDS modules mandated by stage 1 of meaningful use: electronic problem lists, preventive care reminders, lab result reporting, out-of-range lab notification, and drug-drug interaction warnings. We performed analyses with each predictor separately, and performed analyses combining them into a composite group of key EHR features. Based on prior work that examined associations between certain EHR features and clinical outcomes,4,5,9,11,17,18 we developed a matrix between CDS functions and the clinical outcomes of interest and performed analyses accordingly (Table 1).

Our main outcomes of interest were based on national quality metrics related to adult primary care: blood pressure control (defined as a systolic blood pressure less than 140 mm Hg), age- and gender-appropriate cancer screening (mammography, Pap smear, and sigmoidoscopy or colonoscopy), health education for particular conditions, influenza vaccination during the months of October through March, and adverse drug events (as measured by visits coded to be related to an adverse drug event). We also combined the receipt of cancer screening, health education, and influenza vaccination into a composite “preventive care” outcome measure.

We were interested in the difference in outcomes based on whether a practice had a CDS function or not. Specifically, we examined whether there were significant differences in the rates of the clinical outcomes depending on whether a visit was to a practice with the CDS or without the CDS.

We categorized practices into 3 groups: all CDS tools active; without 1 or more CDS function; or any disabled CDS. We considered the practice to have all CDS active if they answered “yes” to having all 5 of the individual CDS. We considered the practice to be without 1 or more CDS functions if they answered “no,” “turned off,” don’t know,” or “blank” to any of the CDS functions. We considered the practice to have “disabled CDS” if they answered “turned off” for any of the CDS functions.

We collapsed practice ownership into 5 categories: physician or physician group, community health center, health maintenance organization, medical or academic health center, and other. All other covariates were used as provided by NCHS.

Statistical Analysis

The unit of analysis was the patient visit. We followed NCHS guidelines based on the hierarchical sampling design. We performed all statistical tests on data with less than a 30% relative standard error (the standard error divided by the estimate itself) and more than 30 sample records.

We used multivariable logistic regression modeling to control for a number of patient and practice level covariates. We modeled age as a continuous variable, and all other covariates as categorical variables. We evaluated categorical variables with the χ2 test and continuous variables with the t test. We performed multivariable logistic regression modeling for each quality measure individually. The main predictors of interest were the status of the EHR CDS function. We considered P values of less than .05 to be significant.

We used SAS statistical software (version 9.3; SAS Institute, Cary, North Carolina) for all analyses.


From 2006-2009, the NAMCS/NHAMCS databases included 104,102 adult primary care visits, representing 2 billion adult primary care visits in the United States. Of these, 45% of visits were to practices with EHRs. Of those visits, 97% also had at least 1 of the 5 CDS functions of interest. Of visits using EHRs with at least 1 CDS function, 77% did not have (for any reason), and 15% had disabled (actively turned off), at least 1 of the 5 CDS functions of interest.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up