The increasingly widespread adoption of electronic health records (EHRs) is substantially changing the American healthcare delivery system. Differences in the actual effectiveness of EHRs and their component applications, including electronic prescribing (e-prescribing), is not well understood. We compared the effects of 2 types of e-prescribing systems on medication safety as an example of how comparative effectiveness research (CER) can be applied to the study of healthcare delivery.

We previously conducted 2 non-randomized, prospective studies with pre—post controls comparing prescribing errors among: (1) providers who adopted a standalone e-prescribing system with robust technical and clinical decision support (CDS) and (2) providers who adopted an EHR with integrated e-prescribing with less robust available CDS and technical support. Both studies evaluated small groups of ambulatory care providers in the same New York community using identical methodology including prescription and chart reviews. We undertook this comparative effectiveness study to directly compare prescribing error rates among the 2 groups of e-prescribing adopters.

 The stand-alone system reduced error rates from 42.5 to 6.6 errors per 100 prescriptions (

 <.001). The integrated system reduced error rates from 26.0 to 16.0 per 100 prescriptions (

= .07). After adjusting for baseline differences, stand-alone users had a 4-fold lower rate of errors at 1 year (

 Despite improved work flow integration, the integrated e-prescribing application performed less well, likely due to differences in available CDS and technical resources. Results from this small study highlight the importance of CER that directly compares components of healthcare delivery.

(Am J Manag Care. 2011;17(12 Spec No.):SP88-SP94)

	Comparative effectiveness research (CER) can be expanded beyond its usual focus on treatment or intervention options to evaluate healthcare delivery systems, including electronic health records (EHRs) and electronic prescribing (e-prescribing).

		 Our small CER study of ambulatory providers found that use of a stand-alone e-prescribing application led to a greater reduction in prescribing errors than use of an integrated EHR with e-prescribing.

		 System features, as well as implementation and training resources, likely contributed to these findings.

		 Given the large national investment in EHRs, future CER research should be conducted to understand the actual effects of EHR systems in active use.

	The Health Information Technology for Economic and Clinical Health (HITECH) Act is providing up to $30 billion for the use of interoperable electronic health records (EHRs) in meaningful ways that improve healthcare quality, safety, and efficiency.

Specific measures will be used to determine if a provider meets criteria for incentives, including electronic prescribing (e-prescribing).

 To decrease intersystem variability, providers must use a certified EHR, including eprescribing that meets certain standards. However, even among certified EHRs, differences in performance arise in actual system use.

	Comparative effectiveness research (CER) is designed to inform healthcare decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options and interventions.

Traditionally, CER has directly compared the outcomes of 2 therapeutic interventions, most commonly medications, using randomized controlled trials. To date, there has been little CER evaluating different health information technology (HIT) in actual use.

	Previously, using identical methodology, we conducted 2 separate prospective studies comparing the effects of 2 types of e-prescribing systems on medication safety.

The systems were implemented and used by ambulatory providers in a single community but had important differences. The first was a stand-alone system with advanced clinical decision support (CDS) and extensive technical support. The second was an e-prescribing application integrated within an EHR with less robust available CDS and more limited technical support. These studies provided an ideal opportunity to perform a novel comparative effectiveness study of these 2 system types. Our hypothesis was that despite technical and CDS limitations, the integrated application would better improve medication safety due to improved work flow integration and increased diversity of patient data from the EHR available to the prescriber at the point of care.

	In this study, we compare the effectiveness of a commercially available stand-alone e-prescribing system to an e-prescribing system integrated within an EHR by analyzing paper prescriptions at baseline and e-prescriptions 1 year later for 21 ambulatory care providers in the same New York community. The data were obtained from 2 previous pre—post studies evaluating prescribing safety among e-prescribing adopters compared with control providers who used handwritten prescriptions. 

 The same group of 15 non-adopters served as concurrent controls for both original studies and, in each case, prescribing errors were high at baseline for all study participants but were significantly higher for non-adopters at 1 year. We obtained institutional review board approval from Weill Cornell Medical College, and providers gave written informed consent.

	The Institute of Medicine classifies medication errors as any error in the medication use process (prescribing, transcribing, dispensing, administering, and monitoring).

We focused only on prescribing errors, such as omitting quantity to be dispensed. Near misses were prescribing errors with potential for harm that were either intercepted or reached the patient but did not cause harm. An example was prescribing penicillin for a patient with a known allergy who did not receive the medication because of pharmacist error detection. Adverse drug events (ADEs) were injuries from a medication, a subset of which was preventable. Rule violations were departures from strict standards of prescribing that were well understood and unlikely to cause harm, such as failure to write “po” for a medication only taken orally. These were not included in error rates but were counted, as they can result in significant rework.

	We studied 11 adult primary care practices in a predominantly rural and suburban region of New York State between September 2005 and July 2008. Fifteen providers from 6 different practices adopted a stand-alone system and 6 providers from 5 practices adopted an integrated system. Physicians were members of the not-for-profit independent practice association. All members were sent a letter in May 2005 detailing incentives for adopting e-prescribing and inviting them to participate in a research study. Discounts on EHR licenses were provided as an incentive. All practices ranged in size from 1 to 7 providers and none were academically affiliated.

	The stand-alone system was a Web-based, commercially available system. Providers had access to an electronic reference guide for dosing recommendations, medication lists, and allergies, if this information had been entered. The system provided CDS alerts for drug allergies, drug—drug interactions, duplicate drug therapies, incorrect drug frequencies, incorrect dosing, and pregnancy and breast-feeding contraindications. The system checked for insurance eligibility and formulary compliance. Prescriptions could be sent to pharmacies electronically. This system was ultimately not certified, as it was not integrated within an EHR. Providers performed other clinical documentation on paper.

E-prescribing System Integrated Within an EHR

	The integrated system was a commercially available system fully integrated within an EHR. Providers had electronic access to all information in the EHR including patient history, medications, allergies, diagnoses, and laboratory and demographic data. The system provided the same types of alerts as the stand-alone system but additionally provided drug—disease interactions and disease-specific drug recommendations. Patient insurance eligibility and formulary compliance were also checked and prescriptions could be sent electronically to pharmacies. Notably, the e-prescribing module was a more immature module due to limited configuration of the CDS at the time of the study. CDS for e-prescribing was fully configured after the study’s completion.

	A for-profit Health Information Service Provider (HSP) provided implementation and ongoing technical support, including routine monitoring of e-prescribing compliance to encourage 100% use.8 Because the HSP was new to providing implementation services at the time the integrated practices went live, those providers received far less training initially (on average 1 hour) compared with stand-alone users (on average 40 hours), who went live later.

 We collected carbon copies at baseline and electronic downloads of all prescriptions written by providers during a 2-week period at 1 year. During both time periods, we obtained a minimum of 75 prescriptions on at least 25 patients per provider, extending data collection if necessary, and limited review to 3 prescriptions per patient to minimize clustering of errors. Non-duplicate prescription pads were removed at baseline to ensure compliance with use of the duplicate prescription pads.

 A research nurse reviewed each prescription in an identical manner guided by extensively utilized, standardized methodology.

Training included review of error definitions, legibility assessments, and review of test and actual cases. Two data collectors jointly reviewed cases, after which reviewers worked independently. Reviewers classified prescribing errors, and rule violations, and evaluated the use of ADE-trigger drugs. Inappropriate abbreviation errors were from the Joint Commission on Accreditation of Healthcare Organizations’ “Do Not Use” list, established to denote abbreviations with great potential to cause medical errors.

	We determined interrater reliability by having 2 reviewers examine the same random sample of 2% of the data. Interrater agreement for overall and type of error was 1.0, indicating excellent agreement.

An ambulatory chart review was performed for suspected near misses or when a drug often used to treat an ADE was prescribed.

Two physicians blinded to the providers’ prescribing method independently reviewed all suspected near misses and ADEs. Interrater agreement for the presence of prescribing errors and near misses was 0.96 and 0.93, indicating excellent agreement.

 We compared error rates per 100 prescriptions of (1) stand-alone and integrated adopters at baseline, (2) stand-alone adopters at baseline and 1 year, (3) integrated adopters at baseline and 1 year, and (4) between stand-alone and integrated adopters at 1 year using mixedeffects Poisson regression models that included e-prescribing system type, study time, and an interaction term system type and study time. We adjusted for clustering at the provider level and assumed an independent correlation structure for these Poisson models. We calculated 95% Poisson confidence intervals (CIs) with cluster robust standard errors for the rates. We used SAS for PC version 9.2 (SAS Institute Inc, Cary, North Carolina) to estimate Kappa statistics, 

 tests, and Stata 11 (StataCorp, College Station, Texas) to

	estimate mixed-effects Poisson regression models and to calculate 95% Poisson CIs with clustered robust standard errors.

	There was no significant difference between provider groups (

). All physicians were board certified in internal or family medicine. The mean number of prescriptions written per provider and patients seen per provider were similar at baseline (0.65 and 0.26, respectively). At 1 year, integrated adopters saw significantly more patients and wrote significantly more prescriptions during the study period (0.002 and 0.002, respectively).

	For stand-alone adopters, 1273 unique patients seen at baseline and 1598 unique patients seen at 1 year received prescriptions (

). For integrated adopters, 481 unique patients seen at baseline and 368 unique patients seen at 1 year received prescriptions. At baseline, patients of stand-alone adopters were significantly older (56 vs 53 years, 

 <.001) and at 1 year more were female (60% vs 54%, 

	We reviewed 2550 prescriptions at baseline, 1901 of which were written by stand-alone adopters and 649 of which were from integrated adopters (

	Rates of baseline prescribing errors for both groups were high, with planned adopters of integrated e-prescribing 1.6- fold less likely to have prescribing errors (

 = .03). Rule violations were also high at baseline for both groups, but with no statistical differences. Near misses were detected infrequently and without statistical differences.

Rates of Errors for Stand-alone E-prescribing Adopters at Baseline and 1 Year

Articles

SPECIAL ISSUE

This is a comparative effectiveness study that evaluates the safety effects of 2 types of commercially available electronic prescribing systems.

The Comparative Effectiveness of 2 Electronic Prescribing Systems

To characterize consumers’ attitudes regarding the perceived benefits of electronic health information exchange (HIE), potential HIE privacy and security concerns, and to analyze the intersection of these concerns with perceived benefits.

A random-digit-dial telephone survey of English-speaking adults was conducted in 2010. Multivariate logistic regression models examined the association between consumer characteristics and concerns related to the security of electronic health records (EHRs) and HIE.

A majority of the 1847 respondents reported they were either “very” or “somewhat” concerned about privacy of HIE (70%), security of HIE (75%), or security of EHRs (82%). Concerns were significantly higher (P <.05) among employed individuals 40 to 64 years old and minorities. Many believed that HIE would confer benefits such as improved coordination of care (89%). Overall, 75% agreed that the benefits of EHRs outweighed risks to privacy and security, and 60% would permit HIE for treatment purposes even if the physician might not be able to protect their privacy all of the time. Over half (52%) wanted to choose which providers access and share their data.

Greater participation by consumers in determining how HIE takes place could engender a higher degree of trust among all demographic groups, regardless of their varying levels of privacy and security concerns. Addressing the specific privacy and security concerns of minorities, individuals 40 to 64 years old, and employed individuals will be critical to ensuring widespread consumer participation in HIE.

(Am J Manag Care. 2011;17(12 Spec No.):SP111-SP116)

A national survey of adults regarding electronic health information exchange (HIE) found that greater participation by consumers in determining how HIE takes place could engender a higher degree of trust among all demographic groups, regardless of their varying levels of privacy and security concerns.

 Statistically significant variations in concerns about the security of electronic health records and HIE were found among employed individuals 40 to 64 years old, and minorities, indicating that HIE policies need to account for demographic and social factors in order to encourage widespread participation.

Consumers desire greater transparency and control over who accesses their personal information.

Electronic health information exchange (HIE), the electronic sharing of health information among healthcare providers, patients, and other entities, promises an array of potential benefits for healthcare consumers through improved quality, safety, and efficiency.

 Health information exchange may also provide consumers with new, more effective, methods for engaging with their healthcare providers to view and manage their health information. In February 2009, the Health Information Technology for Economic and Clinical Health (HITECH) Act was passed, which included substantial funding to support the nationwide adoption of health information technology (health IT) and HIE.

With the benefits of HIE come challenges and opportunities for keeping health information private and secure. To better understand these challenges and opportunities the Office of the National Coordinator for Health Information Technology (ONC) and the Agency for Healthcare Research and Quality supported the Health Information Security and Privacy Collaboration (HISPC) between 2006 and 2009. The HISPC was the first coordinated nationwide effort established to assess and address the effect of variations in organization-level business practices, policies, and state laws governing the privacy and security of electronic health information on nationwide electronic HIE.

High levels of consumer concern about privacy and security may be a barrier to widespread participation in HIE.

 Yet, emerging studies also indicate that consumers recognize that there are benefits to the electronic collection, storage, and sharing of their health information, such as better coordination of care.

What remains unclear is how consumer engagement and participation in HIE is affected by consumers’ perceptions of the risks and benefits of HIE.

Understanding the specific privacy and security concerns of consumers, and assessing their preferences for specific safeguards, could help inform the further development of privacy and security policies and other measures that will be critical to the overall implementation of the HITECH Act This study sought to examine the reasons underlying consumer concerns and how the perceived risks and benefits impact decisions about sharing their health information. In addition, we examined the types of safeguards that could help address these concerns. We also sought to characterize consumers who expressed greater concerns and, therefore, may be less likely to participate in HIE.

A random-digit-dial (RDD) telephone survey was conducted, in English, among adults 18 years or older residing in the United States between August and November of 2010. The sample consisted of 12,795 cell phone—only households and 15,036 landline households. This was intended to be an exploratory study, and we did not oversample for minority subgroups or conduct the survey in Spanish.

The methods for weighting cell phone and landline RDD studies are still in their infancy. We followed a methodology proposed by Brick

and the guidelines in a report from the American Association for Public Opinion Research (AAPOR) Cell Phone Task Force to weigh the data.

 We created base weights using the initial probabilities of selection and adjusted to account for nonresponse using a weightclass approach based on region. We then used a composite estimator to adjust the weights to account for dual, overlapping sample frames and compared weight sums to control totals from the 2009 National Health Interview Survey for post-stratification.

The survey methodology was approved by an Institutional Review Board at RTI International.

) sought to examine consumers’ detailed preferences and attitudes related to privacy and security of HIE. The survey domains included: 1) demographic information; 2) current computer and Internet usage; 3) experience with the healthcare system and general knowledge about health information; 4) general attitudes toward HIE; 5) specific questions about the privacy and security of HIE, and awareness of regulations governing privacy and security; and 6) safeguards that might address these concerns. The questionnaire items underwent multiple rounds of cognitive testing for concept clarity to ensure a general audience could understand them. The survey was then programmed as a computer-assisted telephone interview instrument.

We used descriptive statistics to characterize respondents and attitudes. We also examined bivariate associations between consumer characteristics and attitudes toward privacy and security of HIE and electronic health records (EHRs). For categorical and dichotomous variables we assessed these bivariate associations using 

 test, or Fisher exact test where appropriate. Tests were 2-sided and Bonferroni corrections applied to multiple comparisons.

We ran logistic regression models using an iterative, backward, stepwise selection procedure to examine the association between levels of concern about the security of EHR and security of HIE with the following variables: age (18-39, 40-64, 65 ); race (white, black, Hispanic, other); gender; education (>high school vs >3 visits or 

2). We used SAS version 9.2 (SAS Institute Inc, Cary, North Carolina) and SUDAAN version 10.0.1 (RTI International, Research Triangle Park, North Carolina) for analyses.

Of the 1847 respondents, 68% (n = 1261) were from the landline phone household sample. Approximately 40% (n = 8297) of the sample was ineligible; most were nonworking numbers, businesses, or fax/modems. Among those that were eligible, 6607 (55%) refused to participate. Additionally, some households (n = 3463) did not answer the phone despite 15 callbacks, while others had language barriers or other impairments (n = 472) that prevented them from completing the survey.

Twenty-five percent of the respondents were between 18 and 39 years old, 46% were between 40 and 64 years old, and 29% were 65 years or older (

). According to 2009 US Census data, these age groups are 41%, 43%, and 16%, respectively. A majority of respondents were white (60% vs 72% US) or female (59%). Almost three-quarters (73% vs 84.6% US) of respondents attended high school or beyond.

 Half reported they were either unemployed, retired, or a student.

Concerns About the Security of EHR and HIE

Of all respondents, 82% reported that they were “very” (59%) or “somewhat” concerned (23%) about the security of EHR. Additionally, 75% of respondents reported being “very” (45%) or “somewhat” concerned (30%) about the security of HIE. Security was described as “the protections (policies and technologies) used to keep unauthorized individuals from being able to access and view your electronic health information.” Most of the reasons underlying consumers’ concerns about the security of HIE related to the potential misuse of their personal health information for fraud or identity theft (93%), posting of personal data on the Internet (90%), the receipt of unsolicited advertising and junk mail (88%), or discrimination (77%). Another reason for consumers’ concern regarding the security of HIE was the potential loss of their personal health information (82%).

Consumers With Greater Concern Regarding the Security of EHR

Multivariate analyses regarding concerns about the security of EHR are significantly higher (

 <.05) among respondents between the ages of 40 and 64 years (odds ratio [OR] 2.1, 95% confidence interval [CI] 1.5-2.9), 65 years and older (OR 1.6, CI 1.1-2.3), and full-time employed individuals (OR 1.5, CI 1.1-2.0). Race (

 = .0684) and frequency of physician visits (

 = .1092) approached significance. Consumers With Greater Concerns Regarding the Security of HIE Multivariate analyses show that concerns regarding the security of HIE are significantly higher (

 <.01) among respondents between the ages of 40 and 64 years (OR 1.6, 95% CI 1.1-2.1), full-time employees (OR 1.7, 95% CI 1.3-2.4), and minorities including black (OR 1.6, 95% CI 1.0-2.3), Hispanic (OR 1.7, 95% CI 1.0-2.7), and other minorities (OR 1.7, 95% CI 1.1-2.6).

Nearly 70% of respondents were either “very” (35%) or “somewhat” (35%) concerned about the privacy of HIE. Concern about privacy was described as the concern that an unauthorized individual would gain access to or view their personal health information without their permission. The most frequent reasons underlying consumer privacy concerns were being denied credit (58%) or employment (54%). When asked about how 

 certain events would occur if their health information was accessed by someone that did not have permission, the majority reported that it was “very” or “somewhat” likely that identity theft (74%) or fraud (65%) would occur, and about half of the respondents reported it was at least “somewhat” likely that they would be discriminated against (49%).

Consumers reported potential risks and benefits associated with HIE more generally. In addition to concerns regarding access by unauthorized individuals, a majority of respondents (75%) reported concerns that it was at least “somewhat likely” that their information would be accidentally linked to the wrong person or accidentally released to the wrong physician (65%).

Yet, respondents also indicated that HIE would have benefits. They thought HIE was “very likely” or “somewhat likely” to help doctors better coordinate care (89%), reduce the number of medical tests needed (81%), improve medical care (78%), and improve quality of care (76%).

When asked which of these potential risks or benefits was the most important, consumers reported that benefits were most important. These included the ability of HIE to better coordinate care (23%), improve quality of care for the country (16%), and for themselves personally (15%).

We also asked about the tradeoffs associated with privacy and security. Nearly 75% of respondents agreed that an EHR has benefits that outweigh any risk to privacy and security. Even among respondents who reported being “somewhat” or “very” concerned, a significant majority agreed that the benefits of HIE outweigh the risks (

Sixty percent of respondents reported they would be willing to allow physicians to share their information electronically for treatment, even if they may not be able to protect their privacy all of the time. Approximately half of those who reported they were “very” concerned about privacy or security of EHR and HIE would permit their physician to share their information for treatment even if they may not be able to protect privacy all of the time (Table). A significantly higher proportion of individuals who reported they were “somewhat” concerned would allow their physician to do so (

Consumer Preferences Regarding Who Should Be Involved in Determining Privacy Settings for HIE

Most individuals desired to have their physicians involved in determining their privacy settings. Over a third (39%) preferred that they, their physicians, and their family determine their privacy setting. A little over a quarter (26%) indicated that they and their physician could determine this. Few individuals wanted to have the sole responsibility (18.5%) or have their physician (4.8%) have the sole responsibility in determining their privacy settings.

Over three-quarters (79%) indicated they would want a physician to override their privacy settings in a medical emergency in order to treat them.

Consumer Preferences Regarding Setting Limits on Who May Access Their Health Information

Only about a third of respondents indicated that it was important to limit access to their health information by family (39%) or physicians (32.9%) involved in their healthcare. However, most indicated that it was “somewhat” or “very” important to be able to limit access among physicians not involved in their care (69.1%), friends (68.7%), employers (67.6%), and payers (67.1%).

Permission would be granted by 90% of respondents to electronically share their health information with at least some healthcare providers. Over half (52%) reported they preferred limiting access specifically to chosen providers, whereas 38% indicated they would permit access to any provider.

Most respondents (71%) reported they would be “somewhat” or “very” likely to request a list of disclosures their physician made through an EHR about treatment, payment, and other healthcare purposes. Among those requesting disclosure, nearly all responders wanted to know what was disclosed (95%), who disclosed it (95%), to whom it was disclosed (97%), and why it was disclosed (97%).

About half (49%) of respondents reported that they would be willing to share all of their health information with their healthcare provider, less than a third (32%) would share some of it, and about 12% would refuse to share any information. Respondents who were very concerned about privacy of HIE or security of EHR or HIE were significantly more likely to prefer limiting data sharing to either physicians of their choosing or the treating physician, and not sharing data with any physician, in comparison to those who expressed fewer concerns (

Overall, we found that consumers remain concerned about privacy and security, and these concerns are greater among those aged 40 to 64 years, employed individuals, and minorities. Overall consumer concerns related to HIE focused on unauthorized access, and the potential misuse of their information for fraud and by other parties for denying credit or employment. Despite these concerns, most consumers recognize the benefits of HIE and many believe that the benefits outweigh privacy and security concerns. Although most consumers would permit HIE among at least some of their healthcare providers, about half wanted to choose which providers could access and share their data, and a majority wanted to limit access among those not directly involved with their treatment.

The reasons underlying consumer concerns suggest the need to improve individuals’ engagement in decisions about who determines access to their health information and the need to continue to improve consumer trust in the technology and the healthcare system. Education about data security, consumer rights, and recourse in cases of unauthorized disclosures and misuse of data may help address the reasons underlying consumer concerns and build trust in the use of electronic information.

Greater consumer participation in determining how HIE occurs could engender greater trust among all demographic groups, regardless of varying levels of privacy and security concerns.

Public Attitudes Toward Health Information Exchange: Perceived Benefits and Concerns

 To assess the impact of an electronic medical record (EMR) with clinical decision support (CDS) and performance feedback on provider adherence to guideline-recommended care and blood pressure (BP) control compared with a standard EMR alone.

Quasi-experimental with repeated measures.

The study was conducted in a 4-site, federally qualified health center, Open Door Family Medical Centers, located in New York. The research team, Open Door leadership, providers, and staff developed and implemented a tailored multicomponent CDS system, which included a BP alert, a hypertension (HTN) order set, an HTN template, and clinical reminders. We extracted patient-level data for each encounter 17 months prior to implementation of the intervention (June 2007-October 2008) and 15 months post-intervention (April 2009-June 2010), from the EMR’s data tables for all adult nonobstetric patients with a diagnosis of HTN (N = 3636).

Rates of HTN control were significantly greater in the post-intervention period compared with the baseline period (50.9% vs 60.8%; 

 <.001). Process measures, derived from the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure Guidelines, also improved significantly. Logistic regression with generalized estimating equations showed that patients were 1.5 times more likely to have controlled BP post-intervention than pre-intervention. Correlates of poor BP control were black race, higher body mass index, diabetes, female gender, income, and a greater number of prescribed antihypertensive medications.

Our findings suggest that health information technology that is implemented as part of a multicomponent quality improvement initiative can lead to improvements in HTN care and outcomes.

(Am J Manag Care. 2011;17(12 Spec No.):SP103-SP110)

An analysis of a multifaceted, technology-driven intervention to improve hypertension control in community health centers resulted in the following key findings:

 A theory-driven approach to tailoring health information technology (HIT) to local context, through user input and an iterative testing process, can facilitate its adoption.

 Clinical decision support systems should incorporate features that have been shown to affect quality of care.

 When implemented as part of a multifaceted quality improvement initiative, HIT can play a central role in assessing performance and improving adherence to care standards.

 An HIT-driven intervention can also improve hypertension-related patient outcomes.

Hypertension (HTN) is the most prevalent modifiable risk factor for cardiovascular disease among US adults, and it is among the most common reasons for an outpatient medical visit.

 Despite the availability of effective medications and published guidelines for the treatment of HTN, half of US adults who have been diagnosed with HTN have poorly controlled blood pressure (BP).

 Several recent literature reviews support a variety of implementation strategies aimed at increasing provider adherence to guideline-recommended care processes, including clinical reminder systems and performance feedback.

Numerous studies have also examined the efficacy of clinical decision support systems (CDSS) in improving the quality of preventive care.

 Yet, despite the theoretical and intuitive benefits of such technologies, the existing literature has demonstrated mixed empirical results.

 Moreover, few studies have examined the deployment of CDSS to improve the quality of HTN management.

 Finally, most studies of CDSS have been conducted in academic health centers or the Veterans Administration system.

We are aware of only 1 study of CDSS conducted in community health centers (CHCs).

 Yet, problems with quality of care and disparities are a particular concern for CHCs, which provide care for more than 15 million Americans, many of whom are most at risk for cardiovascular disease—related morbidity and mortality.

 Hicks et al found that among CHCs participating in Health Resources Service Administration Collaboratives, those with CDSS provided better quality of care than those without this technology.

 With primary care practices, including CHCs, responding to federal “Meaningful Use” requirements to adopt health information technology (HIT) by 2014, there is a need for greater understanding of technology-driven quality improvement strategies.

 The primary aim of this study was to compare the impact of a multicomponent intervention, which included an electronic medical record (EMR) with CDSS and registry-linked performance feedback, with a standard EMR alone on provider adherence to care recommendations and BP control as defined by the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7 Guidelines).

We used a quasi-experimental design with repeated measures. For data analysis, BP measures were collected for each patient encounter for 17 months pre-intervention (June 2007-October 2008) and 15 months (April 2009-June 2010) post-intervention. The 5-month period from November 2008 to March 2009 was the intervention adoption period.

The study was conducted at a 4-site federally qualified CHC, Open Door Family Medical Centers (Open Door), located in New York. The health centers provide primary care to approximately 40,000 patients annually, its population being primarily Hispanic (73.5%), with 60% foreign born. Thirty-five percent of patients had Medicaid and 58% were uninsured. In May 2007, Open Door installed eClinical-Works (eCW), an EMR and practice management system, at all 4 sites. Prior to the intervention, HTN control was part of the quarterly provider report card, which included 15 quality improvement measures, that was e-mailed to providers. However, the measures were not benchmarked against other providers or a target outcome. The EMR did not include CDSS components related to HTN.

We created a longitudinal database from EMRs for all adult nonobstetric patients with a diagnosis of HTN who had at least 1 visit during the study period. Hypertensive patients were defined as those with a documented 

International Classification of Diseases, Ninth Revision

The study team, which included Open Door leadership, providers and staff (ie, Chief Medical Officer [CMO], Director of Performance Improvement [DPI], Executive Director, and a physician representative), and staff from Primary Care Development Corporation, a non-profit organization with expertise in practice change and improvement, developed a set of measurable clinical goals related to HTN management that were based on JNC 7 Guidelines (

 The goals were agreed to by the clinic’s Quality Improvement Committee, and became the basis for the development of the intervention and the set of data to be extracted from the EMR for analysis.

As part of the intervention development, we conducted quantitative and qualitative interviews with all clinical staff and administrative leadership. The results established a baseline for evaluating any changes in attitudes as well as informed the study team about what tools the providers wanted from the EMR, their facility in using the EMR, training requirements, and other workflow and support issues. The main intervention components included CDSS, provider performance feedback, and provider training. Based on the pre-intervention interviews, consultation with the study team, and review of the literature, several features for the CDSS were implemented.10,12 These included:

, to present the provider with information to be obtained from the patient related to his/her HTN, and facilitate documentation;

, to prompt clinical support staff to ask patients about taking their medications and document the responses;

 focused on HTN, allowing the provider to access a single screen when ordering tests or treatment (

, to prompt providers to screen for tobacco use and/or update indicated tests (eg, lipid profile).

We provide more detail about the CDSS features in 

. In brief, the system was integrated with charting and order entry processes to support overall HTN-care workflow. For example, CDSS features appeared at the time and location of decision making (eg, when ordering medication), rather than just providing assessments, and occurred in multiple locations in the EMR.

The individual performance feedback was designed to enhance the existing report card system. The DPI ran quarterly reports on performance measures consistent with the main study outcomes, such as the percentage of HTN patients with visits in the previous quarter, whose BP was controlled. The CMO presented the overall performance of each site compared with Open Door’s target BP control at a regular quarterly staff meeting. Provider-level performance reports, which were benchmarked to predetermined targets were also e-mailed quarterly to providers, who could access their personal report through the EMR.

For clinical staff (physician assistants, nurse practitioners, medical doctors), the CMO conducted two 2-hour training sessions. The first included a review of JNC 7 HTN Guidelines and an overview of the HTN quality initiative, including clinical goals and objectives, as well as available baseline data. At the second training session, the CMO and research team demonstrated the new CDSS features, distributed a customized user manual dedicated to HTN CDSS features, and reviewed new policies and procedure guidelines for HTN management that were developed by the study team. Clinical support staff, including medical assistants, registered nurses, and licensed practical nurses, were trained together in a single 45-minute session using a curriculum customized to their roles (eg, importance of recording height and weight, asking about medication adherence, and using CDSS to record assessments).

The primary outcome was BP control before and after the intervention, as defined by JNC 7 criteria (BP <130/80 mm Hg for patients with diabetes or kidney disease and <140/90 mm Hg for all other patients). This was analyzed at the patient level and the encounter level. At the patient level, we report pre- and post-intervention mean systolic and diastolic blood pressures (SBP, DBP) and the proportion of patients with BP controlled at the last visit at both time periods (

). The process measures included the proportion of patients who had a documented body mass index (BMI), electrocardiogram (ECG), lipid profile, and nutrition visit in each study period, timeliness of followup visits when BP was out of normal range, and medication intensification. Medication intensification was defined as the percentage of visits with a new medication added, titration of the original antihypertensive medication, or both, among visits with an elevated BP.

We describe the process for extracting encounter-level data from eCW’s back-end database in a previous article.21 Briefly, data from eCW’s underlying tables were extracted using BridgeIT, a data management utility that mapped Open Door’s EMR data into easily accessed and queried Microsoft Access software. Study variables were extracted from the data warehouse for each patient encounter, saved in a Microsoft Excel file, and transmitted securely to the Data Coordinating Center at Columbia University.

After excluding patients of ethnicities other than black, white, or Hispanic and those with more than 30 visits, 3636 patients (28,263 encounters) were included in the final analysis. Data collected during the intervention adoption period were excluded from all analyses except for the time series in the 

The distribution of sample characteristics (

) among black, white, and Hispanic ethnicities was analyzed using analysis of variance (ANOVA) tests and 

 analysis. A preand post-intervention comparison on BP outcomes, as well as clinical process measures, was conducted using ANOVA tests and Fisher exact tests (Table 1). We also graphed changes in BP control over time using aggregated monthly encounter level data (Figure 1) and conducted an ANOVA test for significance of the BP trends over the 36-month study period.

) for BP control at any visit was developed using logistic regression considering the following covariates: age, BMI, gender, income level, race/ethnicity, diabetes status, and the number of HTN-related medications. Treatment effect was captured by an indicator that takes on a value of 1 if the particular visit occurred during the post-intervention period and 0 if the visit occurred during the baseline period. Generalized estimating equations

 were used to adjust for patients contributing multiple visits to the analysis. Coding for other categorical and continuous variables were based on which yielded the best fit in the univariate model where fit was evaluated using the Quasilikelihood under the Independence Model Criterion

which is analogous to Akaike’s Information Criterion. The multivariate logistic model was built using a stepwise procedure with an entry/exit value of 

Table 2 shows the characteristics of patients with a diagnosis of HTN that had at least 1 visit during the study period. Patients had an average of 8.84 (standard deviation 6.62) clinic visits during the 36-month study period (mean of 5 visits both pre- and post-intervention with no significant difference).

As shown in Table 1, average SBP and DBP were significantly lower in the post-intervention period compared with pre-intervention. Rates of BP control at the last visit during each study period also improved from 50.9% pre-intervention to 60.8% post-intervention. Significant improvements in BP control were also observed among patients with diabetes (33.3% at pre-intervention and 46.9% at post-intervention). However, rates of BP control remained lower than among patients without diabetes.

Table 1 also shows significant improvements in all process measures except medication intensification. Figure 1 shows the trend of monthly cross-sectional measures toward improved HTN control over the course of the study. This trend was statistically significant (

Controlling for other factors that were significantly associated with BP control, in Table 3 the logistic regression model shows that patients were 1.5 times more likely to have BP-controlled post-intervention than pre-intervention. Correlates of poor BP control were black race, male gender, income, greater BMI, diabetes, and larger number of prescribed antihypertensive medications.

We found that a multicomponent intervention that included provider performance feedback and CDSS promoted adherence to HTN clinical guidelines was associated with improvements in BP control. Provider feedback is a wellestablished strategy for improving adherence to guidelinerecommended care, and evidence is growing that CDSS and provider feedback are effective individually and in combination in facilitating quality improvement (QI).

Moreover, as demonstrated in the study sites, clinical information systems embedded in EMR have the potential to provide an easy way to generate performance feedback. Therefore, we chose to combine these 2 guideline implementation strategies.

Health information technology that is implemented as part of a multifaceted quality improvement initiative can lead to improvements in hypertension care and outcomes.

Technology-Driven Intervention to Improve Hypertension Outcomes in Community Health Centers

 This study sought to examine the implementation of an electronic health record— based intervention to improve quality of hypertension care in community health centers. The primary goal was to use qualitative analysis to explain how different components of the intervention contributed to positive patient-level outcomes.

 The intervention included alerts, order sets, templates, clinical reminder algorithms, and provider performance feedback. Semi-structured interviews were conducted with primary care providers before (n = 16) and after (n = 16) intervention, and with key staff and leadership involved in the implementation (n = 6). The research team applied an iterative systematic qualitative coding process to identify salient themes. Several constructs from IT implementation theories guided the analysis.

 The analysis focused on: (1) satisfaction and perceived usefulness of intervention components, (2) perceived proximal changes resulting from intervention, and (3) perceived facilitators of change. Different participants found different components useful. Proximal impact manifested in multiple ways (eg, more aggressive follow-up

	appointments and prescribing) and in increased overall attention to hypertension. Facilitators of success included leadership, organizational culture, provider engagement, rigorous implementation process, framing of intervention as quality improvement (QI), and health center capacity to process data.

 We attribute the success of the intervention to a multifaceted approach where the combination of multiple intervention components resulted in across the-board change in hypertension care practices. In contrast with research that attempts to isolate the impact of circumscribed health information technology (HIT) tools, our experience suggests that HIT can achieve success in patient outcomes when rigorously implemented as a multifaceted intervention and framed as QI activity.

(Am J Manag Care. 2011;17(12 Spec No.):SP95-SP102)

	Qualitative process evaluation of a multifaceted technology-driven hypertension intervention in community health centers resulted in the following key findings and conclusions:

		 Multiple intervention components contributed to changes in hypertension care practices, supporting the multifaceted approach to health information technology implementation.

		 Framing the technology-driven intervention as quality improvement was a facilitator of success.

		 Other facilitators included leadership, organizational culture, provider engagement, rigorous implementation process, and health center capacity to process electronic health record data.

		 Qualitative process evaluation was found to be a feasible and useful method for investigating reasons for the success of a multifaceted technology-based intervention.

	Hypertension affects one-third of the American adult population.

 Appropriate management of hypertension presents a significant opportunity to improve cardiovascular health outcomes through interventions in primary care settings. The costs of healthcare, medication, and missed days of work from hypertensionrelated symptoms and complications were $76.6 billion in 2010.

 Improved control of blood pressure among those diagnosed with hypertension would pay off in lowered incidence of stroke and heart disease.

	Health information technologies (HITs) are a promising tool for improving quality of care in primary care settings, including underserved settings such as community health centers (CHCs).

Much of HIT research tests the efficacy of isolated technology components, such as a decision support tool or computerized order entry. Less is known about the effectiveness of HIT-based interventions in community-based settings that combine multiple intervention components in a comprehensive effort to improve quality-of-care outcomes.

 Evidence of effectiveness of HIT to improve hypertension outcomes is mixed.

	Effective interventions to improve community health outcomes call for complex multi-level and multi-component approaches.

However, it is challenging to design studies that can attribute quantitative health outcomes to particular intervention components.

Instead, the outcomes indicate success of the entire “package.” Process evaluation is an avenue to explore questions about the effective “ingredients” of such interventions. One important purpose of process evaluation is to explain the results of an outcome evaluation,

 also called “interpretive evaluation.”

Process evaluation using qualitative methods may be particularly wellsuited for understanding why a complex intervention led to its outcomes.

	In the domain of information technology (IT) implementation, several models posit factors that explain success of IT. The Technology Acceptance Model (TAM)

builds on general social-behavioral theories and proposes constructs such as perceived usefulness and perceived ease of use as predictors of end-user acceptance and use of IT. The DeLone & McLean Information System Success Model

 predicts organizational and individual impact of IT systems from system use, user satisfaction, system quality, and information quality. A third model proposed by Ovretveit et al

 identifies factors that help or hinder HIT implementation, including characteristics of the HIT system, the implementation process, leadership, resources, and organization culture and climate. These models are helpful in categorizing and labeling factors of importance in HIT implementation research.

	This qualitative study examined a multi-component quality improvement (QI) intervention in a CHC with several practice sites in New York that included clinical decision support within an electronic health record (EHR) and provider feedback. The intervention had a positive impact on provider adherence to hypertension guidelines

 and hypertension control among patients.

 The goal of the qualitative analysis was to explain how different facets of the multi-component intervention contributed to the positive impact of improved hypertension control.

	The study setting was Open Door Family Medical Centers, a federally qualifying CHC with 4 primary care sites located in suburban communities in New York. The CHC provides primary care to approximately 40,000 patients annually. It is a safety-net provider with a patient population approximately 74% Hispanic, 15% non-Hispanic white, and 9% non-Hispanic black; 35% of the patients have Medicaid, 4% Medicare, 4% private insurance, and 57% no insurance. In May 2007, the CHC started using an EHR system (eClinicalWorks). The leadership of the CHC has a strong interest in systematic QI activities and was eager to use the new EHR as a QI tool. At baseline of the study, 14% of all CHC patients were diagnosed with hypertension and an additional 5% were found to have undiagnosed hypertension. About half of hypertensives had controlled blood pressure at their last visit.

	The CHC leadership had identified hypertension as a QI target because it was the most prevalent adult chronic condition, a significant contributor to leading causes of mortality, and appeared feasible to improve through appropriate management. The CHC had recently engaged in a successful QI diabetes program and wanted to now turn their attention to hypertension. The intervention was designed by a collaborative team that included the administrative and medical staff from the CHC and public health researchers. Components of the intervention were informed by pre-intervention interviews with all providers practicing at the 4 study sites. Several key components of the intervention were implemented in the CHC’s EHR system, including alerts of high blood pressure readings, and templates, order sets, and clinical reminder algorithms for hypertension management. The templates and order sets offered standard sets of items to be included in hypertension visits while the clinical reminder algorithm prompted specific actions based on patient characteristics. Providers also received trainings and quarterly report cards (feedback) indicating levels of hypertension control among their panel of patients. The report cards were discussed in group meetings, followed up by one-on-one meetings with the CHC Medical Director for providers who performed below average. Additional details of all components of the intervention are described elsewhere.

	The outcome evaluation design compared repeated measures of hypertension control before and after the intervention. The proportion of visits in which hypertension was controlled was 51% at baseline and increased significantly to 61% in the post-intervention period.

 The process evaluation included quantitative surveys of providers and qualitative interviews of providers and key informants. The quantitative surveys were primarily used to measure provider attitudes preintervention and to inform intervention design. This paper reports results based on the qualitative interviews.

	All providers practicing at the 4 health centers (n = 16) were interviewed about 6 months prior to and at 3 to 4 months following the launch of the intervention. The interview questions focused on perceptions and experiences regarding hypertension care, hypertension guidelines, EHRs, organizational environment, intervention components, and intervention implementation. Key informants (n = 6) representing leadership and staff actively involved in implementation of the intervention were interviewed about the implementation experience about 5 to 6 months after the intervention began. The semistructured interviews were conducted by the evaluators and lasted about 30 to 45 minutes. They were audio-recorded and transcribed for analysis. Human subject participation was approved by the Institutional Review Boards at New York University and Columbia University Medical Center. Informed consent was obtained from all interview participants.

	A team of 4 researchers conducted an iterative process of identifying a set of thematic codes and then applying them systematically to all interview transcripts. The team first reviewed key concepts in the 3 theoretical models (TAM, De- Lone & McLean’s model, and Øvretveit’s model). They then read all interview transcripts, identifying salient themes and notable quotations, and collaboratively drafted a hierarchically organized list of thematic codes. The final coding manual included a total of 78 themes and sub-themes. Using Atlas.ti qualitative analysis software, the researchers applied the codes systematically to all transcripts. To verify agreement, 10% of transcripts were coded by 2 researchers. For this paper, the codes were further reorganized under key process evaluation questions, as described below and shown in the 

	The findings were organized into 3 main domains that emerged as we examined the data in light of process evaluation research questions and implementation theories:

	1. Satisfaction and perceived usefulness of intervention components

	2. Perceived proximal changes resulting from the intervention

	3. Perceived facilitators of change Each of the 3 domains contributes answers to why the intervention had a successful outcome. The Table shows how the 3 domains relate to key functions of process evaluation and to major constructs of the 3 theoretical models used to guide the interpretation of the data.

A qualitative process evaluation attributes the success of a technology-driven hypertension intervention to the combination of multiple intervention components framed as quality improvement.

Special Issue: Health Information Technology - Guest Editors: Michael F. Furukawa, PhD; and Eric Poon, MD, MPH

The Comparative Effectiveness of 2 Electronic Prescribing Systems

Public Attitudes Toward Health Information Exchange: Perceived Benefits and Concerns

Technology-Driven Intervention to Improve Hypertension Outcomes in Community Health Centers

Qualitative Evaluation to Explain Success of Multifaceted Technology-Driven Hypertension Intervention

Use of Health Information Technology to Improve Medication Adherence

Meaningful Use of Health Information Technology: Evidence Suggests Benefits and Challenges Lie Ahead

Progress Toward Meaningful Use: Hospitals' Adoption of Electronic Health Records