Eric W. Ford, PhD, MPH

 To study the association of using an electronic health record (EHR)’s clinical reminder functionality with increased human papillomavirus (HPV) vaccine administration among US office-based physicians.

 National Ambulatory Medical Care Survey data from 2007-2012 were pooled together to determine if EHR systems that employ clinical reminders are associated with increased immunization rates in different populations.

 The administration of HPV vaccine served as the dependent variable, with the EHR reminder being the primary independent variable of interest. Logit regression was used to assess the relationship between using EHR reminders and HPV vaccine administration.

 Analyses indicated that compared with physicians without clinical reminder functions, physicians with clinical reminder functions were more likely to order HPV vaccines. Clinical reminder functions were particularly effective at increasing HPV vaccine use among adolescent males.

 Overall, physicians who report using clinical reminders were more likely to order HPV immunizations. However, the association of clinical reminders with HPV immunizations was not significant in the younger adolescent subpopulation. Further, given that HPV incidence decreases significantly even with small gains in vaccination rates, the increase in HPV immunizations found in the male population aged 11 to 21 years is promising. Therefore, targeting males to receive HPV vaccination immunizations through clinical reminders provides a positively disproportionate return on vaccination rates and disease burden.

A significant amount of time, effort, and money has gone into increasing the meaningful use of electronic health records (EHRs). This study’s findings demonstrate that: 

Using clinical reminders is strongly associated with higher vaccination rates for human papillomavirus (HPV). 

The adolescent male population has an especially higher HPV vaccination rate in the presence of clinical reminders. 

EHR-driven clinical reminders have the potential to increase preventive care among at-risk, but often neglected, subpopulations.

Human papillomavirus (HPV) is the most common sexually transmitted infection in the United States, causing genital warts and cervical and other cancers. It is estimated that 79 million people are currently infected and 14 million people acquire new HPV infections each year in the United States.

 Furthermore, 17,500 female and 9300 male Americans are affected by HPV-associated cancers annually.

 The widespread availability of an HPV vaccine has the promise to reduce the spread of many infectious diseases, including several that lead to cancers.

 Despite the efficacy of HPV vaccines, their use continues to lag behind that of other immunizations (eg, the tetanus, diphtheria, and pertussis vaccine and the first of 2 doses of quadrivalent conjugate meningococcal vaccine) among the targeted age groups.

 Moreover, differences in vaccination rates may be systematically linked to sociodemographic characteristics and represent an inequality in care delivery.

 Therefore, increasing HPV vaccine use to the levels of other commonly administered immunizations and across at-risk populations is desirable.

One tool for improving primary care quality, particularly for routine tasks such as immunization administration, is the electronic health record (EHR). EHRs have the ability to provide clinical reminders that support evidence-based care; as such, there is an expectation that such systems will improve care processes and outcomes. Prior study results have shown that uses of EHR and clinical decision support are associated with improvements in preventive care services, such as health behavioral counseling, screening tests, prevention and management of chronic diseases (eg, cancer and cardiovascular diseases), preventive medications prescribed, and vaccination.

 The promise of improved care was foundational to the US government’s Meaningful Use program to increase physicians’ adoption and implementation of EHR systems.

 Within that context, one can empirically test the question of whether EHRs with clinical reminders have improved adherence to HPV immunization guidelines.

The purpose of this article is to explore the relationships among EHR adoption, clinical reminder use, and HPV immunization rates. Data from the 2007-2012 National Ambulatory Medical Care Survey (NAMCS) are analyzed using logit model regressions, with HPV immunization rates as the dependent variable. The results include analyses of HPV vaccination rates across sociodemographic groups; a discussion of the findings’ implications and areas of future research are promulgated. 

The study’s findings address an important interest area for public health officials, policy makers charged with promoting EHR Meaningful Use criteria, those seeking equity in healthcare provision, and clinicians. For public health advocates, any tool that increases effective immunization adherence in support of disease prevention is an important contribution to the well-being of communities. Additionally, policy makers responsible for the Meaningful Use program need meaningful measurements to assess whether increased EHR adoption leads to better treatment modalities. Mitigating care delivery disparities is an important aim for many government programs, including Meaningful Use. Lastly, for clinicians, evidence that EHRs can help them deliver better care is needed to more effectively assess the return on investment associated with owning and operating such a technology.

This study used the 2007-2012 iterations of the NAMCS to assess the use of clinical reminders in relation to HPV immunization rates in visits to office-based physicians. The NAMCS is a national probability sample survey administered by the National Center for Health Statistics on behalf of the CDC that collects data on patient visits to non—federally employed office-based physicians. For each visit, physicians or staff members complete a 1-page survey containing patient demographics, reasons for the visit, physician’s diagnoses, and medications ordered, supplied, administered, or continued during patient encounters. The NAMCS uses a multistage probability sampling design, which allows for the generation of nationally representative estimates. 

This study analyzed visits by adolescents recommended to receive HPV vaccines to all office-based physicians, including primary care physicians, pediatricians, internists, obstetricians/gynecologists, and other specialists. Detailed descriptions of physician specialties are available from the authors and on the NAMCS website. HPV vaccination is recommended for both girls and boys at age 11 or 12 years. Females aged 13 to 26 years and males aged 13 to 21 years who have not been vaccinated previously are also recommended to receive HPV vaccines.

 To estimate the influence of clinical reminders on HPV vaccines ordered or administered in the ambulatory setting during adolescent visits, the study analyzed 3 adolescent visit samples during 2007-2012: 1) visits with patients aged 11 to 12 years (3388 visits), 2) visits with patients aged 11 to 18 years (14,354 visits), and 3) visits with female patients aged 11 to 26 years and male patients aged 11 to 21 years (25,573 visits).

 The NAMCS contains data on medications ordered, supplied, administered, or continued during patient encounters. The outcome of interest in this study was a dichotomous measure that identifies whether the physician ordered, supplied, administered, or continued HPV vaccine immunizations during adolescent patient visits.

 In the NAMCS survey, physicians reported whether their practices used an EHR system and, if so, which functionalities their EHR included. Based on physician-reported information regarding specific EHR functionalities used in their practices, the key independent variable of this study was related to physicians’ use of the clinical reminder function of the EHR. The clinical reminder was coded by examining physicians’ response to the question, “Does your practice have computerized capabilities of providing reminders for guideline-based interventions or screening tests and how often is the capability used?” “Clinical reminder” was coded as "1" if the response was “Yes” and coded as "0" for those who responded “No,” “Unknown,” or “Yes, but turned off or not used.” In 2012, the NAMCS changed questions on specific EHR functionalities to differentiate between “routine use” and “nonroutine use” of specific functions such as clinical reminders.

A cross-sectional analysis of pooled survey data was conducted. We used weighted multivariable logit regression to estimate the association of clinical reminder use with HPV vaccine immunization rates at adolescent visits. In each regression, we included interaction terms between a dichotomous indicator for patients’ sex (female) and clinical reminder use to examine how the association of clinical reminder use with HPV vaccine immunization rates varies between male and female adolescent patients. All of the regressions controlled for potential confounders, including patient demographic characteristics (race/ethnicity), visit characteristics (ie, preventive care visits, patient’s own primary care physician), and other covariates, such as metropolitan statistical area status and survey year. Instead of adjusted odds ratios, marginal effects of coefficients in logit regressions were estimated and presented in the subsequent result section because odds ratio interpretation of interaction terms in nonlinear models, such as logit models, are problematic.

HPV Vaccine Immunization Rates by Clinical Reminder Use (unadjusted)

Survey-weighted descriptive statistics for HPV vaccine immunizations, clinical reminder use, and other covariates are reported in 

. From 2007 to 2012 in the United States, there were 89.1 million office-based physician visits by adolescents aged 11 to 12 years, 381.6 million ambulatory clinic visits by those aged 11 to 18 years, and 672.7 million visits by female adolescents aged 11 to 26 years and male adolescents aged 11 to 21 years. In ambulatory clinic visits by adolescents aged 11 to 18 years, approximately 40% were with clinical reminders. 

 presents survey-weighted rates of HPV vaccine immunizations by clinical reminder use in the different samples of adolescent visits. The rate of HPV vaccine immunizations varied by the use of clinical reminder function. Compared with physicians without clinical reminder functions, physicians with clinical reminder functions were more likely to order HPV vaccines. The pattern holds consistently across patients’ sex and age-group samples. For example, in female patients aged 11 to 12 years, HPV vaccines were ordered in 6.44% of encounters where a clinical reminder function was present and 5.24% when it was not. In visits by male patients aged 11 to 12 years, HPV vaccines were ordered in 1.50% of encounters with clinical reminders and 0.61% without. The subsequent section will detail further analysis on the statistical significance of these patterns, controlling for patient and visit characteristics.

Estimated Association Between Clinical Reminder Use and HPV Vaccine Immunization

presents the differential estimated association of clinical reminder use with HPV vaccine immunizations by patient sex. Logit coefficients are presented in odd columns and marginal effects are presented in even columns. No statistically significant associations were found between clinical reminder use and HPV vaccine immunization in the samples of: 1) visits with patients aged 11 to 12 years and 2) visits with patients aged 11 to 18 years. In visits with female patients aged 11 to 26 years and male patients aged 11 to 21 years, use of clinical reminders was associated with a higher rate of HPV vaccine immunizations for male and female patients. Furthermore, the coefficients for the interaction term (Clinical reminder use × Female) were negative (marginal effect, —0.0136; 

<.1), and the magnitude of the coefficients on the interaction is less than the magnitude of coefficients on clinical reminder use (marginal effect, 0.0164; 

 <.01). This indicates that the association of clinical reminder use with HPV vaccines ordered is greater in male encounters than in female encounters. 

Compared with male patients aged 11 to 21 years who visited physicians who did not adopt clinical reminders, male patients in the same age group who visited physicians who did use clinical reminders received more HPV vaccines, by 1.64 percentage points. This represents a relative increase of 146% from the mean probability of HPV vaccine immunizations (1.12%). Compared with female patients aged 11 to 26 years who visited physicians who did not adopt clinical reminders, female patients in the same age group who visited physicians who did use clinical reminders received more HPV vaccines, by 0.28 percentage points. This represents a relative increase of 10.7% from the mean probability of HPV vaccine immunizations (2.61%).

We performed sensitivity tests to address concerns that the actual use of the reminder function, not merely the simple adoption of clinical reminders, guarantees the improvement in care. Using 2012 data that are able to differentiate “routine use” and “nonroutine use” of clinical reminders, we compared rates of HPV vaccine immunizations by the level of clinical reminder use. Compared with physicians who did not adopt clinical reminders, physicians routinely using clinical reminders were more likely to order HPV vaccines (

]). On the other hand, physicians nonroutinely using clinical reminders were not consistently more likely to order HPV vaccines compared with physicians not adopting clinical reminders. Thus, these results indicate that we may have underestimated the association of clinical reminder use with HPV vaccine immunizations, which supports the robustness of our primary findings. 

Among physicians reporting that they used clinical reminders, the measure was positively correlated with HPV vaccine immunizations. The associations are greater in magnitude for males aged 11 to 21 years than females aged 11 to 26 years. One explanation for this phenomenon is that physicians may consider HPV to be primarily a gynecological issue. Therefore, the reminder would have a greater impact in populations where the disease risk is less well understood.

No such associations of clinical reminder use and HPV vaccine immunizations were seen in the subpopulation of younger adolescents. The difference between age groups suggests that certain behavioral characteristics associated with recommending the HPV vaccine may be difficult to overcome in the younger population.

 Given that HPV vaccine immunization is designed to have greater impact earlier in life, this result suggests that clinical reminders may not be adequate in increasing vaccination rates in this population. It may be necessary to couple clinical reminders with other behavioral interventions for physicians. Tobacco use, in particular, merits attention, as those who smoke are at elevated risk for oral cancers that need HPV exposure to develop.

The significant increase in vaccination rates associated with clinical reminder use that we saw in the male population aged 11 to 21 years is a promising avenue for intervention. For HPV, the disease burden for the individual and the population decreases significantly even with small increases in vaccination rates.

 Given that males have a higher prevalence of HPV infections, they merit additional attention.

 Thus, targeting males to receive HPV vaccination immunizations through clinical reminders provides a positively disproportionate return on vaccination rates and disease burden. These results also suggest that barriers to HPV vaccination recommendations may be less for males than females. Many parents are resistant to vaccination of their daughters due to the concern that the HPV vaccine may encourage sexual debut of their daughters.

Reviewing the control variables, several key observations can be made. As expected, preventive care visits were associated with significantly higher rates of HPV vaccination, along with visits to patients’ own primary care physicians. We saw no significant associations between race and HPV vaccination. This suggests that disparities in HPV preventive care may be minimal in this sample population. However, it is important to carefully monitor the traditionally underserved population, often with higher rates of cervical cancer, to ensure that access to preventive care visits is available in order to obtain HPV vaccines.

There are limitations to this study. We used a cross-sectional snapshot to determine the correlation between clinical reminders and HPV vaccination rates; thus, we cannot establish a causal relationship. It could be that physicians adopting new technology, such as EHRs, and clinical decision support systems tend to follow recently released recommendations on HPV vaccination. It is also possible that physicians who adhere to clinical guidelines on vaccination are more interested in and likely to adopt clinical support systems as tools to improve their quality of care. Furthermore, the lack of longitudinal data means that patients are not being tracked over time. Given that the HPV vaccine requires 3 doses to be administered over an extended period, it is not possible to assess whether reminders promote better completion of the recommended regime.

Additionally, the NAMCS cannot identify whether the vaccine order is for the first, second, or third dose of HPV vaccine. Besides, the NAMCS data set is a self-report survey by physicians and staff members, who may be prone to over- or under-report preventive services and EHR functionalities. Furthermore, the NAMCS does not sufficiently report on both patient and physician practice characteristics, and these unobservable characteristics, such as patient socioeconomic characteristics or physician age and years of practice, could confound the association between clinical reminder use and our outcome of interest. Finally, the NAMCS information on the diversity and complexity of EHR systems and specific functionalities used in clinical practices was limited. Thus, we cannot fully identify whether and how different EHR systems and their clinical reminder functions help HPV vaccine orders in visits made by adolescents who are recommended to take HPV vaccines. In addition, this study could not address the impact of type of vendor, data architecture, and end-user interface.

Minority access to practices with high-functioning EHRs and the concomitant impact on preventive care is an area that merits more research. If EHRs produce a significant improvement in care quality, as most expect, then this trend represents an emerging disparity. Another area of research that needs further work is related to the herd immunity effect of HPV vaccines. If relatively small increases in vaccination rates lead to significantly large reductions in HPV infection rates, this needs to be better understood. 

Clinical reminders are positively correlated with better care processes related to HPV vaccination. Promoting the routine use of clinical reminders for vaccination will yield significant benefits for the general population. Clinician-initiated recommendation is a key ingredient in starting successful HPV vaccinations.

 Thus, clinical reminders serve as an important gateway for physicians to initiate a conversation on HPV vaccination with families. This is especially important in addressing coverage in traditionally underserved populations. Especially for HPV vaccinations, where strong social barriers may exist in its disease etiology and cause, it is important to decrease other barriers for clinical opportunity.

 School of Health Studies, Northern Illinois University (JB), Dekalb, IL; Health Policy and Management, Johns Hopkins Bloomberg School of Public Health (EWF, SW), Baltimore, MD; Department of Family Medicine, College of Medicine, The Ohio State University (TH), Columbus, OH.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. 

 Concept and design (JB, EWF); acquisition of data (JB); analysis and interpretation of data (JB, SW, TH); drafting of the manuscript (JB, EWF, SW, TH); critical revision of the manuscript for important intellectual content (JB, EWF, TH); statistical analysis (TH); administrative, technical, or logistic support (EWF, SW); and supervision (TH).

 Eric W. Ford, PhD, MPH, Johns Hopkins Bloomberg School of Public Health, 624 N Broadway, Hampton House 533, Baltimore, MD 21205. E-mail: ewford@jhu.edu.

1. CDC. Incidence, prevalence, and cost of sexually transmitted infections in the United States. http://www.cdc.gov/std/stats/sti-estimates-fact-sheet-feb-2013.pdf. Published February 2013. Accessed October 8, 2016.

2. Markowitz LE, Dunne EF, Saraiya M, et al; CDC. Human papillomavirus vaccination: recommendations of the Advisory Committee on Immunization Practices (ACIP). 

3. Trottier H, Franco EL. Human papillomavirus and cervical cancer: burden of illness and basis for prevention. 

4. Zhai L, Tumban E. Gardasil-9: a global survey of projected efficacy. 

2016;130:101-109. doi: 10.1016/j.antiviral.2016.03.016.

5. Suryadevara M, Paton L, Domachowske JB. Adolescent immunization: 2015 and beyond. 

2015;44(4):e82-e88. doi: 10.3928/00904481-20150410-09.

Articles

The use of the electronic health record’s clinical reminder functionality is systematically related to higher human papillomavirus vaccine administration rates.

Electronic Reminder's Role in Promoting Human Papillomavirus Vaccine Use

 To assess the importance of commonly identified issues in electronic health record (EHR) implementation using item response theory (IRT).

 Secondary data from the 2012 American Hospital Association’s Annual Survey Information Technology Supplement were used in the analyses. Results were compared and contrasted with the standard descriptive statistic frequencies that have been used to guide most recommendations made using the same data.

 IRT was used to measure the magnitude of difficulty that particular challenges pose in implementing EHRs that meet federal guidelines for Meaningful Use.

 The IRT analyses yielded significantly different results from descriptive statistics in estimating the magnitude of specific EHR implementation challenges. In particular, IRT revealed that “obtaining physician cooperation” and “ongoing costs of maintaining and upgrading systems” were the most challenging implementation features. However, the frequency counts identified “upfront capital costs” and “complexity of meeting Meaningful Use criteria within implementation timeline” as the most challenging implementation features.

 For managers and policy makers, having an accurate assessment of EHR implementation challenges is essential to designing effective programs. IRT provides a statistical approach that allows prior studies to be assessed more accurately and future studies to retain the easier-to-use, check-all-that-apply survey structure while gaining valuable information.

	Many surveys of hospital administrators have sought to identify barriers to the Meaningful Use of electronic health record (EHR) technology. However, the surveys used often lack the precision to provide a list of barriers ordered from most difficult to least difficult. Instead, surveys tend to report the most common or well-known barriers to adoption. 

		The analysis identifies obtaining physician cooperation as the most challenging barrier to EHR adoption. 

		Some frequently cited barriers, such as dealing with the complexity of the Meaningful Use requirements are shown to be less challenging.

Implementing an electronic health record (EHR) in a hospital brings many challenges—some bigger than others.

 Managers often recognize common but readily addressed EHR implementation issues, while more daunting challenges are identified less frequently or later in the process. Prior research has shown that hospitals are slow to implement key EHR features that enhance patient safety and care quality.

 In particular, professional buy-in and implementation costs have been identified as major impediments to EHR use.

 The latter issue—implementation cost barriers&mdash;is further exacerbated because demonstrating an EHR’s return on investment is difficult.

 Determining which EHR implementation issues hinder the widespread “Meaningful Use” of the technology is important.

The problem in identifying and prioritizing EHR implementation issues stems from managerial knowledge gaps arising from a lack of experience with the process. To identify knowledge gaps, professional organizations survey health system leaders about their EHR implementation challenges.

 The American Hospital Association (AHA)’s Annual Survey Information Technology (IT) Supplement (referred to hereafter as “AHA Survey”) asks managers to pick among a list of options for all EHR implementation challenges that their hospital currently faces or anticipates. Although this approach is likely to capture how common an issue is among hospitals, it does not explicitly address how difficult a specific challenge is. Moreover, using response frequencies as a proxy measure for EHR implementation difficulty may lead to inaccurate inferences about the areas where managers should focus their efforts.

The purpose of this paper is 2-fold: first, the challenges hospital administrators face in implementing an EHR were examined using a holistic approach to item difficulty; second, an established psychometric methodology was employed to quantify the relative difficulty levels of various EHR implementation challenges, based on managers’ responses to a “pick all that apply” questionnaire. Item response theory (IRT)

 was used to analyze the AHA Survey’s questions related to EHR implementation challenges in meeting federal requirements for Meaningful Use.

 Results were compared and contrasted with the standard, descriptive statistics that have been widely reported.

A 2014 literature review of hospital EHR implementation studies identified 6 main categories of barriers

: 1) financial capabilities, related to the total cost of EHR system ownership (acquisition and operation); 2) vendor selection, related to integrating ITs into the workplace; 3) IT staffing, related to having employees with prior implementation experience; 4) health system culture, related to organizational collaboration challenges; 5) organizational complexity, related to the bureaucracy and change management capacity of the hospital; and 6) clinical professional barriers, related to the ITs’ impact on providers’ abilities to deliver care. Many dimensions play roles in the EHR implementation process, but 2 themes emerge consistently across the 6 domains identified in the prior research: experience with EHR implementation and the complexity of managing the particular barrier.

The common factors that affect a barrier’s potential importance can be put into a 2-dimensional framework of barrier complexity and experience. The experience dimension can be translated into where a health system is in its tacit knowledge about EHR implementation barriers. IRT is specifically designed to order respondents’ relative knowledge about specific content. The model is presented in the 

The most complex managerial challenge when implementing an EHR is gaining clinician buy-in.

 By both social and legal conventions, physicians have the professional autonomy to make clinical decisions.

 The introduction of an EHR with predefined order sets, automated reminders, and decision support is perceived by many physicians as a direct attempt to diminish their professional autonomy. In addition, many physicians are not directly employed by health systems, further reducing their organizational commitment. Taken together, the resistance to managerial oversight and lack of organizational identification make gaining physician buy-in a major issue. Nurses have a lesser degree of professional autonomy and are typically employed by the hospital; hence, they tend to be less of a management challenge than physicians when it comes to EHR implementation and use. Nevertheless, the physician—nurse relationship is critical to EHR implementation success.

In the Figure, the clinical staff response (items 3 and 4) begins during the EHR adoption phase. Normatively, doctors and nurses should be brought in at the earliest stage. In practice, they tend to be included after the decision to adopt the EHR has been made during vendor selection.

 The importance of clinical staff engagement is an ongoing management challenge, as the degree of clinical functionality in the EHR and required care process documentation is ever-increasing. Therefore, management experience with implementing more sophisticated functionalities is a learned capability that continues ad infinitum.

The financial capacity (items 1 and 2) of the health system to support the EHR is also an ongoing challenge. The initial cost of purchasing an EHR is considerable, but knowable; however, it is the ongoing cost of operating the systems that many managers underestimate.

 Losses in productivity have a concomitant impact on revenues,

 and the double impact of unexpectedly high ownership costs and reduced revenue cause managers significant challenges. Managers typically become fully cognizant of these total cost of ownership (TCO) issues only after they begin the implementation process; the start date in the Figure reflects this fact.

Similar to the experience-informed learning that occurs with the recognition of TCO issues, the additional levels of organizational complexity (item 10) that are part-and-parcel of an EHR implementation become clearer over time.

 Post-EHR implementation, health systems have to roll out additional functionalities to meet organizational needs and comply with ever-increasing regulations. Hence, this construct also encompasses the entire EHR lifecycle.

Organizations’ cultures (item 9) are also impacted by the introduction of new technologies and the workflow changes that come with them.

 Depending on when organizational members are engaged in the EHR adoption and implementation processes, the impact on organizational culture can start very early on and continue indefinitely. One means of making the cultural shift a positive one is to engage staff members during the vendor selection phase to promote buy-in.

Vendor selection and operations are closely related to issues of EHR product certification (item 6), ensuring system security against privacy breaches (item 5), and ongoing system support (item 7). A major criterion in vendor selection is to reduce the management burden related to these items.

 Therefore, they appear lower on the management complexity axis. Despite being lower on the complexity dimension, ongoing vendor relations mean that it is an enduring management activity.

The lowest EHR implementation challenge construct, in terms of management complexity, is information technology staffing (item 8).

 The employees dedicated to supporting the technology are easier to manage because, unlike that of clinical staff, their job description explicitly includes following the direction of leadership. Nevertheless, managers often underestimate the number of IT staff needed to support an EHR.

 Therefore, the management challenge continues well into the implementation phase until the organization reaches some level of homeostasis with respect to the EHR.

The EHR adoption/implementation challenges used in the AHA Survey are not a perfect fit to the constructs listed in the literature review of Boonstra and colleagues.

 Nevertheless, the AHA Survey does touch on each of the major domains in some fashion and serves as a good starting point from which to explore EHR implementation challenges from a health system perspective.

 was funded by the Office of the National Coordinator for Health Information Technology (ONC) and sent to 5756 US hospitals (including those not members of the AHA). The survey had a 59% response rate for a sample of 3396. The survey question, “What is (are)/would be the primary challenge(s) in implementing an EHR system that meets the federal requirements for meaningful use? (Please check all that apply)” was analyzed. The item offered 10 structured responses, including: 1) upfront capital costs/lack of access to capital to install systems, 2) ongoing cost of maintaining and upgrading systems, 3) obtaining physician cooperation, 4) obtaining other staff cooperation, 5) concerns about security or liability for privacy breaches, 6) uncertainty about certification requirements, 7) limited vendor capacity, 8) lack of adequate IT personnel in hospital to support implementation/maintenance, 9) challenge/complexity of meeting all Meaningful Use criteria within implementation timeframe, and 10) complexity associated with coordinating decisions with system-level leadership. These responses were used in the IRT phase.

IRT is a collection of modeling techniques for analyzing item-level data obtained to measure variation between respondents. Unlike regression methods, IRT does not require a dependent variable against which to parse variance; instead, it compares the items against one another to establish the pattern of responses from a sample of survey or test takers. In addition, by evaluating the confidence intervals (CIs) of the item difficulty measurements, it is possible to assess if there are potentially missing items in the scale. When sorted by item difficulty, each CI should overlap with the next item in the list. CIs that do not overlap indicate that there may be another item that would improve the classification of survey respondents. The CI comparison can be used to detect potential gaps in an assessment scale’s items.

One important assumption of IRT is that the scale being assessed is unidimensional. To test this assumption, a factor analysis is performed in the items to ensure they are measuring a common construct (see 

] for a description of the factor analysis). Following that procedure, the IRT itself can be undertaken to assess both item and respondent characteristics (see 

 display the characteristics of the hospitals in the sample and the correlations among the responses to the 10-item questionnaire, asking: “What is (are, or would be) the primary challenge(s) in implementing an EHR/EHR system that meets the federal requirements for Meaningful Use? (Please check all that apply).” These items were factor-analyzed using the principal factors and unrotated modeling specifications. The amount of variance explained by the first factor was 73.9%, and the factor loading for each item was ≥0.460. The ratio of the Eigen value for the first factor (3.184) to the second (0.839) was 3.8. Overall, these results indicated that the responses form a single construct, with varying levels of organizational challenge being the distinguishing feature.

The results of the IRT analyses are presented in 

, ranked by item difficulty. The “obtaining physician cooperation is a primary challenge” response had the highest item difficulty rating (

 = 1.039), but was the fourth most frequently identified challenge (43.12% of responses). The 2 challenges related to EHR financing— ongoing costs of maintaining and upgrading systems” (

 = 0.286) and “upfront capital costs/lack of access to capital to install systems” (

 = 0.267)—were the second and third most difficult items, respectively, based on the IRT analysis. However, these 2 items were the first (upfront capital cost) and fifth (ongoing cost) most common responses in the descriptive statistics using frequency counts. In addition to being a major challenge, the “ongoing cost” response also had the highest discrimination of any item (

 = 1.696). IRT analysis re-ordered the challenges relative to the response frequency approach for assessing item importance.

The fourth and fifth items based on the difficulty measure related to technical aspects of implementing an EHR. “Challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” was the fourth most difficult challenge (

 = —0.198). This item also had lower discrimination (

 = 0.993). The “lack of adequate IT personnel in the hospital to support implementation/maintenance” item was the next most difficult (

 = —0.303) and had a slightly higher discrimination index (

 = 1.041) than the “…meeting all Meaningful Use criteria…” item. “Obtaining other staff cooperation” item (

 = —1.150) did not group with the other personnel challenge related to gaining physician buy-in. However, it did have relatively high discrimination (

 = 1.331) compared with the physician item (

 = 1.039). The last item with good discrimination was “concerns about security or liability for privacy breaches” (

 = 1.303). The remaining items had both low difficulty and discrimination measures, indicating that these issues are not impacting EHR deployment.

Examining the item difficulty CIs, there are 4 potential gaps in the scale used to measure EHR implementation challenges. The first gap appears between the items “upfront capital costs/lack of access to capital to install systems” and “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline.” There are 4 more potential gaps noted in Table 3. Potential missing items are discussed in the next section.

The findings have both policy and methodological implications. “Obtaining physician cooperation” is the biggest challenge to implementing an EHR based on the IRT analysis. However, the Meaningful Use program provides no explicit rewards to doctors in the hospital environment. In the ambulatory setting, individual physicians are targeted for rewards if they are early adopters, and penalties if they procrastinate about EHR implementation. The rationale for the rewards in the ambulatory environment was to compensate doctors for their productivity levels likely suffering in the near term after implementation. The effect of an EHR implementation in the hospital is similar for doctors, but no offsetting compensation is made available. Hence, increased resistance from physicians is likely to impact hospital managers charged with EHR implementation.

The IRT results also offer methodological insights. “Obtaining physician cooperation” was the most challenging EHR implementation barrier in the IRT results. These findings are similar to those of other studies determining that many physicians do not routinely use EHRs to improve care quality.

 The “upfront capital cost” response was the most frequently identified EHR implementation challenge, but it was only the third most difficult measure based on an IRT analysis. The switching of positions from the counts listing to the IRT is an indication that descriptive statistics alone do not provide a complete or accurate assessment. The “ongoing…” and “upfront…” cost measures were also very challenging for hospitals.

 The Item Discrimination Index is highest for the challenge related to “ongoing costs.” Therefore, this concern is the item most likely to differentiate respondents.

Although inferential statistics in general and logistic regression, in particular, have the potential to highlight differences in hospitals’ EHR implementation challenges, they also have limitations. Logistic regression models require a dependent variable to assess an item’s relative importance. For example, the study by Adler-Milstein and colleagues,

 which analyzed the same battery of EHR implementation challenges using Meaningful Use attestation as the dependent variable, found that cost measures, rather than physician cooperation, were the biggest challenge. The difference arises because EHR implementation challenge items are not being compared with one another; instead, variance is being allocated across them in a zero-sum fashion.

In contrast, IRT identifies that the “challenge/complexity of meeting Meaningful Use within specified timelines” was a major challenge regardless of other organizational characteristics. In the regression analysis, both groups believed the issue to be important, but no statistical difference was detected, so the item was not considered to be significant in the discussion. Moreover, the relatively low Item Discrimination Index for the “complexity of meeting Meaningful Use within specified timelines” measure indicates that the variable will not differentiate hospitals on other characteristics. Therefore, this is a relatively difficult challenge common to most facilities regardless of other characteristics.

IRT identifies the relative placement of EHR implementation barriers by utilizing response patterns within individual hospitals to inform item hierarchical ordering. Although the Meaningful Use attainment measure can be used as proxy for EHR implementation difficulty, it did not assess the underlying organizational capabilities and resources that make some barriers more difficult than others for a particular hospital. In that respect, the regression analysis approach is underspecified, given that omitted variables are likely. Therefore, having a means of assessing organizational barriers/challenges that does not require other constructs to be fully identified allows for alternative explanations as to why some EHR implementation challenges are greater than others.

There are 3 interpretations for why the IRT produced different results than either the frequency count or logistic regression approaches used in the other studies. From a learning perspective, those facilities with more experience will have a better understanding of different challenges’ operational impact. Therefore, those that have achieved Meaningful Use Stage 2 are more likely to prioritize challenges relying on the benefit of hindsight. A second interpretation relates to the innovation adoption-implementation lifecycle. Hospitals further into the process will be facing different challenges than those just beginning. Specifically, “obtain physician buy-in” is likely to be easier when the EHR is an abstract idea and more difficult when the reality of using the systems is encountered. Organizational capacity provides a third view for considering EHR adoption-implementation challenges. Under this interpretation, hospitals with better management will identify the more intractable EHR adoption-implementation challenges with greater frequency.

 Through any of these perspectives, the “obtaining physician cooperation” challenge would be the furthest along on the learning curve.

Another important feature of the IRT analysis was the identification of potential gaps in the EHR implementation challenge scale as it was fielded. The potential gap between “upfront capital costs/lack of access to capital to install systems” and “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” is at the highest item difficulty level. The “upfront capital…” item is clearly written, but the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” response has 2 potential issues.

First, the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” item is poorly phrased. The part of the question using the phrase “challenge/complexity” lacks specificity. What aspect of the implementation is “complex”? EHR implementations require health systems to “build” many of the user interfaces, create order sets, and link to other information systems. Respondents may be responding to this aspect of EHR implementation complexity. Alternatively, EHR implementations require significant staff training evolutions that are different than the concepts captured in the “obtaining staff/physician cooperation” items. There is no mention of staff training in the scale as it was put to the respondents.

A second issue is that the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” item has 2 questions embedded within it. The “challenge/complexity” phrase is one element and the EHR “…implementation timeline” is a second distinct concept. Creating 1 or 2 time-related items would be an improvement. One item would address meeting the timelines set by the organization for implementing the EHR. The second would assess the health system’s ability to meet the Meaningful Use target timelines. Given health systems could join the Meaningful Use program at different points without penalty; the 2 timelines are not necessarily the same.

Disaggregating and more accurately specifying the concepts in the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” question could also fill in the gaps that appear in the scale. Per the earlier discussion, breaking out the EHR “build” features could help fill the gap above the “lack of adequate IT personnel in the hospital to support implementation/maintenance” and “obtaining other staff cooperation” items. One possible solution is to look to the survey itself. The survey asks respondents to list “other barriers to EHR implementation.” Exploring those might yield potential items to fill in the gaps identified in the scale as it is currently employed. The potential “staff training” and “EHR development” items discussed above were drawn from that list.

Having EHR implementation challenge scales that are lacking potential items is an important issue for policy makers. The ONC funds research into areas that have been identified as impediments to effective EHR use. Lacking accurate and complete feedback from health system administrators diminishes the ONC’s ability to target EHR implementation challenges.

Similar to having incomplete information, having the EHR implementation challenges incorrectly sorted may lead to misguided investment strategies. As the IRT analysis demonstrates, “obtaining physician cooperation” is the most difficult item. Changing the culture of the medical community is a far more difficult undertaking than funding EHR purchases and implementations. To alter the physician community’s beliefs, it may be necessary to start while physicians are in medical school. Such a transformation would take a generation to implement.

 Ongoing organizational training may help to improve EHR acceptance.

Both EHR implementation and methodological lessons can be drawn. With respect to EHR implementation, among the challenges evaluated, “obtaining physician cooperation” is the most difficult part of the process. For those hospitals in the initial stages of EHR implementation, the importance of gaining physician buy-in may not be apparent. Similarly, the importance of managing ongoing ownership and operating costs for EHR systems may not be fully understood by early adopters compared with late adopters. Public policies that provide rewards for implementing more sophisticated EHR functionalities can be tailored to ameliorate these challenges. The IRT analysis also helps quantify the relative difficulty of each challenge that health systems face.

Many surveys of healthcare providers and managers employ multiple selection items (eg, “pick all responses that apply”) to reduce respondent burden. To interpret which responses are most significant, researchers typically employ either simple statistics, such as frequency counts to order items, or use inferential approaches, like linear regression, to measure importance in relationship to some other factor. Neither approach is ideal. The frequency counts do not provide an accurate picture of how difficult a particular response is vis-à-vis other options. Frequency is about how often something is encountered, not necessarily how important a particular challenge is to the implementation success. The regression approach is limited to addressing the relationship of survey responses to a particular variable of interest rather than allowing them to be context-free and self-referent.

The IRT approach addresses the 2 aforementioned shortcomings and provides an effective complement to those analytic strategies. For those designing surveys, including the IRT approach as part of the analytic strategy should be established a priori. For those analyzing secondary data with "pick all that apply" options, using IRT in conjunction with other techniques could yield a more complete picture of the phenomena.

 Health Policy and Management, Johns Hopkins Bloomberg School of Public Health (KSC, HK, MP, EWF), Baltimore, MD.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

It is unclear which barriers cause the greatest threats to the successful implementation of an electronic health record (EHR). This paper prioritizes the potential threats to EHR adoption using a novel analytic strategy: item response theory.

Assessing Electronic Health Record Implementation Challenges Using Item Response Theory

To estimate the current uses level of ambulatorycomputerized physician order entry (A-CPOE) among physiciansand to examine the relationship of managed care penetration aswell as other market and practice characteristics to use of A-CPOEby physicians.

This study uses both primary and secondary datasources. The primary data source was a large-scale survey of physicians' use of information technologies in Florida. Secondary dataon managed care penetration were obtained from the FloridaAgency for Health Care Administration, and other market-leveldata were extracted from the area resource file.

A hierarchical logistic regression model was used toexamine the correlation of county-level and practice-level characteristicswith physicians' self-reported use of A-CPOE systems.

Overall, 1360 physicians (32.4%) indicated use of anA-CPOE system. Findings suggest that 1% more managed care penetrationwas associated with 2.1% lower use of A-CPOE (

= .003).Additionally, practice size, multispecialty affiliation, and primarycare practice were significantly and positively correlated withthe use of A-CPOE. Physician age was negatively associatedwith A-CPOE use.

Managed care organizations may experience significantfinancial savings from A-CPOE use by physicians; however,managed care penetration in a community negatively affectsA-CPOE use among physicians in their practices. Further studyregarding the causal nature of this association is warranted.

Prescribing of medications for patients occurs frequentlyin ambulatory facilities. An estimated66.8% of clinical encounters in this setting resultin a physician prescribing or continuing a medicationfor a patient.

Furthermore, 82.2% of prescription writingin the ambulatory setting takes place in a physician'sprivate office.

Given the frequency with whichmedications are prescribed, it is important to note thatpreventable adverse drug events are common in theambulatory clinical setting.

Ambulatory computerized physician order entry(A-CPOE), a health information technology (HIT) notedfor its potential ability to reduce medication errors andimprove quality,

is a software application that supportsthe ordering of medications, diagnostic tests,interventions, and referrals by outpatient providers.

Similar to inpatient systems, A-CPOE systems includelab test ordering functions. However, electronic drugprescribing is by far the most common use of A-CPOEsystems.

Analysis has shown that use of A-CPOE can bringsubstantial financial savings to consumers, physicians,and payers. Potential savings have been estimated to beas much as $44.2 billion annually in the United States ifsuch systems were universally adopted.

Additionally,stories in the lay press have highlighted the benefits ofA-CPOE systems to patients and health payers.

As aresult, the national strategic plan for the adoption of HIThas included A-CPOE, and the Medicare ModernizationAct of 2003 has sought to promote the use of A-CPOEsystems.

Compared with the growing literature on hospital-basedCPOE systems,

empirical research on the useof A-CPOE systems is relatively rare and often consistsof case studies. Among the available studies, Schectmanet al found that the use of an A-CPOE system in one academicambulatory clinic was strongly associated withphysicians' attitudes toward the potential benefits of thesystem.

Overhage et al found that after a brief learningperiod, little or no ongoing training was required forphysicians using a well-designed A-CPOE system.

Pizziet al found that early adopters of A-CPOE systems weremore likely to be generalists, have fewer years in practice,and work in technology-equipped offices.

However, this later study's generalizability was limitedfor a variety of reasons, including a narrow sample ofonline users, a very low response rate (3.7%), and the useof descriptive statistical methods that did not control forother important factors (eg, practice characteristics,market variables). To our knowledge, no study has documentedthe organizational factors (eg, practice size,type, specialty) that relate to the use of A-CPOE in privatephysicians' offices. Nor has any study examined therelationship of market-level factors (eg, managed carepenetration, competition) with A-CPOE adoption decisionsin the office setting.

The purpose of this paper is to fill those gaps in theliterature. First, we sought to estimate the currentA-CPOE utilization level among physicians in privatepractices. In particular, we were interested in the functionof electronic prescription order entry. Second, weexamined the correlation of managed care penetration,as well as other market, organizational, and individualfactors, with the use of A-CPOE among physicians. Assuch, this study makes 2 new contributions to the studyof health information technology (IT). First, it establishesa benchmark for A-CPOE adoption that can be usedto measure progress in the diffusion of this technology.Second, it comprehensibly evaluates physicians' A-CPOE utilization levels in the practice context andcompetitive market environments. Such informationwill allow policymakers and physicians to more effectivelytarget their A-CPOE promotion efforts.

The analyses presented herein used both primaryand secondary data sources. The primary data sourcewas a comprehensive large-scale survey (n = 14 921) ofphysicians' use of HITs in Florida. Secondary data onmanaged care penetration were obtained from theFlorida Agency for Health Care Administration, andother market data were extracted from the area resourcefile (ARF). The following sections describe each of thesedata sources, as well as the variables and analyticalapproach used in the study. A list of variables and theirsources is in 

In the spring of 2005, a large-scale survey of physicianspracticing in Florida was conducted. The survey,which was approved by the Florida State Universityinstitutional review board, targeted all primary carephysicians (family doctors, general internists, generalpediatricians, and obstetrician-gynecologists) in thestate. In addition, 25% of the other physicians practicingin the ambulatory setting (eg, medical and surgical specialists,general surgeons, dermatologists, psychiatrists)were randomly selected from the Florida Department ofHealth's list of physicians with clear and active licenses.The survey included questions regarding types of IT (eg,e-mail, personal digital assistants [PDAs], electronichealth records) used by physicians in the office setting.A complete description of the survey methodology,

including an analysis of potential response bias (nonedetected),

was previously published. Briefly, the surveywas developed based on a comprehensive literaturereview and refined with the aid of experts' advice toestablish content and face validity. Before deployment,the survey instrument was cognitively tested with apanel of physicians for clarity and readability. Thisprocess resulted in several iterations before the finalversion of the questionnaire.Numerous vendors and system variants exist forA-CPOE. For example, electronic order entry can occurvia a stand-alone software application on a personalcomputer (PC), a handheld device (eg, a PDA), or anexisting electronic health record (EHR) system.

As mentioned above, the survey asked respondents toindicate whether or not they used a PC, PDA, and/orEHR system in the scope of their practice. Whenresponding in the affirmative to any of these questions,each respondent was further asked to select from a listthe functions that they used on each of these platforms.For example, those using a PDA were asked toindicate by a check mark whether they used their PDAfor things such as drug references, medication interactions,electronic order entry (eg, labs, x-rays), electronicprescriptions, and various other clinical andadministrative functions. In the current study, wewere interested in the A-CPOE functionality regardlessof system variant (eg, PC, PDA, EHR). In addition,given the nature of the data, it was feasible for a givenphysician to have A-CPOE capabilities via more than 1such platform. Therefore, the selected outcome variablefor the current study was physicians' self-reporteduse of at least 1 of these forms of A-CPOE in theiroffice practice.

Physician-level independent variables from the surveywere measured as follows. Practice size was categorizedinto 4 levels representing the number of physicians at agiven practice location (solo, 2-9, 10-49, or 50+). Themedical-training variable was dichotomized to facilitatecomparison between primary care physicians versusother specialists. Last, physicians practicing in rural communitieswere identified by 1 of the following criteria:1) their office was located in one of Florida's 33 statutorilydesignated rural counties, 2) their office waslocated in a rural part of a nonrural county as designatedby the Rural Urban Commuting Area codes,

or3) their address was in the current Health Resourcesand Services Administration list of defined Florida ruralzip codes. Collectively, along with physician demographicinformation, these practice characteristicsallowed for the examination of the factors that may berelated to the use of A-CPOE.

Using the latest available version (2004) of the ARF,various market-level indicators were matched, by county,to the primary data collected via the physician ITsurvey described above. The ARF dataset is compiled bythe Health Resources and Services Administration andcontains county-level information aggregated fromnumerous national sources.

In particular, we wereinterested in managed care penetration rates. However,the current version of the ARF data contained outdated(1999) information for this variable. Assuch, managed care penetration rates forthe first quarter of 2005, for each Floridacounty, were obtained from the FloridaAgency for Health Care Administration.This agency licenses all health insurers andmanaged care organizations in the stateand collects this information for statutorilyrequired reporting purposes. The countylevelmanaged care penetration rate wascalculated by taking the number of residentsenrolled in an HMO and dividing it bythe total number of residents in that county.Higher managed care penetration rateshave been shown to slow the diffusion ofsome medical technologies.

However,less is known about the relationshipbetween managed care penetration and useof HIT among physicians. Theoretically,use of HIT is consistent with the goals ofmanaged care; that is, to improve informationgathering, increase administrative andclinical efficiencies, and promote clinicaleffectiveness. Therefore, a continuouslymeasured market-level (county) managedcare penetration rate was included in theanalysis. Additionally, a number of the documentedbenefits

associated with ACPOEmay make it an attractiveinvestment for physicians, particularlywhen practicing in more competitive environments.To control for competition, wecomputed the county number of physiciansper (1000) capita using the ARF data.Last, we included several other county-levelmeasures to control for environmentalfactors that may be related to A-CPOEuse. These measures included poverty andunemployment rates, percentage of thepopulation that is white/non-Hispanic, percentage of thepopulation age 65 years or older, and newborns as a percentageof the population.

To analyze the data, standard descriptive statisticswere calculated to examine the data for anomalies andto ensure that the assumptions of all analyses were met.To examine the independent relationship of the market-leveland practice-level characteristics to the outcomevariable, a hierarchical logistic regression model wasused. The model utilized the "enter" method for variableinclusion and a 2-block nested approach. The firstblock represented county-level characteristics such aspercent managed care penetration, poverty rate, physiciansper capita, geographic location (rural vs urban),and other county-level demographic information (seeTable 1). The second block represented practice-levelcharacteristics including physician age and sex as wellas practice scope (primary care or specialty practice),size, and type (single specialty vs multispecialty). Thismethod was selected because physician-level and practice-level characteristics are likely influenced by localmarket forces. As such, controlling for these factors in anested model was warranted. It should be noted that thecurrent statistical approach had a cross-sectional designthat cannot assess causality. Therefore, the resultsshould be interpreted as associations. All analyses wereconducted with SPSS version 13.0 (SPSS Inc, Chicago,Ill), and significance was assessed at the 

Overall, 4203 physicians completed the survey andcould be matched to the ARF data, representing a 28.2%response rate. Demographic and practice characteristics,along with descriptive information on the ARF variablesused, appear in Table 1. Respondents' demographicswere consistent with known physician demographicpatterns in Florida.

Briefly, the average age of physicianswas 50.6 years with a range of 30 to 86 years. Themajority of respondents were Caucasian (68.4%) andmale (75.9%), and worked in a single-specialty practice(85.6%).

In descriptive analysis, the presence of an academicmedical center did not correlate with being in a high orlow HMO penetration area (&#967;

= .430). However,those in rural areas (&#967;

< .001) andthose in central Florida (&#967;

< .001) weremore likely to be in low HMO penetration areas.Alternatively, those in the southern part of the statewere more likely to be in high HMO penetration areas(&#967;

A total of 1360 physicians (32.4%) indicated theyused an A-CPOE function in their office practice. Whenall variables in the hierarchical model were controlledfor, 4 factors had a significant independent relationshipwith the use of an A-CPOE system among physicians (see

). Among the county-level variables examined,only the managed care penetration variable was significantlyand negatively correlated with A-CPOE availability.Specifically, 1% more managed care penetration wasassociated with 2.1% less use of A-CPOE (

= .003).Four practice-level characteristics were related toA-CPOE adoption as well. For example, compared withphysicians in solo practice, those in groups of 10 to 49physicians (odds ratio [OR] = 2.78, 

< .001) were significantly morelikely to indicate using A-CPOE. Moreover, those physicianspracticing in primary care (OR = 1.43, 

= .001)or in a multispecialty practice (OR = 1.43, 

= .013)were more likely to use an A-CPOE system. Physiciansex did not seem to have an effect; however, age wasnegatively related to A-CPOE use. Specifically, a 1-yearincrease in age was associated with a 2.5% decreasein the frequency of physicians reporting A-CPOE use(

The literature about CPOE usage in hospitals isextensive

and growing rapidly. However, less isknown about the use of CPOE in ambulatory settings.Taking into consideration the unique nature of ambulatorymedical practice and the importance of A-CPOE inensuring medication safety, the present study examinedthe overall utilization rate of A-CPOE and the factorsthat are related to the adoption of this technology in theoutpatient setting.

Overall, in mid-2005, about one third of physiciansresponding to the Florida survey indicated using anA-CPOE function in their office practice. This level ismarkedly higher than the estimated 2% of all prescriptionswritten in 1999 that were done electronically

andthe reported 19% of physicians that used an A-CPOEsystem in 2003.

The rate for A-CPOE usage amongrespondents is also higher than recent data on CPOEusage (approximately 10%) in the inpatient setting.

Given the more visible pressures from hospital stakeholders(eg, the Leapfrog group) to adopt CPOE, themotivation for A-CPOE may not be as strong. However,the difference in adoption rates between the inpatientand outpatient settings may reflect the less complex, andless costly, installations associated with the ambulatorysetting.

Also, at this point, it is unclear what rolethe e-prescribing adoption incentives from the MedicareModernization Act of 2003 have on A-CPOE systems.With respect to adoption factors, our findings suggestthat managed care penetration in a community appearsto be negatively related to the use of A-CPOE amongphysicians. This trend is consistent with previousresearch that identified a negative relationship betweenmanaged care penetration rates and the use of magneticresonance imaging and other "high tech" services.

Eric W. Ford, PhD, MPH

Articles

Electronic Reminder's Role in Promoting Human Papillomavirus Vaccine Use

Assessing Electronic Health Record Implementation Challenges Using Item Response Theory

Managed Care Penetration and Other Factors Affecting Computerized Physician Order Entry in the Ambulatory Setting