Electronic Medical Records and Efficiency and Productivity During Office Visits

The American Journal of Managed Care, April 2011, Volume 17, Issue 4

The relationship between electronic medical record use and efficiency/productivity during office visits varied by type of service and reason for visit.

Objective:

To estimate the relationship between electronic medical record (EMR) use and efficiency of utilization and provider productivity during visits to US office-based physicians.

Study Design:

Cross-sectional analysis of the 2006-2007 National Ambulatory Medical Care Survey.

Methods:

The sample included 62,710 patient visits to 2625 physicians. EMR systems included demographics, clinical notes, prescription orders, and laboratory and imaging results. Efficiency was measured as utilization of examinations, laboratory tests, radiology procedures, health education, nonmedication treatments, and medications. Productivity was measured as total services provided per 20-minute period. Survey-weighted regressions estimated association of EMR use with services provided, visit intensity/duration, and productivity. Marginal effects were estimated by averaging across all visits and by major reason for visit.

Results:

EMR use was associated with higher probability of any examination (7.7%, 95% confidence interval [CI] = 2.4%, 13.1%); any laboratory test (5.7%, 95% CI = 2.6%, 8.8%); any health education (4.9%, 95% CI = 0.2%, 9.6%); and fewer laboratory tests (−7.1%, 95% CI = −14.2%, −0.1%). During pre/post surgery visits, EMR use was associated with 7.3% (95% CI= −12.9%, −1.8%) fewer radiology procedures. EMR use was not associated with utilization of nonmedication treatments and medications, or visit duration. During routine visits for a chronic problem, EMR use was associated with 11.2% (95% CI = 5.7%, 16.8%) more diagnostic/screening services provided per 20-minute period.

Conclusions:

EMR use had a mixed association with efficiency and productivity during office visits. EMRs may improve provider productivity, especially during visits for a new problem and routine chronic care.

(Am J Manag Care. 2011;17(4):296-303)

Electronic medical record (EMR) use had a mixed association with efficiency and productivity during office visits, and the relationships varied by type of service and by the major reason for the visit.

  • EMR users had higher intensity and productivity of diagnostic/screening services, especially during visits for a new problem and routine visits for a chronic problem.

  • Use of EMRs may alter the content of office visits and improve a provider’s productivity, which might lead to cost savings and quality improvements.

  • Contrary to expectation, EMR use had no association with efficiency or productivity during visits for preventive care.

Health information technology is widely believed to enhance quality of care and patient safety and to lower costs and improve efficiency.1,2 Despite these potential benefits, adoption of electronic medical records (EMRs) by US physicians in office settings has been slow.3,4 Uncertain financial return and loss in productivity are often cited as barriers to adoption.3,5 Nonetheless, surveys report that physicians perceive that EMR use can improve work flow, the quality of clinical decisions, and the delivery of preventive care.3,5

Empirical studies demonstrating the impact of EMR systems on the efficiency and quality of care in ambulatory settings have been limited.6,7 Most prior work has focused on the relationships between specific EMR functions and medication safety8 and quality of care.9,10 In contrast, relatively few studies have examined the association of EMR with efficiency of utilization and provider productivity.2,6,7 The economic benefits of integrated EMR functionality from commercial systems used in community settings remains uncertain.11

This study examined the association between EMR use and efficiency and productivity during office visits using a large-scale, nationally representative data set. The findings from this study provide important evidence of the value of health information technology in ambulatory care.

BACKGROUND

Electronic Medical Record Use and Efficiency of Utilization

In theory, EMR use has the potential to improve efficiency of utilization. An EMR system may include clinical notes, problem/medication lists, and test results. These functions can provide information about chronic conditions and prior utilization, which might reduce redundant and inappropriate diagnostic/screening services and medications. Computer-generated care suggestions and automated reminders could improve adherence to evidence-based guidelines and might increase provision of some services. Thus, in theory, EMR might increase or decrease utilization of services depending on the EMR functionality and the reason for visit.1

Prior studies of the association of EMR use with the efficiency of utilization have been limited.6,7,12,13 Some evidence suggests that EMR use can improve care for chronic illness and preventive care.14-19 However, 3 large-scale studies found little relationship between EMR use and quality of care in ambulatory settings.20-22 Whether and to what extent that EMR use is associated with the level of utilization during visits remains uncertain.

Electronic Medical Record Use and Provider Productivity

An EMR system can automate manual tasks, streamline documentation, and improve access to information. These EMR functions can support clinical decision making and might reduce physician time, at least in theory.1 Improvements in productivity could allow physicians to see more patients per day or to provide more services to the same patient during each visit.

Evidence of the impact of EMR use on provider time efficiency is mixed,23 and few studies have examined the relationship between EMR use and visit duration in ambulatory settings.24,25 Prior studies of the association of EMR use with provider productivity have also been limited, and study designs have varied in their unit of analysis.26-35 Whether and to what extent that EMR use is associated with visit intensity, duration, and productivity per visit remains an open question.

METHODS

Data Source

The study used data from the public use version of the National Ambulatory Medical Care Survey (NAMCS) from 2006 and 2007. Conducted by the National Center for Health Statistics of the Centers for Disease Control and Prevention, the NAMCS is a survey of visits to US nonfederal office—based physicians. The multistage probability design is based on a random sample of physicians stratified by geographic area and specialty. Patient visits during a randomly selected week were sampled for each participating physician. Patient characteristics, reason for visit, and utilization of medical services were reported on individual patient record forms. Information about the physician and their practice, including EMR use, were captured during a separate intake survey. The NAMCS included weights for visits and physicians that allowed for the generation of nationally representative estimates. The 2006 and 2007 surveys collected information on 62,170 visits to 2625 physician respondents, and the full sample was included in the analysis.

Electronic Medical Record Use

Table 1

The main variable of interest is whether the physician’s practice used an EMR system. The NAMCS asked “Does this practice use electronic medical records (not including billing records)?” The survey also asked whether the practice’s EMR system or another computerized system included any of 13 specific EMR functions (). Nonresponse and survey responses of “turned off” and “unknown” were included as not having the EMR system or function. Based on definitions developed by a consensus panel,4,36 EMR use in this study was defined as a Basic or Fully Functional EMR system with at least the minimum set of functions.

Efficiency of Utilization and Provider Productivity

Table 2

The NAMCS collected details on the number and type of medical services ordered or provided during each visit. The NAMCS allowed physicians to report 2 additional diagnostic/screening services, which were classified by International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) procedure code. These services were included in laboratory tests if the ICD-9-CM code started with 90 or 91 and in radiology procedures if the ICD-9-CM code started with 87, 88, or 92. Measures of efficiency of utilization and provider productivity during office visits were specified based on prior literature11,37,38 and are reported in .

Major Reason for Visit

The NAMCS captured information about the patient’s major reason for visit, and each visit was classified into 1 of 5 visit types. “New problem” included visits for conditions that occurred within 3 months of the visit. “Routine visit for a chronic problem” included visits to receive care or examination for a preexisting chronic condition, illness, or injury that occurred more than 3 months prior to the visit. “Preventive care” included visits for general medical examinations and routine periodic examinations. “Flare-up of a chronic problem” included visits primarily due to sudden exacerbation of a preexisting chronic condition. “Pre/post surgery” included visits scheduled primarily for care required prior to or following surgery.

Analysis

A cross-sectional analysis of pooled survey data was conducted. Two-part models of medical care utilization were specified to analyze the association of EMR use with efficiency of utilization.39 This allowed separate analyses of the relationship of EMR use with the probability of any use and the number of services provided, conditional on some utilization. All productivity measures had a highly skewed distribution and were log-transformed to approximate a normal distribution. Analyses were conducted by averaging across all visits and by major reason for visit.

eAppendix A

Estimation was performed using Stata 10.1 software that accounted for the complex survey design. Survey-weighted probit, Poisson, and ordinary least squares regressions included patient, physician, and practice characteristics reported in at www.ajmc.com. Marginal effects from probit/ordinary least squares regressions and semielasticities from Poisson regressions were calculated, and results can be interpreted as the percent change in the dependent variable associated with EMR use.

RESULTS

eAppendix B

Descriptive statistics for patient, physician, and practice characteristics are reported in eAppendix A. Descriptive statistics for efficiency of utilization and provider productivity during office visits are presented in at www.ajmc.com.

Electronic Medical Record Use in Physician Offices

Table 1 presents nationally representative estimates of physician use of EMR systems and specific EMR functions. In 2006-2007, 32.5% of US office-based physicians reported the use of any EMR system in their practice. On average, EMR systems included 7.25 out of 13 functions. Conditional on some EMR use, patient demographic information (90.2%) and clinical notes (78.4%) were the most commonly used EMR functions. Test orders sent electronically (33.7%) and electronic images returned (28.5%) were the least commonly used.

Although one-third of physicians used any EMR, only 10.9% of US office-based physicians reported the use of a Basic or Fully Functional EMR system, which included the minimum set of functions. On average, Basic/Fully Functional EMR systems included 11.11 out of 13 functions, with 94.2% having medical history and follow-up notes and 85.2% having computerized orders for tests.

Table 3

presents regression results of physician and practice characteristics associated with EMR use. Specialty, geographic region, practice size, ownership, and electronic billing/claims submission were significant predictors of EMR use. Relative to general/family practice, physicians in psychiatry (−7.7%) and pediatrics (−4.2%) had a lower probability of EMR use. Relative to the Midwest region, practices in the Northeast region were 4.7% less likely to use an EMR system. Solo practitioners had a 5.6% lower probability of EMR use than physicians in partnerships/group practices. Relative to practices owned by a physician/physician group, practices owned by an HMO and practices owned by a corporation/other were 43.8% and 12.2% more likely, respectively, to use EMRs. Practices with electronic billing/claims submission had a 5.4% higher probability of EMR use. Metropolitan status, the number of managed care contracts, and percentage of revenue from managed care had no significant association with EMR use.

Patients of EMR users differed from nonusers in some characteristics (eAppendix A). Patient age, race, chronic conditions, and insurance status were different for physicians with EMR use. Patients of EMR users were less likely to be children (aged 17 years and under) and to have Medicaid or self-pay insurance, but were more likely to be aged 18 to 44 years, to be other race, and to have hyperlipidemia, diabetes, and private insurance.

Electronic Medical Record Use and Efficiency of Utilization

Table 4

presents regression results of the association of EMR use with efficiency of utilization during office visits. Use of EMRs had a strong relationship to any use of diagnostic/screening services. On average, EMR users had a higher probability of providing any examination ( 7.7%) and any laboratory test ( 5.7%), especially during routine visits for a chronic problem and visits for a new problem.

eAppendix C

Use of EMRs was associated with fewer laboratory tests and radiology procedures conditional on some utilization. Electronic medical record users provided 7.1% fewer laboratory tests on average across all visits. During visits for pre/post surgery, EMR use was associated with 7.3% fewer radiology procedures ( at www.ajmc.com).

Electronic medical record use had very little association with interventions/medications provided during visits. On average, EMR use was associated with a 4.9% higher probability of any health education. However, no association was found between EMR use and the number of health education interventions provided. Furthermore, EMR use had no significant relationships with utilization of nonmedication treatments or medications.

Overall, EMR use had a mixed association with utilization, and the relationships varied by type of service and by major reason for visit. Surprisingly, EMR use had little to no association with utilization during visits for preventive care, flare-up of a chronic problem, or pre/post surgery.

Electronic Medical Record Use and Provider Productivity

Table 5

presents regression results of the association of EMR use with provider productivity during office visits. Electronic medical record use had a strong relationship with visit intensity during routine visits for a chronic problem. During these visits, EMR users provided 11.2% more total services and 15.3% more diagnostic/screening services per visit.

Electronic medical record use was associated with higher intensity of diagnostic/screening services per visit. Electronic medical record users provided 8.7% more total diagnostic/screening services on average and 12.3% more during visits for a new problem.

No significant association was found between EMR use and the time spent with a provider, on average or by major reason for visit.

Electronic medical record use was also associated with greater productivity during a visit. On average, EMR users provided 7.5% more total services and 9.9% more diagnostic/screening services per 20-minute period. The relationship between EMR use and productivity was strongest during visits for a new problem and routine visits for a chronic problem. During these visits, EMR users provided 9.2% more total services and 11.2% to 12.7% more diagnostic/screening services per 20-minute period.

Overall, EMR use had some association with higher provider productivity that varied by the type of service and by major reason for visit. However, EMR use had no relationship with visit duration; the productivity of interventions/medications provided per visit; nor during visits for preventive care, pre/post surgery, and flare-up of a chronic problem.

DISCUSSION

Electronic medical record use had mixed associations with efficiency of utilization and provider productivity during visits to US office-based physicians. Importantly, the relationships between EMR and utilization differed by type of service and by major reason for visit. Electronic medical record use had a strong relationship with diagnostic/screening services during visits for a new problem and chronic care. However, EMR use had little to no association with interventions/medications during visits for pre/post surgery and preventive care.

Electronic medical record use was associated with higher use of any examination, any laboratory test, and any health education, and with lower utilization of laboratory tests and radiology procedures during some visits. These findings are consistent with studies that found that EMR increased the provision of specific laboratory tests18 and counseling.17 However, other studies have not found conclusive evidence that EMR reduced the number of laboratory tests and radiology procedures.12

Electronic medical record use was also associated with higher visit intensity and productivity, especially during routine visits for a chronic problem and visits for a new problem. Case studies at single institutions have found that revenue per visit increased by 9.8% to 13.7% after EMR implementation.34,35 Other studies have found relationships between EMR and higher relative value units per month.33,35

The lack of a relationship during visits for preventive care was surprising but consistent with the literature. Prior research on computer-generated reminders for preventive care has shown modest effectiveness,15 and national studies using NAMCS have shown little relationship between EMR use and provision of specific quality indicators.20-22 These unexpected findings could reflect the unintended consequences of poor implementation, ineffective training, or resistance to change.40 They could also be due to heterogeneity in the usability and maturity of EMR functions, which lacked certification during this time frame.41 Another explanation related to the level of utilization during visits is that preventive services, particularly health education, may be provided during visits for chronic problems, rather than during general examinations.42

The lack of a relationship with use of medications might suggest that EMR has a stronger impact on the choice of preferred brands and generics43 than on the decision to alter the number of medications prescribed.

Implications for Policy and Practice

Several findings have important implications for policy and practice. First, EMR use was associated with variation in the content of office visits. The content of visits is important because of time pressures on physicians44 and the underprovision of preventive care.45,46 Greater provision of any diagnostic/ screening services suggests that EMR might facilitate the tailoring of specific services to each patient, which could improve quality of care.

Second, EMR use was associated with reductions in laboratory tests and radiology procedures. To the extent that lower utilization reflects avoidance of redundant or inappropriate services, fewer laboratory tests and radiology procedures might result in net cost savings. While cost savings might improve physician return on investment under capitation,11,47 these benefits are more likely to accrue to payers than to physicians.

Finally, EMR might improve the productivity of provider time during visits. Practices using EMR provided more total services without a significant increase in the duration of the office visit. This suggests that EMR use may allow physicians and their staff to provide more services during a fixed length of visit, which may be constrained by a daily schedule. Electronic medical record use might free up time to provide more direct patient care, particularly time-intensive activities such as examinations and health education, or it might improve coordination within the office practice.

Limitations

The study had some limitations. First, a causal relationship between EMR use and efficiency/productivity could not be inferred. The cross-sectional analysis examined the association between EMR use and utilization at 2 points in time. Since the NAMCS lacked a longitudinal design, it was not possible to examine EMR implementation in relation to changes in the dependent variables over time.

Second, omitted variables and selection bias may confound the results. While the regressions controlled for many patient, physician, and practice characteristics, EMR users and their patients might differ in characteristics and context that could not be observed in the data. Greater EMR adoption and use has been associated with organizational characteristics,48,49 care management processes and practice systems,50,51 quality improvement,52 and financial incentives.53 Their presencemay influence utilization apart from the effects of technology, and the estimates may be biased from these confounding factors.

Third, self-reported survey data have inherent limitations, and the NAMCS may have overestimated visit duration and underreported the provision of health education.54 The NAMCS also lacked information to determine the status of EMR implementation, software maturity and system integration, the extent of organizational change and work flow redesign, and user acceptance and actual usage.

Finally, the study did not consider related questions of importance. First, the study did not consider the appropriateness of services provided, nor did it quantify cost savings associated with reductions in utilization. Second, the study was unable to examine the impact of EMR on ancillary staffing and whether associated costs and benefits yielded a positive return on investment.11 Finally, the study did not examine whether EMR was associated with improvements in patient health outcomes, patient safety and reduction in preventable errors, or physician and patient satisfaction. These are important subjects for further research.

Author Affiliations: From the Department of Information Systems, W. P. Carey School of Business, Arizona State University, Tempe, AZ.

Funding Source: None.

Author Disclosure: The author reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis.

Address correspondence to: Michael F. Furukawa, PhD, Assistant Professor, Department of Information Systems, W. P. Carey School of Business, Arizona State University, PO Box 874606, Tempe, AZ 85287. E-mail: michael.furukawa@asu.edu.

1. Hillestad R, Bigelow J, Bower A, et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff (Millwood). 2005;24(5):1103-1117.

2. Blumenthal D, DesRoches C, Foubister V. Health Information Technology in the United States: Where We Stand. Princeton, NJ: Robert Wood Johnson Foundation; June 2008.

3. DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care—a national survey of physicians. N Engl J Med. 2008;359(1):50-60.

4. Hing E, Hall MJ, Ashman JJ. Use of electronic medical records by ambulatory care providers: United States, 2006. Natl Health Stat Report. 2010;(22):1-21.

5. Gans D, Kralewski J, Hammons T, Dowd B. Medical groups’ adoption of electronic health records and information systems. Health Aff (Millwood). 2005;24(5):1323-1333.

6. Chaudhry B, Wang J, Wu SY, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742-752.

7. Hagan S. Evidence on the Costs and Benefits of Health Information Technology. Washington, DC: Congressional Budget Office; May 2008.

8. Eslami S, Abu-Hanna A, De Keizer NF. Evaluation of outpatient computerized physician medication order entry systems: a systematic review. J Am Med Inform Assoc. 2007;14(4):400-406.

9. Delpierre C, Cuzin L, Fillaux J, Alvarez M, Massip P, Lang T. A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach? Int J Qual Health Care. 2004;16(5):407-416.

10. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.

11. Wang SJ, Middleton B, Prosser LA, et al. A cost-benefit analysis of electronic medical records in primary care. Am J Med. 2003;114(5): 397-403.

12. Georgiou A, Williamson M, Westbrook JI, Ray S. The impact of computerised physician order entry systems on pathology services: a systematic review. Int J Med Inform. 2007;76(7):514-529.

13. Garrido T, Jamieson L, Zhou Y, Wiesenthal A, Liang L. Effect of electronic health records in ambulatory care: retrospective, serial, cross sectional study. Br Med J. 2005;330(7491):581.

14. Dorr D, Bonner LM, Cohen AN, et al. Informatics systems to promote improved care for chronic illness: a literature review. J Am Med Inform Assoc. 2007;14(2):156-163.

15. Dexheimer JW, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D. Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2008;15(3): 311-320.

16. Jimbo M, Nease DE, Ruffin MT 4th, Rana GK. Information technology and cancer prevention. CA Cancer J Clin. 2006;56(1):26-36.

17. Adams WG, Mann AM, Bauchner H. Use of an electronic medical record improves the quality of urban pediatric primary care. Pediatrics. 2003;111(3):626-632.

18. O’Connor PJ, Crain AL, Rush WA, Sperl-Hillen JM, Gutenkauf JJ, Duncan JE. Impact of an electronic medical record on diabetes quality of care. Ann Fam Med. 2005;3(4):300-306.

19. Crosson JC, Ohman-Strickland PA, Hahn KA, et al. Electronic medical records and diabetes quality of care: results from a sample of family medicine practices. Ann Fam Med. 2007;5(3):209-215.

20. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400-1405.

21. Keyhani S, Hebert PL, Ross JS, Federman A, Zhu CW, Siu AL. Electronic health record components and the quality of care. Med Care. 2008; 46(12):1267-1272.

22. Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. [published online ahead of print January 24, 2011]. Arch Intern Med.

23. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(5):505-516.

24. Pizziferri L, Kittler AF, Volk LA, et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. 2005;38(3):176-188.

25. Lo HG, Newmark LP, Yoon C, et al. Electronic health records in specialty care: a time-motion study. J Am Med Inform Assoc. 2007;14(5): 609-615.

26. Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente’s experience of implementing an electronic medical record: a qualitative study. Br Med J. 2005;331(7528):1313-1316.

27. Schade CP, Sullivan FM, de Lusignan S, Madeley J. e-Prescribing, efficiency, quality: lessons from the computerization of UK family practice. J Am Med Inform Assoc. 2006;13(5):470-475.

28. Evans DC, Nichol WP, Perlin JB. Effect of the implementation of an enterprise-wide Electronic Health Record on productivity in the Veterans Health Administration. Health Econ Policy Law. 2006;1(pt 2):163-169.

29. Thompson D, Classen D, Garrido T, Bisordi J, Novogoratz S, Zywiak W. The value of vendor-reported ambulatory EHR benefits data. Healthc Financ Manage. 2007;61(4):82-86.

30. Chen C, Garrido T, Chock D, Okawa G, Liang L. The Kaiser Permanente Electronic Health Record: transforming and streamlining modalities of care. Health Aff (Millwood). 2009;28(2):323-333.

31. Welch WP, Bazarko D, Ritten K, Burgess Y, Harmon R, Sandy LG. Electronic health records in four community physician practices: impact on quality and cost of care. J Am Med Inform Assoc. 2007;14(3):320-328.

32. Miller RH, West C, Brown TM, Sim I, Ganchoff C. The value of electronic health records in solo or small group practices. Health Aff (Millwood). 2005;24(5):1127-1137.

33. Clayton PD, Narus SP, Bowes WA 3rd, et al. Physician use of electronic medical records: issues and successes with direct data entry and physician productivity. AMIA Annu Symp Proc. 2005:141-145.

34. Patil M, Puri L, Gonzalez CM. Productivity and cost implications of implementing electronic medical records into an ambulatory surgical subspecialty clinic. Urology. 2008;71(2):173-177.

35. Cheriff AD, Kapur AG, Qiu M, Cole CL. Physician productivity and the ambulatory EHR in a large academic multi-specialty physician group. Int J Med Inform. 2010;79(7):492-500.

36. Hing ES, Burt CW, Woodwell DA. Electronic medical record use by office-based physicians and their practices: United States, 2006. Adv Data. 2007;(393):1-7.

37. Hsiao CJ, Cherry DK, Beatty PC, Rechtsteiner EA. National Ambulatory Medical Care Survey: 2007 summary. Natl Health Stat Report. 2010; (27):1-32.

38. Glied S, Zivin JG. How do doctors behave when some (but not all) of their patients are in managed care? J Health Econ. 2002;21(2):337-353.

39. Mullahy J. Much ado about two: reconsidering retransformation and the two-part model in health econometrics. J Health Econ. 1998;17(3): 247-281.

40. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized physician order entry. J Am Med Inform Assoc. 2006;13(5):547-556.

41. McDonnell C, Werner K, Wendel L. Electronic Health Record Usability: Vendor Practices and Perspectives. Rockville, MD: Agency for Healthcare Research and Quality; May 2010. AHRQ publication 09(10)-0091-3-EF.

42. Mehrotra A, Zaslavsky AM, Ayanian JZ. Preventive health examinations and preventive gynecological examinations in the United States. Arch Intern Med. 2007;167(17):1876-1883.

43. Fischer MA, Vogeli C, Stedman M, Ferris T, Brookhart MA, Weissman JS. Effect of electronic prescribing with formulary decision support on medication use. Arch Intern Med. 2008;168(22):2433-2439.

44. Chen LM, Farwell WR, Jha AK. Primary care visit duration and quality: does good care take longer? Arch Intern Med. 2009;169(20):1866-1872.

45. Pham HH, Schrag D, Hargraves JL, Bach PB. Delivery of preventive services to older adults by primary care physicians. JAMA. 2005;294(4): 473-481.

46. Franks P, Fiscella K, Meldrum S. Racial disparities in the content of primary care office visits. J Gen Intern Med. 2005;20(7):599-603.

47. Cronquist Christensen M, Remler D. Information and communications technology in chronic disease care: what are the implications for payment? Med Care Res Rev. 2007;64(2):123-147.

48. Mehrotra A, Epstein AM, Rosenthal MB. Do integrated medical groups provide higher-quality medical care than individual practice associations? Ann Intern Med. 2006;145(11):826-833.

49. Landon BE, Normand SL, Meara E, et al. The relationship between medical practice characteristics and quality of care for cardiovascular disease. Med Care Res Rev. 2008;65(2):167-186.

50. Casalino L, Gillies RR, Shortell SM, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. JAMA. 2003;289(4): 434-441.

51. Solberg LI, Scholle SH, Asche SE, et al. Practice systems for chronic care: frequency and dependence on an electronic medical record. Am J Manag Care. 2005;11(12):789-796.

52. Gilmer TP, O’Connor PJ, Rush WA, et al. Impact of office systems and improvement strategies on costs of care for adults with diabetes. Diabetes Care. 2006;29(6):1242-1248.

53. Furukawa MF, Ketcham JD, Rimsza ME. Physician practice revenues and use of information technology in patient care. Med Care. 2007;45(2): 168-176.

54. Gilchrist VJ, Stange KC, Flocke SA, McCord G, Bourguet CC. A comparison of the National Ambulatory Medical Care Survey (NAMCS) measurement approach with direct observation of outpatient visits. Med Care. 2004;42(3):276-280.