Currently Viewing:
The American Journal of Managed Care Special Issue: Health Information Technology - Guest Editor: Farzad Mostashari, MD, ScM
Health Information Technology: On the Cusp of Healthcare Transformation
Ashish K. Jha, MD, MPH
The Data Revolution Comes to Healthcare
Farzad Mostashari, MD, ScM, Visiting Fellow, Brookings Institute, Former National Coordinator for Health IT, US Department of Health and Human Services
The Impact of Electronic Health Record Use on Physician Productivity
Julia Adler-Milstein, PhD; and Robert S. Huckman, PhD
Face Time Versus Test Ordering: Is There a Trade-off?
James E. Stahl, MD, CM, MPH; Mark A. Drew, BID; Jeffrey Weilburg, MD; Chris Sistrom, MD, MPH, PhD; and Alexa B. Kimball, MD, MPH
Evolving Vendor Market for HITECH-Certified Ambulatory EHR Products
Marsha Gold, ScD; Mynti Hossain, MPP; Dustin R. Charles, MPH; and Michael F. Furukawa, PhD
Financial Effects of Health Information Technology: A Systematic Review
Alexander F. H. Low, MBA; Andrew B. Phillips, RN, PhD; Jessica S. Ancker, MPH, PhD; Ashwin R. Patel, MD, PhD; Lisa M. Kern, MD, MPH; and Rainu Kaushal, MD, MPH
Employing Health Information Technology in the Real World to Transform Delivery
Marsha Gold, ScD
Redesigning the Work of Case Management: Testing a Predictive Model for Readmission
Penny Gilbert, MBA, BSM, BSN, RN, CPHQ; Michael D. Rutland, MBA, FHFMA, FACHE, FABC; and Dorothy Brockopp, PhD, RN
Currently Reading
Small Practices' Experience With EHR, Quality Measurement, and Incentives
Rohima Begum, MPH; Mandy Smith Ryan, PhD; Chloe H. Winther, BA; Jason J. Wang, PhD; Naomi S. Bardach, MD; Amanda H. Parsons, MD; Sarah C. Shih, MPH; and R. Adams Dudley, MD, MBA
Will Meaningful Use Electronic Medical Records Reduce Hospital Costs?
William E. Encinosa, PhD; and Jaeyong Bae, MA

Small Practices' Experience With EHR, Quality Measurement, and Incentives

Rohima Begum, MPH; Mandy Smith Ryan, PhD; Chloe H. Winther, BA; Jason J. Wang, PhD; Naomi S. Bardach, MD; Amanda H. Parsons, MD; Sarah C. Shih, MPH; and R. Adams Dudley, MD, MBA
A study to assess clinician attitudes and experiences after participating in a New York City cardiovascular disease focused quality recognition and financial incentive program using health information technology.
Most respondents (80%) indicated the intent to generate quality reports after the program ended and allocate staff time to focus on QI activities (70%) (Table 2). Incentive clinicians were more likely to report that that they would generate quality reports (87% incentive vs 72% control, P = .07), track practices’ progress toward meeting quality measurement goals (91% vs 78%, P = .09), and hold regular meetings or check-ins (71% vs 57%, P = .14) compared with control clinicians.


Small practice clinicians had positive experiences with the rewards and financial recognition program designed to improve the delivery of clinical preventive services. Clinicians in the incentive group were more likely than those in the control group to report participating in quality improvement activities offered by the program, such as reviewing the quality reports, using order sets, and participating in program training sessions. The high level of buy-in to the program is demonstrated by the reported usability and accuracy of the quality reports and by reported agreement with the ABCS prioritization of preventive cardiovascular care.

Past studies document instances of clinician skepticism about the validity of clinical quality measurements or accuracy of reports, leading to less engagement of clinicians in quality improvement efforts.15,16 In addition, because of the lack of transparency in data used for reporting or payment, some P4P programs have been seen as a threat to clinicians’ autonomy and sense of control.17 The Health eHearts program addressed issues seen in earlier studies by generating reports directly from the practices’ EHRs, offering transparency into the data used for quality measurement, and also by providing QI assistance and help with troubleshooting problem areas with the intent of improving clinician sense of control over measured performance.

Alignment of the program goals with the practice’s organizational structure and culture has been associated with successful P4P implementation.20 The majority of clinicians agreed with the prioritization of the ABCS and found them to be meaningful to their practice. Positive clinician attitude has been associated with successful implementation of EHRs21 and is potentially an important contributor to continued EHR use, especially in small independently owned practices that do not have dedicated staff for quality measurement or EHR-based reporting. 

Robust EHRs can systematize and streamline work flow by allowing clinicians to use key features, such as CDSS.22 However, small practices are less likely to utilize these features.23,24 These survey results suggest that providing QI assistance along with incentives can be effective in engaging clinicians both during a program and potentially for sustaining continued QI activities.


Our study has several limitations. As a self-reported survey, it is subject to social desirability bias whereby clinicians may be inclined to respond positively instead of with criticism. In this study, the differences between the incentive group and the control group answers were likely equally affected by this bias, implying that the differences observed in reported engagement with quality improvement activities would not be affected by this limitation, though the overall experience ratings may be higher than if respondents were not affected by this bias.

It is also possible that the overall ratings of the experience in the program are more positive than the experience for all participants in the program, since some participants did not respond. However, we received a high response rate of 74% and there were few  significant differences in practice characteristics between respondents and nonrespondents.

Further Research

Further research should examine the effect of sustaining QI efforts in the absence of incentives. A recent study using independent data comparing PCIP and non-PCIP comparison practices in New York State also found that technical assistance visits were instrumental in improving quality.25 It is still not clear whether after establishing routine quality measurement, or receipt of QI technical assistance, that practices will sustain these activities. Most respondents indicated intentions of continuing QI work, but fewer responded that they anticipated investing ongoing resources (meetings, staff time). Further study is warranted regarding the sustainability of the intervention and the power of good intentions in the absence of resources.


Incentives may not be necessary to motivate clinicians to participate in a program focusing on increasing the delivery of clinical preventive services. However, practices that received incentives were more likely to report using quality improvement–related  activities. An incentive system implemented in the context of robust information systems may drive use of specific EHR tools or follow-through on quality improvement activities.

As part of the Patient Protection and Affordable Care Act,26 new models of care delivery and reimbursement are being implemented and tested. Ways to facilitate clinician engagement, especially for small independently owned practices, are needed. Our study supports the hypothesis that clinician buy-in and engagement is possible if the program ensures that quality measures reports used in the program are clinically meaningful and that quality reports are relevant and accurate.

Author Affiliations: From Primary Care Information Project (RB, MSR, CHW, JJW, AHP, SCS), New York City Department of Health and Mental Hygiene, Long Island City, NY; Department of Pediatrics (NSB), Department of Internal Medicine (RAD), Philip R. Lee Institute for Health Policy Studies (RAD), University of California San Francisco, San Francisco, CA.

Funding Source: This study was partially funded by the Agency for Healthcare Research and Quality (R18HS018275, R18 HS019164), New York City Tax Levy and Robin Hood Foundation.

Author Disclosures: The authors (RB, MSR, CHW, JJW, NSB, AHP, SCS, RAD) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (RB, MSR, JJW, NSB, SCS, RAD); acquisition of data (RB, MSR, CHW, JJW, SCS); analysis and interpretation of data (RB, MSR, CHW, JJW, NSB, SCS); drafting of the manuscript (RB, MSR, CHW, JJW, AHP, SCS); critical revision of the manuscript for important intellectual content (RB, MSR, CHW, JJW, NSB, AHP, SCS, RAD); statistical analysis (RB, MSR, JJW); obtaining funding (AHP); administrative, technical, or logistic support (RB, MSR, CHW, JJW, SCS); and supervision (MSR, JJW, SCS, RAD).

Address correspondence to: Sarah C. Shih, MPH, New York City Department of Health and Mental Hygiene, Primary Care Information Project, 42-09 28th St, 12th Fl, Queens, NY 11101. E-mail:
1. Grossbart SR. What’s the return? assessing the effect of “pay-forperformance” initiatives on the quality of care delivery. Med Care Res Rev. 2006;63(1 suppl):29S-48S.

2. Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Eng J Med. 2007; 356(5):486-496.

3. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay for performance on patient outcomes. N Eng J Med. 2012; 366(17):1606-1615.

4. Ryan AM. Effects of the premier hospital quality incentive demonstration on Medicare patient mortality and cost. Health Serv Res. 2009;44(3): 821-842.

5. Ryan AM, Blustein J, Casalino LP. Medicare’s flagship test of pay-for-performance did not spur more rapid quality improvement among low-performing hospitals. Health Aff (Millwood). 2012;31(4):797-805.

6. Werner RM, Dudley RA. Medicare’s new hospital value-base purchasing program is likely to have only a small impact on hospital payments. Health Aff (Millwood). 2012;31(9):1932-1940.

7. Van Herck P, De Smedt D, Annemans L, et al. Systematic review: effects, design choices, and context of pay-for-performance in health care. BMC Health Serv Res. 2010;10:247.

8. Scott A, Sivey P, Ait Ouakrim D, et al. The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane Database Syst Rev. 2011;(9):CD008451.

9. Chung S, Palaniappan LP, Trujillo LM, Rubin HR, Luft HS. Effect of physician-specific pay-for-performance incentives in a large group practice. Am J Manag Care. 2010;16(2):e35-e42.

10. Chung S, Palaniappan L, Wong E, Rubin H, Luft H. Does the frequency of pay-for-performance payment matter? experience from a randomized trial. Health Serv Res. 2010;45(2):553-564.

11. Rao SR, Desroches CM, Donelan K, Campbell EG, Miralles PD, Jha AK. Electronic health records in small physician practices: availability, use, and perceived benefits. J Am Med Inform Assoc. 2011;18(3): 271-275.

12. Tollen LA. Physician organization in relation to quality and efficiency of care: a synthesis of recent literature. The Commonwealth Fund. 2008;(89).

13. Crosson FJ. The delivery system matters. Health Aff (Millwood). 2005; 24(6):1543-1548.

14. Young GJ, Meterko M, White B, et al. Physician attitude towards pay-for-quality programs: perspectives from the front line. Med Care Res Rev. 2007;64:331-343.

15. Casalino LP, Alexander GC, Jin L, Konetzka RT. General internists’ views on pay-for-performance and public reporting of quality scores: a national survey. Health Aff (Millwood). 2007;26(2):492-499.

16. Pham HH, Bernabeo EC, Chesluk BJ, Holmboe ES. The roles of practice systems and individual effort in quality performance. BMJ Qual Saf. 2011;20(8):704-710.

17. Epstein AM, Lee TH, Hamel MB. Paying physicians for high-quality care. N Engl J Med. 2004;350(4):406-410.

18. Medicare and Medicaid Programs; Electronic Health Record Incentive  Program; Final Rule 42370 CFR Parts 412, 413, 422 et al. 2010;75:44314-44588.

19. Bardach NS, Wang JJ, De Leon SF, et al. Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: a randomized trial. JAMA. 2013;310(10):1051-1059.

20. Young GJ, Beckman H, Baker E. Financial incentives, professional values and performance: a study of pay-for-performance in a professional organization. J Organiz Behav. 2012;33:964-983.

21. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.

22. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10):742-752.

23. DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care: a national survey of physicians. N Engl J Med. 2008;359(1):50-60.

24. Simon SR, Kaushal R, Cleary PD, et al. Physicians and electronic health records: a statewide survey. Arch Intern Med. 2007;167(5): 507-512.

25. Ryan AM, Bishop TF, Shih S, Casalino LP. Small physician practices in New York needed sustained help to realize gains in quality from use of electronic health records. Health Aff (Millwood). 2013;32(1):53-62.

26. The Patient Protection Affordable Care Act.
Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up