A study to assess clinician attitudes and experiences after participating in a New York City cardiovascular disease focused quality recognition and financial incentive program using health information technology.
Published Online: November 21, 2013
Rohima Begum, MPH; Mandy Smith Ryan, PhD; Chloe H. Winther, BA; Jason J. Wang, PhD; Naomi S. Bardach, MD; Amanda H. Parsons, MD; Sarah C. Shih, MPH; and R. Adams Dudley, MD, MBA
Objectives: To assess clinician attitudes and experiences in Health eHearts, a quality recognition and financial incentive program using health information technology.
Study Design: Survey of physicians.
Methods: A survey was administered to 140 lead clinicians at each participating practice. Survey domains included clinicians’ experiences and attitudes toward the selected clinical quality measures focused on cardiovascular care, use of electronic health records (EHRs), technical assistance visits, quality measurement reports, and incentive payments. Responses were compared across groups of practices receiving financial incentives with those in the control (no financial rewards).
Results: Survey response rate was 74%. The majority of respondents reported receiving and reviewing the quality reports (89%), agreed with the prioritization of measures (89%), and understood the information given in the quality reports (95%). Over half of the respondents had a quality improvement visit (56%), with incentive clinicians more likely to have had a visit compared with the control group (68% vs 43%, P = .01). The incentive group respondents (92%) were more likely to report using clinical decision support system alerts than control group respondents (82%, P = .11).
Conclusions: Clinicians in both incentive and control groups reported positive experiences with the program. No differences were detected between groups regarding agreement with selected clinical measures or their relevance to the patient population. However, clinicians in the incentive group were more likely to review quarterly performance reports and access quality improvement visits. Incentives may be used to further engage clinicians operating in small independently owned practices to participate in quality improvement activities.
Am J Manag Care. 2013;19(11 Spec No. 10):eSP12-eSP18
With adequate technical support, small practices can be engaged in recognition and financial rewards programs.
Clinician buy-in to the design of the program was high. A majority of the clinicians reported receiving, reviewing, and understanding the quality reports; were in agreement with the focus on cardiovascular quality measures; thought the measures were clinically meaningful; and understood the information.
Financially incentivized clinicians were slightly more engaged and participated in quality improvement visits and trainings, such as using clinical decision support systems and other electronic health record functionalities.
Use of incentives and pay-for-performance (P4P) to realign payment to address problems of low quality of care or gaps in preventive services has had limited success in improving the quality of healthcare.1-6 For the most part, studies on P4P have focused on large group practices.7-10 Small practices, where the majority of patients still receive care nationally,11 historically face greater obstacles to improving care because they have lacked the scale and organizational structure to conduct quality improvement activities or participate in P4P.12,13
It is important to assess clinician attitudes toward key program features, such as the selection of target quality measures, trust in performance reports, and relevance of quality targets. Understanding clinician motivations and opinions toward a quality improvement program may help predict the extent to which they change their clinical behavior.14 Specific program features, such as the frequency and type of performance feedback and available assistance for meeting program goals, could potentially affect clinician awareness and understanding of particular programs. Clinician skepticism about the accuracy of reports, or distrust of or lack of transparency in data used for reporting or payment, may lead to less engagement of clinicians in incentive programs or quality improvement efforts.15-17
With widespread implementation of electronic health records (EHRs),18 EHR-enabled solo and small group practices have been shown to be capable of responding to quality improvement (QI) initiatives, as well as programs that incentivize using quality measurement.19 It is unknown how clinicians will feel about quality measurement and pay-forperformance using EHR-derived quality measures. To address this gap in the literature, we surveyed clinicians participating in Health eHearts, a cluster-randomized trial of the effect of a financial incentive and QI assistance program on measures of cardiovascular care compared with the effect of providing quality reports and QI assistance. The Primary Care Information Project (PCIP), a bureau of the New York City Department of Health and Mental Hygiene, piloted Health eHearts in practices that recently adopted an EHR and that were receiving ongoing QI visits to improve practice work flows using health information technology. Survey domains included overall experience with the program, as well as experience with the tools supporting QI efforts. In addition, we assessed whether there were differences in experiences or attitudes and whether these attitudes differed for practices receiving incentives or not.
Practice Selection and Assignment
PCIP recruited 140 small practices to participate in Health eHearts. The program duration was April 2009 to September 2011. Practices were eligible if they have been “live” on the EHR for at least 3 months, had a minimum of 200 patients with cardiovascular diagnoses related to the quality measurement targets, and were transmitting quality measures through the EHR to PCIP. Practices agreed to be randomized into “recognition” or “rewards” groups. Rewards consisted of financial incentives for each numerator met for 4 areas of cardiovascular care: aspirin therapy, blood pressure control, cholesterol control, and smoking cessation intervention (ABCS). Incentive amounts ranged from $20 to $150 per patient with goal achieved, with higher payments for harder to treat patients (eg, comorbid diseases or lower socioeconomic status). The recognition group served as a control. Both groups (control and incentive) received quarterly quality performance reports, telephone and onsite coaching on work flow redesign, and training on documentation, and were invited to a recognition program at the end of the year. The quality reports summarized practices’ progress on the ABCS and compared their performance with other practices in Health eHearts and trends over the previous 6 months.
Survey Administration and Instrument
Health eHearts was a 2-year program, with cohort 1 enrolled at the beginning and continuing for 2 years and cohort 2 enrolled at the beginning of year 2. Practices were surveyed before and after each program year. This study focuses on the survey administered to all participating practices at the end of Health eHearts. A 33-item survey (29 items in the control group version) was administered in October 2011. A lead clinician from each practice was invited to respond to the survey first by mail, followed by at least 3 reminder phone calls to nonresponding clinicians. Survey administration continued through February 2012.
The instrument was developed in collaboration between PCIP and researchers from University of California San Francisco (UCSF) who were contracted as evaluators for the overall evaluation of the program. The instrument focused on several aspects of the Health eHearts program: clinicians’ experiences and attitudes toward the selected quality measures (ABCS), training on use of the EHR or achievement of ABCS, QI visits, tracking patients for preventive services using the EHR, quality reports, incentive payments (incentive group only), recognition programs in general, and demographics. The survey was pretested with program staff and a clinician in PCIP. Items used in this survey were based on an earlier instrument co-developed with UCSF to assess barriers and facilitators for small practices to participate in P4P. Topics identified as barriers included: accuracy and regularity of reports relevant to the practice’s patient population, measurement targets that were meaningful to the practice population, availability of training or assistance to conduct QI activities, and use of practice tools, such as the EHR, to identify patients and document for quality measurement reports.
The survey was considered part of program evaluation activities conducted by PCIP and was deemed exempt by the Institutional Review Board at New York City Department of Health and Mental Hygiene. Clinicians in the control group were offered a $100 honorarium for participating in the survey.
Frequencies and averages were calculated for practice characteristics stratified by whether the practice was in the incentive or control group. All items in the survey were recorded into dichotomous variables and then stratified by incentive and control groups. Significant statistical differences between the incentive and control group were determined using χ2 tests. Data were analyzed using SAS software, version 9.2 (SAS Institute, Cary, North Carolina).
Items were recoded in the following manner: Answer choices of “all of the time with all of my patients,” “all of the time with a portion of my patients,” or “some of the time with a portion of my patients” were considered use of the functionalities and a “never” response was considered nonuse of the functionalities. Clinician responses on questions about their experience or use of the quality reports were recoded as agreement with the statement (“agree/strongly agree”) or disagreement (response of “neutral,” “disagree/strongly disagree”). QI visits and training was recoded as helpful (“helpful/very helpful”) or not helpful (“not at all helpful/slightly helpful”). Responses to items regarding clinician attitude toward future intentions to perform quality improvement activities were grouped into a positive response if they selected “likely” or “very likely” and a negative response if they selected “not likely.” Responses of “don’t know,” “not applicable,” and missing values were excluded.
Clinician and Practice Characteristics
Of the eligible 140 clinicians (70 per group), 104 completed the survey (response rate of 74%, 54 incentive and 50 control clinicians, P = .18). The majority of respondents specialized in family or internal medicine (98.1%) and the average respondent had been in practice over 18 years (Table 1). Mean length of time “live” on the EHR was 37 months, with an average of 7000 encounters per year. No statistically significant differences were observed between the incentive group and the control group for either clinician or practice-level characteristics. No statistically significant differences were observed between survey respondents and nonrespondents except for the proportion of the patient who were self-pay (3.9% for respondents and 7.0% for nonrespondents; data not shown).
Clinician Experience With Health eHearts
Overall, clinicians reported positive experiences. Respondents reported receiving and reviewing the quality reports (89%), agreed with the prioritization of ABCS (89%), thought the ABCS were clinically meaningful for their population (87%), and understood the information given in the quality reports (95%) (Figure). Clinicians in the program were using the EHR tools at least some of the time (Figure).
PDF is available on the last page.