Sarah C. Shih, MPH

To assess performance on quality measures among small primary care practices that recently adopted an electronic health record (EHR), and how performance differs between practices that have achieved patient-centered medical home (PCMH) recognition and those that have not.

Comparison of practice characteristics and performance on quality measures across 150 independent practices from 2009 to 2011 by recognition status for Physician Practice Connections—PCMH.

PCMH-recognized practices performed significantly better than nonrecognized practices on 5 out of 7 clinical quality measures at baseline, and the differences were maintained over the 2-year study period. Both groups improved on all clinical quality measures. Though the magnitude of differences was small, PCMHrecognized practices had a higher number of patients diagnosed with hypertension and proportionally more black patients. A significant difference in PCMH-recognized practices is that they received, on average, 4 additional quality improvement visits compared with nonrecognized practices.

Among small practices that have adopted EHRs, practices with PCMH recognition consistently outperformed practices without recognition on most clinical quality measures. With adequate assistance, small, resource-strapped practices can continue to have higher performance on clinical quality measures.

Patient-centered medical home (PCMH) recognition in small practices is associated with higher performance on quality measures over time.

		Small practices with PCMH recognition perform better on quality measures related to chronic conditions

		The differences observed between the practice groups appear to be sustainable over the 2-year study period.

		Quality measure performance is correlated with greater participation in a quality improvement program.

The patient-centered medical home (PCMH) concept has been widely promoted as a way to enhance primary care and deliver better care to patients with chronic conditions.

 This model of care has stimulated the attention of payers, Medicaid policy makers, physicians, and patient advocates, as it has the potential to address several of the shortcomings of the current healthcare system.

Components of the PCMH model include practice-based policies and processes intended to improve healthcare quality, patient experience, clinician satisfaction, and costs of care.

 Pilots and demonstration projects have yielded mixed results, but positive reports associate PCMHs in large practices and institutions with decreases in costs for patients enrolled in Medicaid; increased care of Medicaid patients; and positive patient and clinician experiences.

 Recently published studies have also shown that components of the medical home (eg, care coordination, pre-visit planning) are associated with positive effects on clinical outcomes for patients and quality of life for clinicians.

The process of transforming and achieving recognition as a medical home can be difficult, especially for small practices with few resources.

Additionally, small practices located in low income neighborhoods face challenges unique to their settings (eg, smaller panel size for quality measurement, lack of information systems, and lack of support staff) which can further exacerbate their ability to generate the documentation needed to demonstrate transformation and meet the standards for recognition.

 Promoting the adoption of PCMH standards in small primary care practices in lower income neighborhoods with a large proportion of chronically ill patients can be advantageous for reducing healthcare inequalities.

As part of a larger public health initiative, the Primary Care Information Project (PCIP), a bureau within the New York City Department of Health and Mental Hygiene, has offered assistance to small, independent practices in New York City to complete the application for the National Committee for Quality Assurance’s (NCQA’s) Physician Practice Connections—Patient Centered Medical Home (PPC-PCMH) recognition program.15 This study assesses the clinical quality outcomes of PCIP-participating practices that have achieved PPC-PCMH recognition from NCQA, and compares performance on quality measures from 2009 through 2011 in practices that have been PCMH-recognized with those that have not.

In an effort to transform primary care and improve population health, PCIP assists primary care practices in the adoption and use of electronic health records (EHRs). From its inception in 2005 through early 2012, PCIP assisted over 3000 primary care clinicians located in over 600 independently owned practices to adopt preventionoriented EHR systems. Collectively, PCIP participants represent over 20% of New York City’s primary care clinicians and serve approximately 2.6 million patients.

Recognizing that digitizing the information around healthcare alone would not bring about the changes needed to improve the delivery of clinical preventive services, PCIP offers additional support to address both technical needs and quality improvement (QI).

 PCIP employs a field team dedicated to assisting clinicians in optimizing work flows and leveraging data from their own EHR systems to track clinical quality performance. Much of the QI curriculum was modeled around the standards articulated by the NCQA’s PPC-PCMH program. This included an initial visit to identify practice leadership or champion for QI activities; follow-up meetings with the practice or provider to review progress on meeting QI objectives; using the functions within the EHR, such as the registry tool that generates lists of patients by selected characteristics (eg, diagnosis of hypertension with blood pressure over 140/90 mm Hg without a visit to practice in past 6 months and no scheduled visit in next 3 months); and reviewing the quality reporting tool—a specific feature within eClinical- Works co-developed with PCIP to display quality measures tied to the city’s health policy agenda.

 PCIP offered visits to practices every 4 to 6 weeks; staff were available by e-mail and phone to answer questions in addition to the on-site visits.

To encourage and facilitate the PPC-PCMH recognition process, PCIP established a multi-site application and offered to practices the ability to apply under the 2008 standards. Fees for the application were also subsidized by PCIP. Because some of the PCMH standards had been met across all PCIP practices, such as having an EHR with clinical decision support, clinicians did not have to repeat some of the standard documentation related to their EHRs in their NCQA applications. PCIP practices eligible to be included in the multi-site application were eligible to receive 36.75 points, which at the time was adequate for obtaining Level 1 recognition. However, all practices still needed to provide supporting documentation to demonstrate to NCQA their use of the systems; if practices wished to do so, they could work towards a higher recognition level by submitting additional documentation. NCQA then reviewed the documentation gathered by practices and submitted by PCIP to determine whether clinicians in the application met Level 1, 2, or 3 PPC-PCMH standards. By early 2012, PCIP had assisted roughly 25% of its member practices in achieving NCQA recognition, including 269 sites representing 169 practices and 657 clinicians to meet 2008 standards. Practices applying for PPC-PCMH recognition after July 1, 2012, are required to follow 2011 standards.

Primary care practices were included in this analysis if they had 5 or fewer clinicians and adopted the eClinicalWorks EHR system prior to October 2009. Pediatricfocused practices were excluded as the quality measures analyzed were limited to adult primary care. Practices were also excluded if data were unavailable for either of the 2 analysis periods: October 2009 and October 2011. Because the process of obtaining NCQA recognition can take between 6 to 12 months, practices were categorized into the PCMH-recognized group if they achieved NCQA recognition at any time up until February 2012.

As part of a contract negotiated with eClinicalWorks, PCIP established an automated monthly data transmission process to receive summarized data generated from each practice’s EHR. Practices receiving support from PCIP have agreed to provide data transmitted through this process. The data transmitted consisted of clinician-level aggregated counts of patients meeting numerator and denominator definitions for each of the quality measures, developed to be consistent with National Quality Forum measure definitions. 

 EHR utilization data are also received monthly, including number of labs reviewed, order sets used, number of monthly encounters per full-time equivalent (FTE), number of unique patients seen per month, as well as proportions of patients by age group, gender, race/ethnicity, or with a diagnosis of hypertension, diabetes, or both. PCIP staff used Salesforce, a commercially available, Webbased customer relationship management (CRM) application to monitor activities with the practices (eg, site visits with QI or other PCIP staff, attendance at classes or seminars hosted by PCIP or the vendor), maintain program administration and contact information, and track practice milestones such as initial contact for recruitment and date of EHR “go-live.”

Seven clinical quality measures of care and documentation were selected for analysis: (1) antithrombotic therapy for patients with diabetes or ischemic vascular disease; (2) blood pressure control in hypertensive patients with and without diabetes (2 separate measures); (3) body mass index (BMI) recorded; (4) glycated hemoglobin (A1C) testing in patients with diabetes; (5) smoking status recorded; and (6) smoking cessation intervention, including counseling, prescription drugs, and referrals to the New York State fax-to-quit hotline for identified smokers (

). These 7 measures were selected because they are indicators of clinical performance aimed at preventing and reducing smoking, diabetes, and cardiovascular disease, all of which are leading contributors to a large burden of morbidity and mortality within New York City and the United States.

 These chronic conditions are also frequently reported by PCIP field staff as some of the most common conditions among patients in practices seeking NCQA PCMH recognition.

Practices were grouped together based on whether they had an NCQA PCMH recognition status. We chose February 2012 as the cutoff date to categorize practices with or without an NCQA recognition status (Level 1, 2, or 3). Frequencies and descriptive statistics were generated and simple t tests and χ² tests were conducted to compare baseline practice characteristics for practices that did and did not receive PCMH recognition (

). To illustrate the time distribution of practices achieving PCMH recognition, we graphed NCQA recognition dates (

). Separately, we plotted the monthly cumulative average QI visits (on-site assistance provided to practices) by whether practices were PCMH-recognized or not from October 2009 to October 2011 (

We compared performance over time on quality measures over the same time period (October 2009-October 2011). Performance on each quality measure was constructed as a rate (numerator divided by denominator) for each practice for each month. To calculate meaningful rates and create stable comparisons, practices were excluded from individual measure analyses if their denominator for that measure was less than or equal to 10. Simple 

 tests were used to compare the performance on measures between practices with and without PCMH recognition at the baseline (October 2009) and current (October 2011) time points and to compare changes in quality measure performance within the groups (Table 1). Average rates of PCMH-recognized (solid lines) and nonrecognized practices (dashed lines) were graphed for each quality measure from October 2009 through October 2011 (

Mixed model regression was used to examine the differences in improvement between PCMH and non-PCMH practices over time for each quality measure, allowing each practice to have a different intercept to account for unexplained practice heterogeneity. 

shows the odds ratios (ORs) and 95% CIs generated from a general estimation model for the effects of PCMH recognition, time, and the interaction between the 2 variables across practices. All statistical analyses were conducted with SAS version 9.2 (SAS Institute Inc, Cary, North Carolina) and figures were drawn with R 2.15. For all statistical tests, a 2-tailed 

 value of less than .05 was considered statistically significant.

Out of 776 practices enrolled in PCIP as of July 2012, 349 were live on their system prior to October 2009. Of those, 150 practices met the selection criteria for inclusion in this analysis. Of the 150 included practices, 39.3% have achieved PCMH recognition from NCQA (Figure 1A), including 1 Level 2 and 12 Level 3 practices. The earliest practice to achieve PCMH recognition did so in February 2010, and the latest practice became PCMH-recognized in February 2012; half of practices gained recognition by December 2010.

PCMH-recognized and nonrecognized practices did not statistically differ at baseline in months using EHR, total encounters, number of clinicians, clinician FTE, or insurance type. PCMH-recognized practices had a slightly, though statistically significant, larger proportion of patients with hypertension (Table 2). A comparison of performance between practices by recognition level yielded no significant differences on any measure (data not shown).

At baseline, PCMH-recognized practices performed significantly better than nonrecognized practices on 5 of the 7 quality measures (Table 1). There was no difference in performance at baseline between PCMH-recognized and nonrecognized practices for the 2 blood pressure control measures. At the later time point (October 2011), PCMH-recognized practices outperformed nonrecognized practices on A1C testing, BMI recorded, blood pressure control in patients with both hypertension and diabetes, smoking status recorded, and smoking cessation intervention measures. There was no significant difference in performance between the practice groups on antithrombotic therapy and blood pressure control in patients with hypertension only.

Overall, both groups of practices improved on every measure over the 2-year period (Table 1). For 6 of the 7 measures, time using the EHR was a significant predictor of improvement (Table 3). For both of the blood pressure control measures, the interaction term was significant with an OR greater than 1, indicating that PCMH-recognized practices improved at a faster rate than nonrecognized practices. For the A1C testing measure and the BMI recorded measure, the interaction term was significant with an OR less than 1, suggesting that nonrecognized practices improved at a faster rate than PCMH-recognized practices. By the end of the study period, PCMH-recognized practices, on average, had received 4 more visits than nonrecognized practices with PCIP QI staff (Figure 1B).

Practices in both the PCMH-recognized and nonrecognized groups significantly improved performance on key clinical quality measures over the 2-year study period. At both the baseline and later time points, practices that had achieved PCMH recognition consistently performed better on process measures, including measures referring to in-office screenings or documentation proficiency, than practices that had not achieved PCMH recognition. PCMH-recognized practices performed about the same as nonrecognized practices at baseline on the blood pressure control measures in both diabetic and nondiabetic patients.

Time using the EHR was the most significant predictor of improvement on all quality measures, and the average time using the EHR was roughly the same for both groups of practices. Although both groups of practices improved on all measures over time, for some measures, PCMH recognition was associated with even greater improvement. PCMH-recognized practices improved at a faster rate than nonrecognized practices on both of the blood pressure control measures over the 2-year period, suggesting that PCMH recognition may have a positive effect on the quality of care for patients with multiple chronic conditions. Given that improved performance on these measures requires not only clinician intervention (eg, medication order) but also patient engagement (eg, medication compliance, lifestyle changes), it may be that practices that achieve recognition as a PCMH have learned to provide an enhanced level of care, enabling them to do better on these types of measures. There are different pathways of implementing the PCMH model and practices emphasize different components depending on their circumstances 

; however, PCMH pilots across the country have shown a similar correlation between component processes of the PCMH model and improvements in quality of care for patients with multiple morbidities.

Additionally, practice facilitator programs have been shown to have a positive impact on practice transformation; the PCIP QI curriculum may have had similar effects on the adoption of new processes or guidelines.

For other measures, PCMH recognition was not associated with a greater rate of improvement. Both groups improved at approximately the same rate on antithrombotic therapy and the 2 smoking measures, although a large gap remained in performance between PCMH-recognized and nonrecognized practices. Nonrecognized practices improved at a higher rate on BMI recorded and A1C testing. For both of these measures, nonrecognized practices performed significantly below PCMH-recognized practices at baseline, and our results indicated that over time, nonrecognized practices were able to reduce the initial gap in performance. Also worth noting is that the baseline rate of BMI recorded was quite high for PCMH-recognized practices (86%), and these practices may have reached a performance ceiling over the course of the study, whereas nonrecognized practices had more room to improve, leaving an opportunity to close the performance gap.

Articles

Small practices with NCQA patient-centered medical home recognition perform better on quality measures, especially those related to chronic conditions.

Patient-Centered Medical Home and Quality Measurement in Small Practices

To assess clinician attitudes and experiences in Health eHearts, a quality recognition and financial incentive program using health information technology.

 A survey was administered to 140 lead clinicians at each participating practice. Survey domains included clinicians’ experiences and attitudes toward the selected clinical quality measures focused on cardiovascular care, use of electronic health records (EHRs), technical assistance visits, quality measurement reports, and incentive payments. Responses were compared across groups of practices receiving financial incentives with those in the control (no financial rewards).

Survey response rate was 74%. The majority of respondents reported receiving and reviewing the quality reports (89%), agreed with the prioritization of measures (89%), and understood the information given in the quality reports (95%). Over half of the respondents had a quality improvement visit (56%), with incentive clinicians more likely to have had a visit compared with the control group (68% vs 43%, 

 = .01). The incentive group respondents (92%) were more likely to report using clinical decision support system alerts than control group respondents (82%, 

 Clinicians in both incentive and control groups reported positive experiences with the program. No differences were detected between groups regarding agreement with selected clinical measures or their relevance to the patient population. However, clinicians in the incentive group were more likely to review quarterly performance reports and access quality improvement visits. Incentives may be used to further engage clinicians operating in small independently owned practices to participate in quality improvement activities.

Am J Manag Care. 2013;19(11 Spec No. 10):eSP12-eSP18

		With adequate technical support, small practices can be engaged in recognition and financial rewards programs.

		Clinician buy-in to the design of the program was high. A majority of the clinicians reported receiving, reviewing, and understanding the quality reports; were in agreement with the focus on cardiovascular quality measures; thought the measures were clinically meaningful; and understood the information.

		Financially incentivized clinicians were slightly more engaged and participated in quality improvement visits and trainings, such as using clinical decision support systems and other electronic health record functionalities.

Use of incentives and pay-for-performance (P4P) to realign payment to address problems of low quality of care or gaps in preventive services has had limited success in improving the quality of healthcare.

For the most part, studies on P4P have focused on large group practices.

 Small practices, where the majority of patients still receive care nationally,

historically face greater obstacles to improving care because they have lacked the scale and organizational structure to conduct quality improvement activities or participate in P4P.

It is important to assess clinician attitudes toward key program features, such as the selection of target quality measures, trust in performance reports, and relevance of quality targets. Understanding clinician motivations and opinions toward a quality improvement program may help predict the extent to which they change their clinical behavior.

 Specific program features, such as the frequency and type of performance feedback and available assistance for meeting program goals, could potentially affect clinician awareness and understanding of particular programs. Clinician skepticism about the accuracy of reports, or distrust of or lack of transparency in data used for reporting or payment, may lead to less engagement of clinicians in incentive programs or quality improvement efforts.

With widespread implementation of electronic health records (EHRs),

 EHR-enabled solo and small group practices have been  shown to be capable of responding to quality improvement (QI) initiatives, as well as programs that incentivize using quality measurement.

 It is unknown how clinicians will feel about quality measurement and pay-forperformance using EHR-derived quality measures. To address this gap in the literature, we surveyed clinicians participating in Health eHearts, a cluster-randomized trial of the effect of a financial incentive and QI assistance program on measures of cardiovascular care compared with the effect of providing quality reports and QI assistance. The Primary Care Information Project (PCIP), a bureau of the New York City Department of Health and Mental Hygiene, piloted Health eHearts in practices that recently adopted an EHR and that were receiving ongoing QI visits to improve practice work flows using health information technology. Survey domains included overall experience with the program, as well as experience with the tools supporting QI efforts. In addition, we assessed whether there were differences in experiences or attitudes and whether these attitudes differed for practices receiving incentives or not.

PCIP recruited 140 small practices to participate in Health eHearts. The program duration was April 2009 to September 2011. Practices were eligible if they have been “live” on the EHR for at least 3 months, had a minimum of 200 patients with cardiovascular diagnoses related to the quality measurement targets, and were transmitting quality measures through the EHR to PCIP. Practices agreed to be randomized into “recognition” or “rewards” groups. Rewards consisted of financial incentives for each numerator met for 4 areas of cardiovascular care: aspirin therapy, blood pressure control, cholesterol control, and smoking cessation intervention (ABCS). Incentive amounts ranged from $20 to $150 per patient with goal achieved, with higher payments for harder to treat patients (eg, comorbid diseases or lower socioeconomic status). The recognition group served as a control. Both groups (control and incentive) received quarterly quality performance reports, telephone and onsite coaching on work flow redesign, and training on documentation, and were invited to a recognition program at the end of the year. The quality reports summarized practices’ progress on the ABCS and compared their performance with other practices in Health eHearts and trends over the previous 6 months.

Health eHearts was a 2-year program, with cohort 1 enrolled at the beginning and continuing for 2 years and cohort 2 enrolled at the beginning of year 2. Practices were surveyed before and after each program year. This study focuses on the survey administered to all participating practices at the end of Health eHearts. A 33-item survey (29 items in the control group version) was administered in October 2011. A lead clinician from each practice was invited to respond to the survey first by mail, followed by at least 3 reminder phone calls to nonresponding clinicians. Survey administration continued through February 2012.

The instrument was developed in collaboration between PCIP and researchers from University of California San Francisco (UCSF) who were contracted as evaluators for the overall evaluation of the program. The instrument focused on several aspects of the Health eHearts program: clinicians’ experiences and attitudes toward the selected quality measures (ABCS), training on use of the EHR or achievement  of ABCS, QI visits, tracking patients for preventive services using the EHR, quality reports, incentive payments (incentive group only), recognition programs in general, and demographics. The survey was pretested with program staff and a clinician in PCIP. Items used in this survey were based on an earlier instrument co-developed with UCSF to assess barriers and facilitators for small practices to participate in P4P. Topics identified as barriers included: accuracy and regularity of reports relevant to the practice’s patient population, measurement targets that were meaningful to the practice population, availability of training or assistance to conduct QI activities, and use of practice tools, such as the EHR, to identify patients and document for quality measurement reports.

The survey was considered part of program evaluation activities conducted by PCIP and was deemed exempt by the Institutional Review Board at New York City Department of Health and Mental Hygiene. Clinicians in the control group were offered a $100 honorarium for participating in the survey. 

Frequencies and averages were calculated for practice characteristics stratified by whether the practice was in the incentive or control group. All items in the survey were recorded into dichotomous variables and then stratified by incentive and control groups. Significant statistical differences between the incentive and control group were determined using χ

 tests. Data were analyzed using SAS software, version 9.2 (SAS Institute, Cary, North Carolina).

Items were recoded in the following manner: Answer choices of “all of the time with all of my patients,” “all of the time with a portion of my patients,” or “some of the time with a portion of my patients” were considered use of the functionalities and a “never” response was considered nonuse of the functionalities. Clinician responses on questions about their experience or use of the quality reports were recoded as agreement with the statement (“agree/strongly agree”) or disagreement (response of “neutral,” “disagree/strongly disagree”). QI visits and training was recoded as helpful (“helpful/very helpful”) or not helpful (“not at all helpful/slightly helpful”). Responses to items regarding clinician attitude toward future intentions to perform quality improvement activities were grouped into a positive response if they selected “likely” or “very likely” and a negative response if they selected “not likely.” Responses of “don’t know,” “not applicable,” and missing values were excluded.

Of the eligible 140 clinicians (70 per group), 104 completed the survey (response rate of 74%, 54 incentive and 50 control clinicians, 

 = .18). The majority of respondents specialized in family or internal medicine (98.1%) and the average respondent had been in practice over 18 years (

). Mean length of time “live” on the EHR was 37 months, with an average of 7000 encounters per year. No statistically significant differences were observed between the incentive group and the control group for either clinician or practice-level characteristics. No statistically significant differences were observed between survey respondents and nonrespondents except for the proportion of the patient who were self-pay (3.9% for respondents and 7.0% for nonrespondents; data not shown).

Overall, clinicians reported positive experiences. Respondents reported receiving and reviewing the quality reports (89%), agreed with the prioritization of ABCS (89%), thought the ABCS were clinically meaningful for their population (87%), and understood the information given in the quality reports (95%) (

). Clinicians in the program were using the EHR tools at least some of the time (Figure).

Nearly all clinicians (95%) responded that they understood the information summarized in the reports (Figure). A majority (69%) agreed that the data in the reports accurately reflected the practice’s performance and enough information was provided to track progress toward meeting targets (77%). There were few differences between the groups, although clinicians receiving incentives were more likely to report that they received and reviewed the reports compared with control clinicians (

There were significant differences between incentive and control group in their program participation (

). Over half of the respondents had a QI visit (56%); however, more clinicians in the incentive group reported having visits compared with the control group (68% vs 43%, 

= .01). Both groups reported that the visit was helpful (85% vs 80%, 

= .57), and the incentive group was more likely to report that the PCIP staff was accessible (69% vs 43%, 

 = .02). More clinicians in the incentive group had positive responses to the training using webinars (group online workshops) and web exes (virtual “visit” using the Internet; PCIP staff can  access the participant computer terminal and “talk through” use of the EHR) compared with clinicians in the control group. Overall, respondents expressed interest in more QI visits (81%).

Tracking Patients for Preventive Services Using EHR Tools

All respondents reported some use of the EHR functionalities (Figure 1). Clinical Decision Support System (CDSS) alerts (automated alerts and reminders for preventive services) and smart forms (automated question flows that assist clinicians in taking patient histories) were the most used. Although not statistically significant, incentive clinicians were more likely to report using EHR tools with the exception of the use of order sets to identify patients in need of preventive services (83% incentive vs 59% for control, 

Intention to Continue Activities After Health eHearts

Most respondents (80%) indicated the intent to generate quality reports after the program ended and allocate staff time to focus on QI activities (70%) (Table 2). Incentive clinicians were more likely to report that that they would generate quality reports (87% incentive vs 72% control, 

= .07), track practices’ progress toward meeting quality measurement goals (91% vs 78%, 

= .09), and hold regular meetings or check-ins (71% vs 57%, 

 = .14) compared with control clinicians.

Small practice clinicians had positive experiences with the rewards and financial recognition program designed to improve the delivery of clinical preventive services. Clinicians in the incentive group were more likely than those in the control group to report participating in quality improvement activities offered by the program, such as reviewing the quality reports, using order sets, and participating in program training sessions. The high level of buy-in to the program is demonstrated by the reported usability and accuracy of the quality reports and by reported agreement with the ABCS prioritization of preventive cardiovascular care.

Past studies document instances of clinician skepticism about the validity of clinical quality measurements or accuracy of reports, leading to less engagement of clinicians in quality improvement efforts.

 In addition, because of the lack of transparency in data used for reporting or payment, some P4P programs have been seen as a threat to clinicians’ autonomy and sense of control.

 The Health eHearts program addressed issues seen in earlier studies by generating reports directly from the practices’ EHRs, offering transparency into the data used for quality measurement, and also by providing QI assistance and help with troubleshooting problem areas with the intent of improving clinician sense of control over measured performance.

Alignment of the program goals with the practice’s organizational structure and culture has been associated with successful P4P implementation.

The majority of clinicians agreed with the prioritization of the ABCS and found them to be meaningful to their practice. Positive clinician attitude has been associated with successful implementation of EHRs

 and is potentially an important contributor to continued EHR use, especially in small independently owned practices that do not have dedicated staff for quality measurement or EHR-based reporting. 

Robust EHRs can systematize and streamline work flow by allowing clinicians to use key features, such as CDSS.

 However, small practices are less likely to utilize these features.

These survey results suggest that providing QI assistance along with incentives can be effective in engaging clinicians both during a program and potentially for sustaining continued QI activities.

Our study has several limitations. As a self-reported survey, it is subject to social desirability bias whereby clinicians may be inclined to respond positively instead of with criticism. In this study, the differences between the incentive group and the control group answers were likely equally affected by this bias, implying that the differences observed in reported engagement with quality improvement activities would not be affected by this limitation, though the overall experience ratings may be higher than if respondents were not affected by this bias.

It is also possible that the overall ratings of the experience in the program are more positive than the experience for all participants in the program, since some participants did not respond. However, we received a high response rate of 74% and there were few  significant differences in practice characteristics between respondents and nonrespondents.

A study to assess clinician attitudes and experiences after participating in a New York City cardiovascular disease focused quality recognition and financial incentive program using health information technology.

Small Practices' Experience With EHR, Quality Measurement, and Incentives

 To describe the benefits and limitations of incentive arrangements used to engage small primary care practices to adopt electronic health records (EHRs).

 Retrospective review of program outreach efforts and EHR enrollment rates.

Comparison of benefits and limitations of various promotional strategies in addition to a city subsidy. Measures of enrollment progress include the mean number of outreach attempts, the mean number of days from first attempt to enrollment, and the enrollment yield. Selected practice demographics were collected for comparison purposes.

 Of 890 providers representing 217 practices who were eligible for the city subsidy, 37.7% enrolled, with a mean of 96.6 days from first attempt to enrollment and a mean of 10 outreach attempts. The offer for full payment of technical assistance fees and hardware yielded an additional 100 providers representing 43 practices, a 14.1% enrollment rate. This group also had the highest mean number of days from first attempt to enrollment (236.0 days) and the highest mean number of outreach attempts (22 attempts). The offer for a partial rebate had the lowest yield (a 6.2% enrollment rate), a mean of 169.3 days from first attempt to enrollment, and a mean number

 Because of diverse needs and levels of awareness in the EHR adoption process, a flexible milestone-based process is needed to engage primary care providers. In particular, community influence and additional funding were necessary for increasing enrollment among providers in medically underserved neighborhoods. These providers also required persistent and numerous follow-up attempts. Because billions of dollars in federal aid are being offered, effective local recruitment strategies are needed to facilitate provider engagement to increase EHR adoption rates.

	Multiple methods are needed to engage primary care providers to adopt electronic health records.

		Beyond monetary incentives, educated outreach and recruitment support are needed to convince providers to adopt electronic health records.

		A contact relationship management tool or database is necessary for tracking progress and for coordinating recruitment efforts.

	The federal government has appropriated an unprecedented amount of funding through the American Recovery and Reinvestment Act of 2009 to spur the adoption of electronic health records (EHRs).

 Despite evidence that the adoption of EHRs has the potential to improve healthcare,

 rates of using fully integrated health information systems remain low among practices with 5 or fewer physicians.

The most frequently cited barriers include cost and potential loss of productivity; on average, EHRs cost almost $44,000 per full-time equivalent provider, with ongoing costs of $8400 annually. 

 Furthermore, practices operating in medically underserved areas and providing care to vulnerable populations are less likely to adopt EHRs because of limited resources and competing priorities.

	Before the passage of the American Recovery and Reinvestment Act of 2009, the New York City (New York) Department of Health and Mental Hygiene established the Primary Care Information Project (PCIP) to help implement EHRs and to assist providers in using information technology (IT) to improve healthcare. The city secured more than $60 million in public and private funding to subsidize the purchase and implementation of EHRs for 2000 independent primarycare providers. The PCIP focused on enrolling providers serving a 10% or greater threshold of Medicaid or uninsured patients. In addition, the PCIP extended extra attention to engage providers practicing in areas identified as District Public Health Offices (DPHOs). The DPHO areas are located in central Brooklyn, east and central Harlem, and the south Bronx and have the highest urban poverty concentrations, health problems among the population, and mortality rates.

	Despite significant resources offered by the city, primary care providers were not eager to adopt EHRs, and substantial effort was needed to generate interest and engagement among provider communities. Less than one-fourth of providers contacted by the PCIP had the knowledge or educational resources to make an informed decision about adopting EHRs.

	Nationally, the Regional Extension Centers designated by the Office of the National Coordinator for Health Information Technology to assist providers in adoption and meaningful use of EHRs will be facing similar challenges. Provider engagement has been a substantial first hurdle in the path to adopting information systems. This communication detailsFebruary 2007 through June 2009 and the lessons learned to engage more than 1100 providers to adopt an EHR.

	The PCIP focused recruitment efforts on primary care providers (eg, physicians, nurse practitioners, midwives, doctors of osteopathic medicine, and physician assistants in any of the specialties of internal medicine, family practice, pediatrics, geriatrics, or obstetrics and gynecology) practicing in New York City’s poorest neighborhoods or serving Medicaid and uninsured patients. Practices located within the 5 boroughs with at least 10% Medicaid or uninsured patients were eligible to receive software subsidies and support from the PCIP. The standard city subsidy program (PCIP standard program) covered software licensing costs for eClinicalWorks (Westborough, MA) and technical assistance for up to 2 years. Practices were responsible for purchasing hardware (eg, computers, printers, fax servers) and for covering infrastructure costs (eg, Internet service, IT consultant). A technical assistance fee of $4000 per full-time equivalent provider was also paid by the practice. The PCIP required the technical assistance fee as an indication of commitment to the implementation process. The technical assistance fee covered services from the PCIP on guidance with software integration, connection to electronic laboratory and pharmacy interfaces, community immunization registry, customized training and on-site sessions with experts in using the software for billing and effective documentation, and up to 10 coaching sessions with quality improvement specialists. Additional details of the PCIP standard program have been summarized in a previous communication.

	Because small practices did not generally belong to a single umbrella organization and were widely dispersed, partnering organizations were identified to enhance the engagement process or to “get the word out.” As a result, the following 4 programs were implemented to supplement the PCIP standard program: (1) community leadership, (2) health plan rebate, (3) Regional Health Information Organization incentive, and (4) private donation (

). The community leadership used social networking to identify potential practices and leveraged provider-to-provider recommendations without additional monetary incentives. The health plan rebate was offered by a local health plan to practices with a minimum number of health plan members. Practices in this supplemental program would receive $2000 per full-time equivalent provider after implementing the EHR. The Regional Health Information Organization incentive was offered to practices located in Queens and sections of Brooklyn. Through the practice’s membership with the Regional Health Information Organization, they would receive $3400 per full-time equivalent provider before EHR adoption to help defray the technical assistance fee required by the PCIP standard program. The private donation was available only to practices in the DPHO areas and covered the costs of the hardware and technical assistance fee in full. The costs and support provided by the supplemental programs are summarized in Table 1.

	A team of 4 staff members from the PCIP conducted outreach to recruit small medical practices, community health centers, and hospital ambulatory facilities. A customer relationship management tool, www.salesforce.com (The Landmark @ One Market, San Francisco, CA), was used to track communication with providers and to monitor progress toward enrollment. The PCIP obtained lists of providers from the American Medical Association and from health plan databases. Additional providers were identified by word of mouth from friends in the community, leveraging relations with physician leaders in New York City, and from ad hoc referrals. The PCIP generated interest in EHR adoption through education, software demonstrations, and e-mails containing program literature. Specific activities included announcements or articles in the PCIP partner newsletters, e-mail blasts to partner-affiliated provider networks, and in-person events. Follow-up activities were conducted by phone calls, e-mails, and facsimiles. The outreach team encouraged providers to participate in software demonstrations before enrollment.

	Once engaged, the PCIP asked providers to submit an application by e-mail. The application form collected information on practice demographics, patient and payer mix, IT, and organizational readiness and goals for EHR adoption. The PCIP then reviewed applications and provided status updates within 3 days, followed by a phone call within 24 hours. Staff members sent contracts via e-mail to practices that met the minimum criteria for the city software subsidy. Providers were required to complete an IT assessment in preparation for implementation. Practices were given a deadline to submit a signed contract within 2 weeks. However, deadlines wereflexible and allowed leeway for time-intensive steps, such as hardware and funding procurement by the practice. The outreach team persisted with phone follow-up until the contracts were completed and payment was received.

	The PCIP monitored each stage of the enrollment process using the customer relationship management tool. The tool alerted outreach team members of an individual practice’s progress from one step to the next in the application and agreement process. Practices that declined to enroll in the PCIP were categorized as “Decided Not to Sign Up”; others who did not move on to enrollment within 3 months were categorized as “Aged.” For both categories, no additional phone follow-up was conducted by the PCIP. Providers who delayed enrollment because of various constraints were categorized as “On Hold,” and PCIP staff established a time frame in which to reinitiate contact.

	Staff members generated descriptive statistics to assess the effect of each strategy on improving enrollment. Fewer than 10 practices qualified for more than 1 of 4 supplemental programs; analysis considered only the most recent supplemental program to which a practice was exposed. The mean number of outreach attempts, the mean number of days from first attempt to enrollment, and the enrollment yield were calculated. Selected practice demographics were collected for comparison. Descriptive statistics of enrollment yield, the mean number of outreach attempts, and themean number of days from first attempt to enrollment were limited to federally qualified health centers or small independent practices.

	Of 30,000 providers estimated to practice in New York City, the PCIP contacted 7263 potentially eligible primary care providers. Of those, 4379 were identified as eligible for the PCIP standard program and were affiliated with a federally qualified health center or a small practice. Roughly 25% of eligible providers enrolled by April 2009, and the remaining providers decided not to sign up, became an aged application, or were put on hold.

	Through the PCIP standard program, 890 providers representing 217 independent practices joined, an enrollment yield of 37.7%. On average, these practices took 96.6 days to sign an agreement and 10 outreach attempts (including phone, e-mail, and facsimile). The average practice that enrolled through the PCIP standard program had 4.1 providers, with a patient population of 44.6% Medicaid and 8.6% uninsured (

	An additional 48 providers representing 24 practices joined through the community leadership process. These practices had the highest mean proportion of Medicaid insured patients (73.8%) and were similar in practice size (2 providersper practice) to the other supplemental subsidy programs.

	The health plan rebate had the fewest additional providers join (33 providers representing 18 practices), with the lowest enrollment yield of 6.2% from an eligible pool of 529 providers. The Regional Health Information Organization incentive brought in more providers (51 representing 18 practices) and required a little less effort by the outreach team. The private donation brought in an additional 100 providers representing 43 practices but required the most recruitment effort and the longest lag time for signing an agreement. Statistics for the number of attempts and the time to enrollment across the programs are summarized in Table 2.

	Within the first 28 months, the PCIP assisted 1122 providers representing 320 independent primary care practices to adopt EHRs. Although the PCIP offered various supplemental incentives, persuading physicians to implement EHRs was challenging and labor intensive. Summarized herein are experiences of the PCIP with provider engagement and recruitment strategies for EHR adoption.

Make Use of the Customer Relationship Management Tool. 

The strategies developed by the PCIP were refined based on available call logs and communication activities entered into the customer relationship management tool www. salesforce.com. These data permitted the staff to continually assess their efforts through the early stages of the outreach process and to identify needs for follow-up, as well as to coordinate communication efforts to avoid repeated or conflicting messages.

 In comparing modes of communication with practices, phone contact was the most effective method and was overwhelmingly preferred by providers, supplemented by faxing. Although they were encouraged to use electronic mail, not all providers had an e-mail account, and several practices shared a single e-mail address. Few on-site presentations were conducted for larger practices; door-to-door visits to the practices to discuss EHR adoption was the least effective strategy and was difficult to scale broadly.

 A significant facilitating factor for engaging practices is identifying the “right” contact at practices. For small practices, targeting the lead physician and practice decision maker resulted in a higher likelihood of enrollment.

 It was important to be persistent with practices that did not sign on within the first few attempts. More than 75% of the practices needed more than 10 “touches” before submitting the agreement. Most practices received between 10 and 20 follow-up contacts from the PCIP, and it was not uncommon to have at least 30 to 40 communications with a practice before obtaining a signed agreement. In an average week, staff would start with 15 to20 providers expressing interest but would only receive 3 to 4 signed agreements.

 In addition to persistence, providers were more responsive if the PCIP staff insisted on target dates, which emphasized the importance of acting soon. However, missed deadlines could be extended when helpful. Other tactics that sustained engagement included stressing that the number of free software licenses was limited (eg, available on a “first come first served” basis) and informing practices of their application progress to create a sense of accomplishment.

 Aside from general cost barriers, providers did not know what to expect or how to anticipate the requirements for adopting an EHR. Most of the time spent by the outreach team was to educate providers and to answer questions about start-up costs and effect on resources. Outreach staff had to quickly learn and be able to inform providers on various questions ranging from how much to set aside for hardware, monthly Internet access fees, hiring an IT consultant to assist in setup, and other potential expenses, such as building modifications for additional electrical outlets or locations for monitors and keyboards. Providers also wanted to know how much time to set aside before the transition and the amount of downtime during the installation. In-person demonstrating of software or connecting providers to other clinicians already using an EHR was helpful to gain buy-in about the benefits of the EHR.

 Supplemental programs, such as the private donation program available to the PCIP, were necessary to reach providers practicing in medically underserved areas. The DPHO area providers often agreed about the benefits of EHR adoption but would not consider adoption without immediate financial aid. Rebates or partial upfront subsidies were not effective in convincing providers, as funding was too distant or insufficient. Few providers used the rebate or partial reimbursement, and most were not interested in loans or financing.

 The PCIP dedicated 4 full-time staff members with a public health background and a master’slevel education to conduct outreach efforts. Increasing the scale of their efforts was possible through leveraging community leaders and collaborating with physician membership organizations or associations, as well as health plans. These relationships garnered the practice’s attention and followthrough to the enrollment process, especially for high-volume Medicaid providers. By having community leaders send letters, copresent enrollment materials, or repeat messages of the PCIP in their routine communications, physicians were more likely to recognize follow-up phone calls or materials sent by the PCIP, as well as provide a quicker response.

	Although challenging, motivating primary care practices to adopt an EHR is not impossible. From the experience of the PCIP, supplemental funding programs are necessary to reach providers practicing in medically underserved areas. Without full upfront funding, EHR adoption would not be feasible for practices like those in the New York City DPHO areas with extremely limited resources, minimal staffing, and time constraints. It is encouraging to see that the Medicaid “meaningful use” incentives provide an initial payment of $21,250 and subsequent funding after demonstrating use of clinical systems.

	Additional studies and reports will help identify strategies to engage physicians or identify solutions to facilitate the adoption of health IT. Much of the health benefits to be gained by widespread adoption of clinical information systems can only be realized if most providers participate. The same challenges that the PCIP faced in enrolling practices to implement an EHR will be faced by the Regional Extension Centers designated across the country. Although an unprecedented amount of funding has been made available through the American Recovery and Reinvestment Act of 2009 and through support services from the Regional Extension Centers for EHR implementation, it is important to anticipate that a major barrier remains to fully engage primary care providers.

	We thank Dr Sheryl Silfen for her early guidance and assistance in reviewing multiple versions of the manuscript.

 California Health Information Partnership and Services Organization (RS), Oakland, CA; Accenture (VON), New York, NY; Johns Hopkins Bloomberg School of Public Health (JWMB), Baltimore, MD; New York City Department of Health and Mental Hygiene (VR, SCS), New York, NY; and Office of the National Coordinator (MLK), Washington, DC.

Substantial outreach efforts are needed in addition to offering subsidies or incentives in engaging primary care providers to adopt electronic health records.

Promoting Electronic Health Record Adoption Among Small Independent Primary Care Practices

: To determine whether a questionnaire that assesses the presence of practice systems is associated with clinical performance rates for diabetes care.

: Cross-sectional study of the relation between a survey-based measure of practice systems within 5 of the domains of the chronic care model (CCM) and high-quality care for diabetes during 2005 among 40 medical groups in Minnesota.

: Correlations were calculated between (1) practice systems as measured by the Physician Practice Connectionsâ€“Readiness Survey (PPC-RS) questionnaire and (2) process and outcome measures of diabetes quality from a standardized system managed by Minnesota Community Measurement.

: Most process and outcome measures were correlated at 0.31 to 0.52 (P <.05) with the PPC-RS total score as well as with several of the CCM domains. Only yearly eye exams and blood pressure control lacked correlation with any CCM domain, but delivery system redesign and selfmanagement support lacked correlation with quality measures. 

: As measured by the PPC-RS questionnaire, the presence of practice systems overall and within several domains of the CCM was associated with high-quality care for diabetes. The PPC-RS may be a useful and relatively simple tool for evaluating and guiding improvement of practice systems for diabetes care quality.

Practice systems within the framework of the chronic care model appear to be

the key to improving the quality of care for patients with chronic conditions.

The Physician Practice Connectionsâ€“Readiness Survey (PPC-RS) can identify practice systems that are associated with higher rates of diabetes care quality.

The PPC-RS is a relatively simple and valid way to evaluate and guide improvement of these systems.

This information should be useful to medical practices, payers, and researchers. 

In its reports on the problems with medical care quality in the United States, the Institute of Medicine highlighted the need for transformational improvements in care, with particular priority for chronic conditions.

 Subsequent comprehensive performance measurements across the United States by McGlynn et al documented the extent of the quality improvements needed.

 Diabetes is a particularly useful condition for highlighting the issues involved in improvement, because (1) it is common and costly; (2) evidence-based quality measures of both care processes and outcomes are endorsed by the National Quality Forum, the National Committee for Quality Assurance (NCQA), and the Ambulatory Quality Alliance; and (3) improving those measures has received a great deal of attention.

 We know from the wide variation in performance rates among different practices and from dramatic improvements in some settings that it should be possible to make substantial improvement.

 However, Saaddine et al report relatively small improvements in national performance over the last decade, and only on some measures.

Organized clinical practice systems have been well demonstrated to be the key way to provide more consistent and comprehensive longitudinal care for either preventive services or chronic conditions.

we mean organized processes designed to ensure that certain information is collected and information or services are provided routinely to patients or healthcare personnel

. These systems have been organized into a conceptual framework (the chronic care model [CCM]) that is now widely accepted as the best way of thinking about the activities to improve chronic illness care.

 The CCM organizes systematic care for chronic illness into 6 domains: health system, delivery system redesign, decision support, clinical information system, self-management support, and community resources. Bodenheimer has summarized the evidence for those systems and the CCM,

 and Feifer et al and O&#8217;Connor et al have shown how systems can improve both care processes and patient outcomes.

Others have shown that implementation of some practice systems were associated with improvement in diabetes care.

Unfortunately, relatively few of these practice systems are present, even in large medical groups. Casalino et al called these systems care management processes (CMPs) and found that among 1040 physician organizations nationally with at least 20 physicians, the average organization had only 5 of 16 possible CMPs present.

 Of the 4 CMPs related specifically to diabetes care, only one third of these organizations used at least 3 of these CMPs, and half used either none or only 1.

If practice systems are important for improving the care of diabetes and other chronic conditions, it would be very useful to have a valid and reliable way to measure the presence of such systems in individual clinics and medical groups as a step toward encouraging their presence and as a guide to internal quality improvement efforts. The NCQA has developed the Physician Practice Connections (PPC) tool to serve this need for purposes of research and quality improvement, and as a basis for rewarding the use of systems in office practices. The PPC, which uses the CCM as a conceptual framework, was developed from an extensive literature review and input from experts and key stakeholders, as well as from an analysis of &#8220;defects&#8221; in office-based practice using Six Sigma methods.

A paper version of the PPC, the PPC Readiness Survey (PPC-RS), was adapted for research and quality improvement purposes. It has been tested for reliability against in-practice audits and found to be reasonably accurate in research applications, with a positive predictive value (PPV) that ranged from 55% to 100% for various components when completed by a group&#8217;s medical director.

 The main problem in testing was false-negative reports, with each type of respondent tending to underreport some practice systems that appeared to be present according to an on-site audit. Thus, although agreement rates were somewhat lower than the PPV, positive reports of systems were generally confirmable in this research application, where responses would not influence financial or nonfinancial rewards.

A version of the PPC now is being used by the American Board of Internal Medicine and others to encourage wider knowledge and use of systems as a means for improving quality, and in a modified Web-based format that requires attached documentation for each of the survey elements (

). Some health plans are also using the PPC recognition program as a basis for paying for the use of systems in office practice. The same standards are being used by Bridges to Excellence as a component of their pay-for-performance program (

). Although the evidence previously cited links the use of systems to enhanced clinical performance, it is important to be able to demonstrate that there is a relationship between the systems assessed by this instrument and evidence-based measures of quality of care, both for processes and outcomes. We report such a test of this relationship for diabetes care among the 40 medical groups in Minnesota that provide primary care to a large majority of its citizens.

Practices participating in Minnesota Community Measurement (MN CM) provided a good opportunity to test the association between the systems tool and clinical process and outcome measures. MN CM has pioneered standardized measurement and public reporting of performance rates for preventive services and various conditions at the medical group level.

 More information about this organization and review of their reports on most of the medical groups in the state is available at their Web site: 

. In brief, MN CM is sponsored by all of the health plans in the state and has developed methods to use health plan administrative data to identify patients of each medical group in a way that allows stratified sampling of individual records for chart audits in a standard format and aggregation by a contracted analysis team. Medical group&#8211;level rates for process and outcome measures reported by MN CM are based on a minimum of 60 chart reviews per medical group that are performed by external auditors using the same prescribed methods for each medical group. The results also are validated by an independent auditor agency. 

Minnesota also has pioneered an overall measure of diabetes care intermediate outcomes, called optimal diabetes care, which was described by Nolan and Berwick in their important commentary urging the use of such &#8220;all-or-none&#8221; measures.22 This measure is based on individuals as the unit of analysis, so that each patient is only counted as a positive for this measure if all of the following goals has been achieved: glycosylated hemoglobin (A1C) &#8804;8.0%, low-density lipoprotein (LDL) cholesterol <130 mg/dL, blood pressure <130/85 mm Hg, and documentation of both regular aspirin use and nonsmoking status in the medical record. As a result of such a stringent requirement, the average performance rate is about 15%, although a few individual physicians have attained rates of 50%.23 This study relies primarily on the results of this measure as calculated by MN CM from care provided in 2005. It also reports rates from that same source for the 5 individual components as well as for 5 process measures (testing rates for A1C, LDL cholesterol, blood pressure, eye exams, and microalbuminuria).

Out of the approximately 65 medical groups with a large enough cohort of patients to be included in MN CM public reports, only 41 had publicly reported data for both diabetes and depression (the depression indicators were used for a separate study). All 41 groups agreed to participate and all 41 returned completed copies of the PPC-RS survey that allowed us to measure the presence of practice systems relevant for diabetes care.

 Two medical groups shared a common owner, so their performance rates are combined in the MN CM reports. Hence, the results represent 40 unique medical groups.

The PPC-RS survey used in this study was completed by medical directors in the summer of 2005. That PPCRS version consisted of 6 sections with a total of 181 items, and it took approximately 45 minutes to complete. Some of the questions addressed organizational demographics, characteristics, culture, or specific diseases other than diabetes (see the following paragraph) and thus did not constitute measures of the presence or absence of systems related to diabetes. These items are used to describe medical groups and also as covariates when testing the association between systems and quality measures. The PPC-RS version used also did not include measures of community resources because it was difficult to identify measures for this CCM domain that had strong face validity for use at the medical group level. Thus, 53 questions were actually used in the final scoring of the PPC-RS. There were 3 questions related to the health system, 8 related to delivery system redesign, 10 related to the clinical information system, 9 related to decision support, and 23 related to selfmanagement support. The items addressed in each domain are listed in 

 provides descriptive information on both PPC-RS scores and outcome measures. The medical groups have a high proportion of these important practice systems, at least in the case of diabetes. The mean overall proportion of systems is 67%, varying from 56% for delivery system redesign to 83% for health system supports. The 5 PPC-RS scores (excluding the total) have intercorrelations ranging from 0.15 to 0.52 with a mean intercorrelation of 0.38 (data not shown). Despite relatively high proportions of patients who achieved control for important individual diabetes outcome measures, the mean group rate for the composite measure of optimal diabetes care was only 19%.

The PPC-RS total, clinical information system, and decision support scores also showed significant positive associations with 3 of 5 of the tests that were being performed at high rates (A1C, LDL cholesterol, and blood pressure), but there was little in the way of significant associations with eye exams or microalbumin testing rates. The health system score relates only to LDL testing. And the remaining 2 PPC-RS domains (delivery system redesign and self-management support) were not related to any of the testing measures. When repeating the correlation analysis in 

 and partialing out medical group location, number of physicians, type of practice, and percentage of patients with commercial or private insurance, the resulting correlations involving the PPC-RS total score were only slightly attenuated (reduced on average by 

These results demonstrate that practice systems for diabetes, as measured by self-report on the PPC-RS survey, correlate fairly well with most process and outcome measures for quality of diabetes care. This finding in one region&#8217;s medical groups that have relatively high levels of both practice systems and care quality for patients with diabetes is important for several reasons. First, it suggests that the PPC-RS may be a useful tool for assessing the types of practice infrastructure associated with improving quality of care, at least for diabetes. Second, it confirms findings from controlled trials that suggest that practice infrastructure (and practice systems in particular) are important for providing high-quality care to patients with chronic conditions and shows this relationship in routine practice settings. It also provides support for the idea that the CCM is a useful way of organizing our thinking and quality improvement efforts.

It is interesting that decision support and health system quality improvement systems were especially important domains in this observational study. Sperl-Hillen et al previously demonstrated among 17 clinics in 1 medical group that decision support as measured by the Assessing Chronic Illness Care survey was the only domain of the CCM that was associated with improvement over 1 year in glycemic and lipid control.

 In another study of this group&#8217;s diabetes care quality improvements over 10 years, improvements in both A1C and LDL cholesterol were found to have been driven primarily by drug intensification (increasing dose or adding medications), leadership commitment to improvement of diabetes care, greater continuity of primary care, participation in local and national diabetes care improvement initiatives, and allocation of multidisciplinary resources at the clinic level to improve diabetes care.

 All but possibly drug intensification are clearly components of health system supports. Finally, Shojania et al conducted a meta-regression analysis of 64 trials of quality improvement strategies for diabetes. Only 2 of 11 strategies were associated with more than small improvements: changes in team structure and care management (where managers could adjust medications without physician approval).

The Casalino et al studies of CMPs among many large medical groups across the country (National Survey of Physician Organizations) suggest that these care processes frequently are not present. They found an average of only 5 of 16 possible CMPs and 1.7 of 4 possible CMPs specific for diabetes.

 The 4 diabetes CMPs they measured were clinical practice guidelines in conjunction with physician reminder systems, case management, performance feedback to individual physicians, and disease registries. These 4 systems all are covered by PPC-RS questions and relate to 4 of the CCM domains, but constitute only 10% of the systems assessed by that instrument. Thus, the PPC-RS should be thought of as a much more detailed assessment, one only indirectly related to the CMP findings of the National Survey of Physician Organizations studies. Nevertheless, it is interesting that the National Survey of Physician Organizations project found that the number of diabetes CMPs was strongly associated with external incentives to improve quality, computerized clinical information systems, and ownership by hospitals or health maintenance organizations.

 used a survey that was based on CCM components included in a qualitative instrument called Assessing Chronic Illness Care

 to study health plan activities associated with the 6 Healthcare Effectiveness Data and Information Set (HEDIS)28 measures of diabetes quality in Medicare contract enrollees. Although such systems used by a health plan are inherently different from those used in a medical group or clinic, these researchers found that the mean number of systems for the upper and lower quartiles of quality scores was 17.5 and 12.5, respectively (out of a possible 32). After adjusting for structural and geographic differences, only practitioner input and use of clinical guideline software remained as significant predictors of this difference.

Although promising, our results have limitations. The instrument itself only imperfectly captures data about which systems are present and especially about the extent to which they are extensively used and function well. For example, in our preliminary studies, agreement between the PPC-RS and an on-site audit was very high for quality improvement questions (97%), but only moderate for registry (78%) and clinical information system (71%) questions, and low for care management questions (28%-65%).20 Thus, the low correlations of performance measures with the self-management support domain may have been confounded by this problem. Most of the agreement discrepancies were because medical directors did not report systems that we were able to determine to be present at the audit, so the direction of the bias would be to underestimate the degree of correlation. Although it would have been nice if the domains with higher agreement rates had the strongest relationship to performance, that was not consistently true, probably because agreement was confounded by the relative importance of various domains.

The limited number of medical groups in our sample also made it difficult to test relationships between specific questions within the PPC-RS and our quality measures. Because there were no data about patient characteristics, it was not possible to adjust the performance measures for differing patient populations. The sample size also limited the extent to which the effects of multiple covariates could be examined simultaneously, which is why this study emphasizes the use of bivariate and partial correlations. Thus, larger studies that test these associations over time and that adjust for multiple confounders simultaneously would be valuable. Finally, our region is somewhat atypical in the size of its medical groups, in the extent to which quality is a high priority, and in the extent to which organized practice systems are used to improve quality.

Nevertheless, these preliminary findings are important. They support the importance of practice systems for quality care of diabetes and suggest that internal group use of the PPC-RS may provide a helpful guide for improvement efforts. Moreover, when combined with review of documentation and sample on-site audits, use of the standards in the PPC-RS could be valuable for external assessment by public performance reporting or incentive programs. In that regard, a version of the PPC is being considered by the American College of Physicians, American Academy of Family Physicians, American Academy of Pediatrics, and American Osteopathic Association as the tool for qualifying primary care practices as Patient-Centered Medical Homes.

As a set of practicespecific structural measures, the PPC-RS would be less subject to some of the issues of sample size, attribution, and risk adjustment that plague clinical process and outcome measures used at the individual practice, site, or small group level.

. Washington, DC: National Academy Press; 1999. 

The Physician Practice Connections-Readiness Survey can identify practice systems that are associated with higher rates of quality diabetes care.

Sarah C. Shih, MPH

Articles

Patient-Centered Medical Home and Quality Measurement in Small Practices

Small Practices' Experience With EHR, Quality Measurement, and Incentives

Promoting Electronic Health Record Adoption Among Small Independent Primary Care Practices

Practice Systems Are Associated With High-quality Care for Diabetes