The American Journal of Managed Care August 2010
Relationship Between Quality Improvement Processes and Clinical Performance
This study examines the clinical effects of care management and quality improvement interventions implemented by physician groups on pay-for-performance success.
Objectives: To examine the association between performance on clinical process measures and intermediate outcomes and the use of chronic care management processes (CMPs), electronic medical record (EMR) capabilities, and participation in external quality improvement (QI) initiatives.
Study Design: Cross-sectional analysis of linked 2006 clinical performance scores from the Integrated Healthcare Association’s pay-for-performance program and survey data from the 2nd National Study of Physician Organizations among 108 California physician organizations (POs).
Methods: Controlling for differences in PO size, organization type (medical group or independent practice association), and Medicaid revenue, we used ordinary least squares regression analysis to examine the association between the use of CMPs, EMR capabilities, and external QI initiatives and performance on the following 3 clinical composite measures: diabetes management, processes of care, and intermediate outcomes (diabetes and cardiovascular).
Results: Greater use of CMPs was significantly associated with clinical performance: among POs using more than 5 CMPs, we observed a 3.2-point higher diabetes management score on a performance scale with scores ranging from 0 to 100 (P <.001), while for each 1.0-point increase on the CMP index, we observed a 1.0-point gain in intermediate outcomes (P <.001). Participation in external QI initiatives was positively associated with improved delivery of clinical processes of care: a 1.0-point increase on the QI index translated into a 1.4-point gain in processes-of-care performance (P = .02). No relationship was observed between EMR capabilities and performance.
Conclusion: Greater investments in CMPs and QI interventions may help POs raise clinical performance and achieve success under performance-based accountability schemes.
(Am J Manag Care. 2010;16(8):601-606)
Physician organizations are being exposed to incentives to drive improvements in quality, yet little is known about which investments may contribute to gains in clinical performance. Findings from this study suggest the following:
- Physician organizations that invest more heavily in care management processes such as patient registries, physician reminders and feedback, patient reminders and education, and case managers may achieve better performance scores.
The use of organized quality improvement efforts such as participation in a quality improvement demonstration program may be associated with increased delivery of recommended care processes, which in the context of this study translated into better performance on the clinical measures that were rewarded in the pay-for-performance program.
As performance-based accountability and payment become more prevalent, these study results suggest potential ways for physician organizations to improve the quality of care they deliver.
Although these strategies are intended to stimulate changes by physician organizations (POs), POs face substantial challenges in understanding what investments they could undertake to improve performance scores and achieve success under P4P and public accountability schemes. It is unknown whether investments in chronic care management processes (CMPs), clinical information technology, or quality improvement (QI) activities are associated with better performance. The limited empirical evidence about what factors are associated with better performance hinders providers’ investment decision making and our collective ability to close the quality gap.
To address this question, we examined the relationship between the use of organized processes to improve quality and PO clinical performance scores collected in the context of California’s P4P program. We examined (1) the use of CMPs, (2) electronic medical record (EMR) capabilities, and (3) participation in externally organized QI initiatives, hypothesizing that greater use of these processes is associated with better clinical performance.
We combined IHA clinical data with survey data from the 2nd National Study of Physician Organizations (NSPO2).9-11 Among 225 unique IHA POs, 180 reported clinical data in 2006 for patients enrolled in commercial health maintenance organization and point-of-service plan products. The IHA’s program allows POs the option of being scored using aggregated health plan encounter data or via audited PO self-reported data; self-reporting groups have higher scores than non–self-reporting groups in large measure because of greater data completeness. To eliminate possible bias due to differential reporting procedures, we used only health plan aggregated data for 14 clinical measures (PO-level numerators and denominators) that existed for all 180 POs.
Control variables included organization type (medical group or IPA), PO size, and percentage of revenue from Medicaid (details are provided in a Technical eAppendix available at www.ajmc.com). We hypothesize that (1) medical groups will perform better than IPAs because of greater integration of CMPs, (2) larger organizations will outperform smaller organizations owing to economies of scale and availability of resources to support QI, and (3) greater Medicaid revenue will be negatively related to performance because of the effect of lower reimbursement, which hinders investments in organized care processes, and challenges in caring for low-income or disabled patients.
Data on organization characteristics and QI processes were collected in the NSPO2 telephone survey conducted between March 2006 and March 2007 among all US POs that had 20 or more physicians and that treated patients with asthma, diabetes, congestive heart failure, or depression. Of 218 California POs with usable contact information, 20 were classified as ineligible (was out of business, did not treat 1 of the target diseases, or had no contact information). The survey response rate for the remaining 198 POs was 64.1%. We matched 127 California NSPO2 respondents with 180 POs having IHA clinical scores, resulting in a final sample of 108 POs. We found no significant differences between these 108 POs and 72 POs with IHA clinical data and without NSPO2 data in terms of medical group or IPA structure and the number of physicians in the group; however, the 108 POs were more likely to have greater enrollment, have fewer Medicaid enrollees, and be located in northern California. RAND Corporation’s Human Subjects Protection Committee approved the study.
Dependent Variables. We constructed 3 dependent variables (diabetes management, processes of care, and intermediate outcomes) using an opportunities compositing approach, summing all instances in which recommended care was delivered or an outcome was achieved and dividing by the number of times that patients within a PO were eligible for relevant indicators in each composite measure.1,12 The mean scores range from 0 to 1 and reflect the percentage of opportunities that were delivered. The individual measures were constructed using National Committee for Quality Assurance Health Employer Data and Information Set specifications.13 The diabetes management composite reflected screening and control measures, the intermediate outcomes composite incorporated outcomes for coronary artery disease and diabetes, and the processes-of-care composite included 11 clinical measures (preventive screens, immunizations, asthma maintenance, and upper respiratory tract infection).
Independent Variables. We included the use of CMPs, EMR capabilities, and participation in external QI initiatives as index variables. The CMP measures, based on the chronic care model by Wagner et al,14,15 captured whether the PO used or provided the following: (1) electronic registries or patient lists, (2) guideline-based reminders, (3) feedback data to physicians, (4) patient reminders for follow-up care, (5) nonphysician staff to educate patients about CMPs, and (6) nurse case managers. A process-specific measure was constructed for each of these 6 CMPs (based on whether the PO used or provided the CMPs for diabetes, asthma, and congestive heart failure), resulting in a score ranging from 0 to 3 for each CMP. Construction of the CMP index was tailored to each dependent variable, with a diabetes-specific index ranging from 0 to 6 for the diabetes management composite, an index ranging from 0 to 18 for the processes-of-care composite, and an index ranging from 0 to 6 for the intermediate outcomes composite.
Electronic medical record capabilities were measured on an index ranging from 0 to 5 based on EMR components used by most PO’s physicians such as ambulatory care progress notes or electronic prescribing.2,16-19 Participation in external QI initiatives was measured using an index ranging from 0 to 2 based on whether the PO participated in QI demonstration programs (eg, Bridges to Excellence,20 Institute for Healthcare Improvement Quality Collaboratives,21 Pursuing Perfection,22 Improving Chronic Illness Care,23 rapid-cycle QI activities24).
The model controlled for differences in PO size, organization type (medical group or IPA), and percentage of revenue from Medicaid. Because of the skewed nature of PO size, this was specified as the natural logarithm of the number of physicians in the practice. Table 1 gives descriptive statistics of the independent and dependent variables.
We ran a multivariate ordinary least squares regression model weighted for different sample sizes of the denominator in the “opportunities” composite dependent variable to address heteroskedasticity that results from using aggregated data. Because the dependent variables are PO means, the equal variance assumption of the regression is violated if the sample sizes across POs are different, which could lead to the treatment of smaller POs as more important than they really are without the correction. We ran a Pearson product moment correlation matrix using a 2-tailed significance test to assess the degree of collinearity among the variables in the model. Weak associations were found between our independent variables (<±0.20) except for size and group structure (value, 0.42) and medical group and EMR capabilities (value, 0.38). Given the small sample, we used the dfbeta statistics in SAS (SAS 9.1; SAS Institute, Cary, NC) to test the sensitivity of our results to possible influential observations.
The signs on all coefficients were as hypothesized (Table 2), and the models achieved moderate predictive power (R2 range, 0.17-0.30). The CMP index demonstrated significant positive associations with performance on 2 of the composite measures, namely, diabetes management and intermediate outcomes. Higher performance in diabetes management (3.2 points higher on a 0-100 performance scale) was associated with substantial investments in CMPs (>5 CMPs on a 0-6 scale); each 1.0-point increase on the CMP index translated into a 1.0-point gain for the intermediate outcomes composite (P <.001). Higher engagement in external QI initiatives was significantly positively associated with the processes-of-care component; a 1.0-point increase on the QI index translated into a 1.4-point gain on the CMP index (P = .02). The results were robust to the removal of influential observations.
Among the control variables, medical group organization type was significantly associated with higher performance for 2 of the composite measures (3.0-4.6 points higher for medical groups compared with IPAs). Physician organization size was positively associated with higher performance on the processes-of-care composite (1.5 points) (P = .002). The median PO size was 168 physicians (minimum of 8 and maximum of 2207); therefore, the net effect of increasing the number of physicians within a PO from 10 to 100 physicians on the log scale would translate into a 3.5-point gain for the processes-of-care composite, with an effect size of 1.5 (Table 2). We observed no relationship between Medicaid revenue and performance.
Enhancing our understanding of the types of investments that will yield positive gains in performance is essential to enabling POs and their physicians to improve care for patients. Although causality in a cross-sectional design cannot be established, the associations we observed in this broad sample of physician group practices suggest a positive relationship between clinical performance and the use of CMPs and involvement in external QI initiatives. A positive association was observed between the use of CMPs and clinical performance (approximately 1.0-3.2 points higher) for 2 of 3 composite clinical outcomes measured in this study. We assumed that the component parts of our CMP measure are equally valued; in other words, adding a patient registry has the same positive relationship with performance as providing patient feedback. A larger study could help to discern whether the differential contributions of specific CMPs such as disease registries or performance feedback to physicians are more important for improving clinical performance or whether the broader set of CMPs is required.