Cheryl L. Damberg, PhD; Stephen M. Shortell, PhD, MPH, MBA; Kristiana Raube, PhD, MPH; Robin R. Gillies, PhD; Diane Rittenhouse, MD, MPH; Rodney K. McCurdy, MHA; Lawrence P. Casalino, MD, PhD; and John Adams, PhD
Objectives: To examine the association between performance on clinical process measures and intermediate outcomes and the use of chronic care management processes (CMPs), electronic medical record (EMR) capabilities, and participation in external quality improvement (QI) initiatives.
Study Design: Cross-sectional analysis of linked 2006 clinical performance scores from the Integrated Healthcare Association’s pay-for-performance program and survey data from the 2nd National Study of Physician Organizations among 108 California physician organizations (POs).
Methods: Controlling for differences in PO size, organization type (medical group or independent practice association), and Medicaid revenue, we used ordinary least squares regression analysis to examine the association between the use of CMPs, EMR capabilities, and external QI initiatives and performance on the following 3 clinical composite measures: diabetes management, processes of care, and intermediate outcomes (diabetes and cardiovascular).
Results: Greater use of CMPs was significantly associated with clinical performance: among POs using more than 5 CMPs, we observed a 3.2-point higher diabetes management score on a performance scale with scores ranging from 0 to 100 (P <.001), while for each 1.0-point increase on the CMP index, we observed a 1.0-point gain in intermediate outcomes (P <.001). Participation in external QI initiatives was positively associated with improved delivery of clinical processes of care: a 1.0-point increase on the QI index translated into a 1.4-point gain in processes-of-care performance (P = .02). No relationship was observed between EMR capabilities and performance.
Conclusion: Greater investments in CMPs and QI interventions may help POs raise clinical performance and achieve success under performance-based accountability schemes.
(Am J Manag Care. 2010;16(8):601-606)
The lackluster performance of the US health system1,2
has led to several interventions, including provider profiling,3
transparency of performance information,4 and pay for performance (P4P),5-7
that are being implemented in an effort to close the quality gap. A key example is the Integrated Healthcare Association (IHA)’s P4P initiative, which is the largest P4P program in the United States, targeting 225 California managed care medical groups and independent practice associations (IPAs). The IHA’s P4P program uses financial incentives and public reporting of performance scores to drive improvements in clinical quality, patient experience, and adoption of information technologies.8
Although these strategies are intended to stimulate changes by physician organizations (POs), POs face substantial challenges in understanding what investments they could undertake to improve performance scores and achieve success under P4P and public accountability schemes. It is unknown whether investments in chronic care management processes (CMPs), clinical information technology, or quality improvement (QI) activities are associated with better performance. The limited empirical evidence about what factors are associated with better performance hinders providers’ investment decision making and our collective ability to close the quality gap.
To address this question, we examined the relationship between the use of organized processes to improve quality and PO clinical performance scores collected in the context of California’s P4P program. We examined (1) the use of CMPs, (2) electronic medical record (EMR) capabilities, and (3) participation in externally organized QI initiatives, hypothesizing that greater use of these processes is associated with better clinical performance.METHODSData Sources
We combined IHA clinical data with survey data from the 2nd National Study of Physician Organizations (NSPO2).9-11
Among 225 unique IHA POs, 180 reported clinical data in 2006 for patients enrolled in commercial health maintenance organization and point-of-service plan products. The IHA’s program allows POs the option of being scored using aggregated health plan encounter data or via audited PO self-reported data; self-reporting groups have higher scores than non–self-reporting groups in large measure because of greater data completeness. To eliminate possible bias due to differential reporting procedures, we used only health plan aggregated data for 14 clinical measures (PO-level numerators and denominators) that existed for all 180 POs.
Control variables included organization type (medical group or IPA), PO size, and percentage of revenue from Medicaid (details are provided in a Technical eAppendix
available at www.ajmc.com
). We hypothesize that (1) medical groups will perform better than IPAs because of greater integration of CMPs, (2) larger organizations will outperform smaller organizations owing to economies of scale and availability of resources to support QI, and (3) greater Medicaid revenue will be negatively related to performance because of the effect of lower reimbursement, which hinders investments in organized care processes, and challenges in caring for low-income or disabled patients.
Data on organization characteristics and QI processes were collected in the NSPO2 telephone survey conducted between March 2006 and March 2007 among all US POs that had 20 or more physicians and that treated patients with asthma, diabetes, congestive heart failure, or depression. Of 218 California POs with usable contact information, 20 were classified as ineligible (was out of business, did not treat 1 of the target diseases, or had no contact information). The survey response rate for the remaining 198 POs was 64.1%. We matched 127 California NSPO2 respondents with 180 POs having IHA clinical scores, resulting in a final sample of 108 POs. We found no significant differences between these 108 POs and 72 POs with IHA clinical data and without NSPO2 data in terms of medical group or IPA structure and the number of physicians in the group; however, the 108 POs were more likely to have greater enrollment, have fewer Medicaid enrollees, and be located in northern California. RAND Corporation’s Human Subjects Protection Committee approved the study.Analytic ApproachDependent Variables.
We constructed 3 dependent variables (diabetes management, processes of care, and intermediate outcomes) using an opportunities compositing approach, summing all instances in which recommended care was delivered or an outcome was achieved and dividing by the number of times that patients within a PO were eligible for relevant indicators in each composite measure.1,12
The mean scores range from 0 to 1 and reflect the percentage of opportunities that were delivered. The individual measures were constructed using National Committee for Quality Assurance Health Employer Data and Information Set specifications.13
The diabetes management composite reflected screening and control measures, the intermediate outcomes composite incorporated outcomes for coronary artery disease and diabetes, and the processes-of-care composite included 11 clinical measures (preventive screens, immunizations, asthma maintenance, and upper respiratory tract infection).Independent Variables.
We included the use of CMPs, EMR capabilities, and participation in external QI initiatives as index variables. The CMP measures, based on the chronic care model by Wagner et al,14,15
captured whether the PO used or provided the following: (1) electronic registries or patient lists, (2) guideline-based reminders, (3) feedback data to physicians, (4) patient reminders for follow-up care, (5) nonphysician staff to educate patients about CMPs, and (6) nurse case managers. A process-specific measure was constructed for each of these 6 CMPs (based on whether the PO used or provided the CMPs for diabetes, asthma, and congestive heart failure), resulting in a score ranging from 0 to 3 for each CMP. Construction of the CMP index was tailored to each dependent variable, with a diabetes-specific index ranging from 0 to 6 for the diabetes management composite, an index ranging from 0 to 18 for the processes-of-care composite, and an index ranging from 0 to 6 for the intermediate outcomes composite.
Electronic medical record capabilities were measured on an index ranging from 0 to 5 based on EMR components used by most PO’s physicians such as ambulatory care progress notes or electronic prescribing.2,16-19
Participation in external QI initiatives was measured using an index ranging from 0 to 2 based on whether the PO participated in QI demonstration programs (eg, Bridges to Excellence,20
Institute for Healthcare Improvement Quality Collaboratives,21
Improving Chronic Illness Care,23
rapid-cycle QI activities24
The model controlled for differences in PO size, organization type (medical group or IPA), and percentage of revenue from Medicaid. Because of the skewed nature of PO size, this was specified as the natural logarithm of the number of physicians in the practice. Table 1
gives descriptive statistics of the independent and dependent variables.Statistical Analysis
We ran a multivariate ordinary least squares regression model weighted for different sample sizes of the denominator in the “opportunities” composite dependent variable to address heteroskedasticity that results from using aggregated data. Because the dependent variables are PO means, the equal variance assumption of the regression is violated if the sample sizes across POs are different, which could lead to the treatment of smaller POs as more important than they really are without the correction. We ran a Pearson product moment correlation matrix using a 2-tailed significance test to assess the degree of collinearity among the variables in the model. Weak associations were found between our independent variables (<±0.20) except for size and group structure (value, 0.42) and medical group and EMR capabilities (value, 0.38). Given the small sample, we used the dfbeta statistics in SAS (SAS 9.1; SAS Institute, Cary, NC) to test the sensitivity of our results to possible influential observations.RESULTS
The signs on all coefficients were as hypothesized (Table 2
), and the models achieved moderate predictive power (R2
range, 0.17-0.30). The CMP index demonstrated significant positive associations with performance on 2 of the composite measures, namely, diabetes management and intermediate outcomes. Higher performance in diabetes management (3.2 points higher on a 0-100 performance scale) was associated with substantial investments in CMPs (>5 CMPs on a 0-6 scale); each 1.0-point increase on the CMP index translated into a 1.0-point gain for the intermediate outcomes composite (P <.001). Higher engagement in external QI initiatives was significantly positively associated with the processes-of-care component; a 1.0-point increase on the QI index translated into a 1.4-point gain on the CMP index (P = .02). The results were robust to the removal of influential observations.
PDF is available on the last page.