Objective: To determine whether a questionnaire that assesses the presence of practice systems is associated with clinical performance rates for diabetes care.

Study Design: Cross-sectional study of the relation between a survey-based measure of practice systems within 5 of the domains of the chronic care model (CCM) and high-quality care for diabetes during 2005 among 40 medical groups in Minnesota.

Methods: Correlations were calculated between (1) practice systems as measured by the Physician Practice Connections–Readiness Survey (PPC-RS) questionnaire and (2) process and outcome measures of diabetes quality from a standardized system managed by Minnesota Community Measurement.

Results: Most process and outcome measures were correlated at 0.31 to 0.52 (P <.05) with the PPC-RS total score as well as with several of the CCM domains. Only yearly eye exams and blood pressure control lacked correlation with any CCM domain, but delivery system redesign and selfmanagement support lacked correlation with quality measures.

Conclusions: As measured by the PPC-RS questionnaire, the presence of practice systems overall and within several domains of the CCM was associated with high-quality care for diabetes. The PPC-RS may be a useful and relatively simple tool for evaluating and guiding improvement of practice systems for diabetes care quality.

(Am J Manag Care. 2008;14:85-92)

In its reports on the problems with medical care quality in the United States, the Institute of Medicine highlighted the need for transformational improvements in care, with particular priority for chronic conditions.^1-3 Subsequent comprehensive performance measurements across the United States by McGlynn et al documented the extent of the quality improvements needed.⁴ Diabetes is a particularly useful condition for highlighting the issues involved in improvement, because (1) it is common and costly; (2) evidence-based quality measures of both care processes and outcomes are endorsed by the National Quality Forum, the National Committee for Quality Assurance (NCQA), and the Ambulatory Quality Alliance; and (3) improving those measures has received a great deal of attention.⁵ We know from the wide variation in performance rates among different practices and from dramatic improvements in some settings that it should be possible to make substantial improvement.^6,7 However, Saaddine et al report relatively small improvements in national performance over the last decade, and only on some measures.⁸

Organized clinical practice systems have been well demonstrated to be the key way to provide more consistent and comprehensive longitudinal care for either preventive services or chronic conditions.^9-13 By practice systems, we mean organized processes designed to ensure that certain information is collected and information or services are provided routinely to patients or healthcare personnel. These systems have been organized into a conceptual framework (the chronic care model [CCM]) that is now widely accepted as the best way of thinking about the activities to improve chronic illness care.^13-15 The CCM organizes systematic care for chronic illness into 6 domains: health system, delivery system redesign, decision support, clinical information system, self-management support, and community resources. Bodenheimer has summarized the evidence for those systems and the CCM,¹⁶ and Feifer et al and O’Connor et al have shown how systems can improve both care processes and patient outcomes.^11,12 Others have shown that implementation of some practice systems were associated with improvement in diabetes care.^7,17

Unfortunately, relatively few of these practice systems are present, even in large medical groups. Casalino et al called these systems care management processes (CMPs) and found that among 1040 physician organizations nationally with at least 20 physicians, the average organization had only 5 of 16 possible CMPs present.⁹ Of the 4 CMPs related specifically to diabetes care, only one third of these organizations used at least 3 of these CMPs, and half used either none or only 1.¹⁸

If practice systems are important for improving the care of diabetes and other chronic conditions, it would be very useful to have a valid and reliable way to measure the presence of such systems in individual clinics and medical groups as a step toward encouraging their presence and as a guide to internal quality improvement efforts. The NCQA has developed the Physician Practice Connections (PPC) tool to serve this need for purposes of research and quality improvement, and as a basis for rewarding the use of systems in office practices. The PPC, which uses the CCM as a conceptual framework, was developed from an extensive literature review and input from experts and key stakeholders, as well as from an analysis of “defects” in office-based practice using Six Sigma methods.¹⁹

A paper version of the PPC, the PPC Readiness Survey (PPC-RS), was adapted for research and quality improvement purposes. It has been tested for reliability against in-practice audits and found to be reasonably accurate in research applications, with a positive predictive value (PPV) that ranged from 55% to 100% for various components when completed by a group’s medical director.²⁰ The main problem in testing was false-negative reports, with each type of respondent tending to underreport some practice systems that appeared to be present according to an on-site audit. Thus, although agreement rates were somewhat lower than the PPV, positive reports of systems were generally confirmable in this research application, where responses would not influence financial or nonfinancial rewards.

A version of the PPC now is being used by the American Board of Internal Medicine and others to encourage wider knowledge and use of systems as a means for improving quality, and in a modified Web-based format that requires attached documentation for each of the survey elements (www.abim.org/pims/details/sepp.aspx). Some health plans are also using the PPC recognition program as a basis for paying for the use of systems in office practice. The same standards are being used by Bridges to Excellence as a component of their pay-for-performance program (www.bridgestoexcellence.org). Although the evidence previously cited links the use of systems to enhanced clinical performance, it is important to be able to demonstrate that there is a relationship between the systems assessed by this instrument and evidence-based measures of quality of care, both for processes and outcomes. We report such a test of this relationship for diabetes care among the 40 medical groups in Minnesota that provide primary care to a large majority of its citizens.

METHODS

Practices participating in Minnesota Community Measurement (MN CM) provided a good opportunity to test the association between the systems tool and clinical process and outcome measures. MN CM has pioneered standardized measurement and public reporting of performance rates for preventive services and various conditions at the medical group level.²¹ More information about this organization and review of their reports on most of the medical groups in the state is available at their Web site: www.mnhealthcare.org. In brief, MN CM is sponsored by all of the health plans in the state and has developed methods to use health plan administrative data to identify patients of each medical group in a way that allows stratified sampling of individual records for chart audits in a standard format and aggregation by a contracted analysis team. Medical group–level rates for process and outcome measures reported by MN CM are based on a minimum of 60 chart reviews per medical group that are performed by external auditors using the same prescribed methods for each medical group. The results also are validated by an independent auditor agency.

Minnesota also has pioneered an overall measure of diabetes care intermediate outcomes, called optimal diabetes care, which was described by Nolan and Berwick in their important commentary urging the use of such “all-or-none” measures.22 This measure is based on individuals as the unit of analysis, so that each patient is only counted as a positive for this measure if all of the following goals has been achieved: glycosylated hemoglobin (A1C) ≤8.0%, low-density lipoprotein (LDL) cholesterol <130 mg/dL, blood pressure <130/85 mm Hg, and documentation of both regular aspirin use and nonsmoking status in the medical record. As a result of such a stringent requirement, the average performance rate is about 15%, although a few individual physicians have attained rates of 50%.23 This study relies primarily on the results of this measure as calculated by MN CM from care provided in 2005. It also reports rates from that same source for the 5 individual components as well as for 5 process measures (testing rates for A1C, LDL cholesterol, blood pressure, eye exams, and microalbuminuria).

Out of the approximately 65 medical groups with a large enough cohort of patients to be included in MN CM public reports, only 41 had publicly reported data for both diabetes and depression (the depression indicators were used for a separate study). All 41 groups agreed to participate and all 41 returned completed copies of the PPC-RS survey that allowed us to measure the presence of practice systems relevant for diabetes care.²⁴ Two medical groups shared a common owner, so their performance rates are combined in the MN CM reports. Hence, the results represent 40 unique medical groups.

The PPC-RS survey used in this study was completed by medical directors in the summer of 2005. That PPCRS version consisted of 6 sections with a total of 181 items, and it took approximately 45 minutes to complete. Some of the questions addressed organizational demographics, characteristics, culture, or specific diseases other than diabetes (see the following paragraph) and thus did not constitute measures of the presence or absence of systems related to diabetes. These items are used to describe medical groups and also as covariates when testing the association between systems and quality measures. The PPC-RS version used also did not include measures of community resources because it was difficult to identify measures for this CCM domain that had strong face validity for use at the medical group level. Thus, 53 questions were actually used in the final scoring of the PPC-RS. There were 3 questions related to the health system, 8 related to delivery system redesign, 10 related to the clinical information system, 9 related to decision support, and 23 related to selfmanagement support. The items addressed in each domain are listed in Table 1.

Among the 53 items used in scoring, 33 asked whether a particular system was present or not, and 20 asked whether the system was available electronically or on paper only. These latter 20 items were considered present if the system was available on paper or through electronic systems, or both. The internal performance measurement questions asked whether measures were for individual physicians or practice sites. Wherever a system required unique content for a particular chronic condition, the survey asked separately for responses with regard to diabetes, cardiovascular disease, depression, and asthma. Among the 53 items used for scoring in this study, 19 were specific to diabetes and 34 were not associated with a specific condition. A copy of the survey is available from NCQA (email: scholle@ncqa.org). Because all items were coded as present or not present, the score for each domain represents the proportion of the total possible items present. A total PPC-RS score was calculated as the mean of the 5 domains.

The range, mean, and standard deviation were calculated for each PPC-RS domain score, the total PPC-RS score, and the diabetes process and outcome measures. Bivariate associations within the set of PPC-RS domains and within the set of diabetes outcome measures were tested using Pearson correlations to understand the extent of overlap and uniqueness of these measures. Pearson correlations of PPC-RS scores and diabetes outcome measures were computed. Because both measures were assumed to be interval-level data, and scatter plots of PPC-RS scores and outcomes suggested linear relationships, we favored Pearson correlations over other measures of association. Correlations of PPC-RS scores and outcomes were repeated to partial out medical group location, number of physicians, and type of practice.

This study was reviewed, approved, and monitored by the HealthPartners Institutional Review Board.

RESULTS

The main descriptive characteristics of the participating medical groups are reported in Table 2. Medical group locations were evenly split between those in the metropolitan area of Minneapolis and St. Paul and those outside this region. The sample demonstrates substantial variation, but nearly all primary care physicians in Minnesota work in single-specialty or multispecialty groups, so all groups are large relative to what occurs in most of the United States. The number of physicians at these medical groups ranged from 7 to 2000 with a median of 61.

Table 3 provides descriptive information on both PPC-RS scores and outcome measures. The medical groups have a high proportion of these important practice systems, at least in the case of diabetes. The mean overall proportion of systems is 67%, varying from 56% for delivery system redesign to 83% for health system supports. The 5 PPC-RS scores (excluding the total) have intercorrelations ranging from 0.15 to 0.52 with a mean intercorrelation of 0.38 (data not shown). Despite relatively high proportions of patients who achieved control for important individual diabetes outcome measures, the mean group rate for the composite measure of optimal diabetes care was only 19%.

Table 3 also shows that each of the PPC-RS scores as well as the optimal diabetes care measure vary substantially. Individual diabetes outcome measures also have high variance and ample room for improvement. However, in these practices, 3 of the 5 process measure rates (A1C measurement twice yearly, LDL cholesterol test yearly, and blood pressure measurement yearly) have limited room for improvement and low variance.

In Table 4, PPC-RS total score and individual domain scores are correlated with the optimal diabetes care measure, its components, and individual process performance measures (which have no combined all-or-none score). The PPCRS total score, health system score, and decision support score were consistently and significantly positively associated with the optimal diabetes care measure and most of its components, except for blood pressure control. The other 3 PPC-RS domains had little demonstrable relationship to outcome measures.

The PPC-RS total, clinical information system, and decision support scores also showed significant positive associations with 3 of 5 of the tests that were being performed at high rates (A1C, LDL cholesterol, and blood pressure), but there was little in the way of significant associations with eye exams or microalbumin testing rates. The health system score relates only to LDL testing. And the remaining 2 PPC-RS domains (delivery system redesign and self-management support) were not related to any of the testing measures. When repeating the correlation analysis in Table 4 and partialing out medical group location, number of physicians, type of practice, and percentage of patients with commercial or private insurance, the resulting correlations involving the PPC-RS total score were only slightly attenuated (reduced on average by r = -0.03, data not shown).

The significant correlations between the PPC-RS score and the various outcome and process measures ranged from 0.35 to 0.46, with a mean of 0.42. Although these correlations are not high, they are not trivial in terms of effect size. In this sample of 40 medical groups, a correlation of 0.43 was detected with 80% power.

To illustrate the relative magnitude of performance differences, Table 5 compares medical groups scoring in the lowest quartile on the PPC-RS with those scoring in the highest quartile with respect to mean rates of optimal diabetes care (including the components of this optimal care) and the individual process measures. This comparison shows that even measures with nonsignificant correlations have numerical values in the right direction.

DISCUSSION

These results demonstrate that practice systems for diabetes, as measured by self-report on the PPC-RS survey, correlate fairly well with most process and outcome measures for quality of diabetes care. This finding in one region’s medical groups that have relatively high levels of both practice systems and care quality for patients with diabetes is important for several reasons. First, it suggests that the PPC-RS may be a useful tool for assessing the types of practice infrastructure associated with improving quality of care, at least for diabetes. Second, it confirms findings from controlled trials that suggest that practice infrastructure (and practice systems in particular) are important for providing high-quality care to patients with chronic conditions and shows this relationship in routine practice settings. It also provides support for the idea that the CCM is a useful way of organizing our thinking and quality improvement efforts.

It is interesting that decision support and health system quality improvement systems were especially important domains in this observational study. Sperl-Hillen et al previously demonstrated among 17 clinics in 1 medical group that decision support as measured by the Assessing Chronic Illness Care survey was the only domain of the CCM that was associated with improvement over 1 year in glycemic and lipid control.²⁵ In another study of this group’s diabetes care quality improvements over 10 years, improvements in both A1C and LDL cholesterol were found to have been driven primarily by drug intensification (increasing dose or adding medications), leadership commitment to improvement of diabetes care, greater continuity of primary care, participation in local and national diabetes care improvement initiatives, and allocation of multidisciplinary resources at the clinic level to improve diabetes care.⁷ All but possibly drug intensification are clearly components of health system supports. Finally, Shojania et al conducted a meta-regression analysis of 64 trials of quality improvement strategies for diabetes. Only 2 of 11 strategies were associated with more than small improvements: changes in team structure and care management (where managers could adjust medications without physician approval).²⁶

The Casalino et al studies of CMPs among many large medical groups across the country (National Survey of Physician Organizations) suggest that these care processes frequently are not present. They found an average of only 5 of 16 possible CMPs and 1.7 of 4 possible CMPs specific for diabetes.^9,18 The 4 diabetes CMPs they measured were clinical practice guidelines in conjunction with physician reminder systems, case management, performance feedback to individual physicians, and disease registries. These 4 systems all are covered by PPC-RS questions and relate to 4 of the CCM domains, but constitute only 10% of the systems assessed by that instrument. Thus, the PPC-RS should be thought of as a much more detailed assessment, one only indirectly related to the CMP findings of the National Survey of Physician Organizations studies. Nevertheless, it is interesting that the National Survey of Physician Organizations project found that the number of diabetes CMPs was strongly associated with external incentives to improve quality, computerized clinical information systems, and ownership by hospitals or health maintenance organizations.¹⁸

Fleming et al¹⁷ used a survey that was based on CCM components included in a qualitative instrument called Assessing Chronic Illness Care²⁷ to study health plan activities associated with the 6 Healthcare Effectiveness Data and Information Set (HEDIS)28 measures of diabetes quality in Medicare contract enrollees. Although such systems used by a health plan are inherently different from those used in a medical group or clinic, these researchers found that the mean number of systems for the upper and lower quartiles of quality scores was 17.5 and 12.5, respectively (out of a possible 32). After adjusting for structural and geographic differences, only practitioner input and use of clinical guideline software remained as significant predictors of this difference.

Although promising, our results have limitations. The instrument itself only imperfectly captures data about which systems are present and especially about the extent to which they are extensively used and function well. For example, in our preliminary studies, agreement between the PPC-RS and an on-site audit was very high for quality improvement questions (97%), but only moderate for registry (78%) and clinical information system (71%) questions, and low for care management questions (28%-65%).20 Thus, the low correlations of performance measures with the self-management support domain may have been confounded by this problem. Most of the agreement discrepancies were because medical directors did not report systems that we were able to determine to be present at the audit, so the direction of the bias would be to underestimate the degree of correlation. Although it would have been nice if the domains with higher agreement rates had the strongest relationship to performance, that was not consistently true, probably because agreement was confounded by the relative importance of various domains.

The limited number of medical groups in our sample also made it difficult to test relationships between specific questions within the PPC-RS and our quality measures. Because there were no data about patient characteristics, it was not possible to adjust the performance measures for differing patient populations. The sample size also limited the extent to which the effects of multiple covariates could be examined simultaneously, which is why this study emphasizes the use of bivariate and partial correlations. Thus, larger studies that test these associations over time and that adjust for multiple confounders simultaneously would be valuable. Finally, our region is somewhat atypical in the size of its medical groups, in the extent to which quality is a high priority, and in the extent to which organized practice systems are used to improve quality.

Nevertheless, these preliminary findings are important. They support the importance of practice systems for quality care of diabetes and suggest that internal group use of the PPC-RS may provide a helpful guide for improvement efforts. Moreover, when combined with review of documentation and sample on-site audits, use of the standards in the PPC-RS could be valuable for external assessment by public performance reporting or incentive programs. In that regard, a version of the PPC is being considered by the American College of Physicians, American Academy of Family Physicians, American Academy of Pediatrics, and American Osteopathic Association as the tool for qualifying primary care practices as Patient-Centered Medical Homes.^29,30 As a set of practicespecific structural measures, the PPC-RS would be less subject to some of the issues of sample size, attribution, and risk adjustment that plague clinical process and outcome measures used at the individual practice, site, or small group level.