The American Journal of Managed Care January 2010
'All-or-None' (Bundled) Process and Outcome Indicators of Diabetes Care
In this study, providers were more likely to achieve processes-ofcare goals when diabetes care was bundled at the indicator level than at the patient level.
Objectives: To evaluate processes and outcomes of diabetes care using bundled indicators from a primary care registry of osteopathic training programs.
Study Design: Retrospective cohort analysis.
Methods: This study examined care delivered to 7333 patients across 95 family practice and internal medicine residency programs (July 1, 2005, through September 15, 2008) to determine diabetes care performance using measures of processes of care and outcomes. Two summary (bundled) reports of care for each measure were constructed. The first used the frequency of indicated care delivered (indicator-level bundle), and the second used the frequency of patients’ receiving all indicated care (patient-level bundle).
Results: Use of the indicator-level bundle demonstrated that outcomes goals were achieved at a rate of 44.5%. Use of the patient-level bundle demonstrated that outcomes goals were achieved at a rate of only 16.2%, a significant difference (P <.001). Eight evidence-based processes of diabetes care were then examined using the 2 bundling methods. The indicator-level analysis mean rate for the bundled processes of care was 77.3%, whereas the patient-level analysis mean rate was only 33.5%. This was also significantly different (P <.001).
Conclusions: The method of bundling care measures can have a profound effect on the reporting of goals achieved. This can in turn influence the assessment of provider performance and opportunity gaps in diabetes care delivery. In this study, providers were more likely to achieve processes-of-care goals when diabetes care was bundled at the indicator level than at the patient level. Standardization of summary reporting of diabetes care should be developed to enhance consistent interpretation of performance.
(Am J Manag Care. 2010;16(1):25-32)
The quality of diabetes care in the United States falls short of national standards, and performance measurement is intended to improve quality of care.
- Bundling performance outcomes may provide a more patient-centered measure of quality of care.
- The way in which processes of care and outcomes are bundled has a significant effect on quality outcomes performance levels.
- The results of this study should inspire further discussion about the methods by which quality of care should be measured and reported.
Despite high-quality studies6,7 supporting the benefit of multimodal intensive diabetes management, care has fallen short by all measures.8-10 For example, it has been repeatedly shown that less than 50% of persons with diabetes achieve target glycosylated hemoglobin (A1C) levels.11 One proposed method of improving diabetes care is to create incentives for physicians to better manage patients. Performance measurement is a system that can be used to provide incentives for care. With the recent increased focus on physician performance by the Centers for Medicare & Medicaid Services and by other payers, ambulatory measures of quality in diabetes care have been developed.3,11-13
Many experts believe that economic incentives are not aligned to reward higher quality of care. The financial incentives of the US primary care health system are based on the number of patients seen (quantity of care), not on quality of care. However, momentum is gaining to provide incentives for quality of care, or pay for performance. In a survey of 252 health maintenance organizations, more than half (covering >80% of the total enrolled) included pay for performance in their contracts.14 Several clinical trials have evaluated pay for performance.15-17 Lindenauer et al18 reported that hospitals that engaged in pay for performance achieved greater improvements in all composite measures of quality.
As performance measures of care have proliferated, there has been a drive to create summary measures of provider care. The next generation of performance measures may move beyond individual care goals and give recognition only when all composite end points have been reached.15,17,18 The theory behind this “all-or-none” (bundled) performance measurement is that, if all steps are not completed or outcomes achieved, the quality of care is still lacking. Models that measure bundled performance have been used in the measurement of hospital-delivered care. The Centers for Medicare & Medicaid Services in their 8th Scope of Work19 moved to a bundled approach in defining hospital care measures. For example, this has been applied to pneumonia care, congestive heart failure, and acute myocardial infarction.19 In addition, this model has been successful in reducing surgical infection rates in the hospital.20 However, the effect of bundling is unexplored in the outpatient setting.
There are several ways that measures can be bundled. Care can be bundled by the processes of care that are completed. This evaluates the systems built into a practice that assure continuity of care, such as reminders for eye examinations among persons with diabetes. More commonly, intermediate outcomes can be bundled to determine if all goals (eg, low-density lipoprotein cholesterol level, blood pressure, and A1C level) are achieved. This measures actions of the patient and of his or her physician.
Furthermore, these 2 measures can be bundled by patient and by indicator. This bundling method can be applied to processes-of-care and outcomes achievement. The indicatorlevel bundle is the percentage of all processes of care indicated for all patients that are performed, and the patient-level bundle is the percentage of all patients who have received all indicated processes of care. An example is given in Table 1.
The relative value of these methods depends on how the performance information is being used. The indicator-level method provides a measure of operational efficiency, whereas the patient-level method provides a more patient-centric measure, potentially having more meaning to a patient and answering the question “what is my probability of receiving all indicated care or achieving all recommended outcomes?” The American Osteopathic Association developed the Clinical Assessment Program (AOA-CAP) database to serve as a quality improvement tool for physicians in training to evaluate the safety of patient care in the ambulatory setting. This primary care registry of osteopathic training programs uses evidence-based standards of care to consistently collect information on diabetes care.
The objective of this study was to evaluate diabetes care at family practice and internal medicine osteopathic residency training programs using the AOA-CAP database. Furthermore, we evaluated how the bundling of processes-of-care and outcomes measures affected the overall performance score. This study was approved by the Ohio State University Institutional Review Board.
The AOA-CAP database, a Web-based primary care registry of osteopathic training programs, was used in this study. The AOA-CAP database collects information from family practice and internal medicine residency programs on processes of care and outcomes in a sample of their patients. For this study, we only accessed the diabetes measure data set. To enter information in the AOA-CAP database, residency programs are instructed to acquire a random sample from their diabetes medical records. Residents enter data using a standard set of disease-specific processes-of-care and outcomes measures. These reported measures are guided by national standardsetting organizations such as National Committee for Quality Assurance and the American Diabetes Association. These data are provided to the AOA annually from programs as part of the residency accreditation process. Reports regarding performance are then provided back to the program.
Subjects and Settings
Data were abstracted from AOA family practice and internal medicine residency programs between July 1, 2005, and September 15, 2008. Residents were instructed to enter only those patients having confirmed diagnosis of type 2 diabetes mellitus with at least 2 visits to the clinic in the previous year for diabetes. Patients treating their disease with lifestyle modification only were not included in this study. Programs were asked to choose 40 randomly selected patients who met the inclusion and exclusion criteria for the AOA-CAP database. However, not all programs had 40 patients who met these criteria. Programs contributing fewer than 20 patients were excluded from analysis. Data entered into this database were deidentified. The database provides information on care delivered to patients with diabetes, defined as having at least 2 visits with an International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis of diabetes mellitus during the study year and being treated for diabetes with a medication during the study year.
Processes-of-Care and Outcomes Measures Processes and outcomes measures of diabetes care were used to assess the adequacy of diabetes care (see eAppendix available at www.ajmc.com). The processes-of-care and outcomes measures are consistent with those recommended by the National Quality Forum, National Committee for Quality Assurance, and American Medical Association Physician Consortium for Performance Improvement. Measures were vetted by the AOACAP steering committee. Processes-of-care measures identify the interaction between healthcare providers and patients, including diagnosis, surveillance of complications, and treatment of disease. Outcomes measures are the result of the interaction between patient and physician and the ability to get a patient to target goal. A summary of diabetes processes-of-care and outcomes indicators is given in Table 2.
The processes-of-care and outcomes measures were bundled using 2 methods. These are indicator-level and patientlevel analyses.
Processes of Care. An indicator-level processes-of-care bundle was created by developing a denominator of all processes of care for which patients with diabetes were eligible and a numerator of the number of times the indicated process of care was delivered. A patient-level processes-of-care bundle was created by using the patients as the denominator and the number of times the patients received all indicated care as the numerator.
Intermediate Outcomes. An indicator-level outcomes bundle was created by using the denominator of all opportunities for patients to achieve goals of blood pressure, lipid levels, and glucose control. The numerator represents the number of times the goals were achieved across all patients. Similarly, a patient-level outcomes bundle was created by using the patients as the denominator and the number of times the patients achieved all of the following goals: blood pressure less than 130/85 mm Hg, low-density lipoprotein cholesterol level less than 100 mg/dL, and A1C level less than 7% (to convert cholesterol level to millimoles per liter, multiply by 0.0259).
Percentile distributions of programs were calculated based on the indicator or the patient as the unit of analysis. Performance was based on the proportion of goals achieved. SAS version 9.1 (SAS Institute, Cary, NC) was used in the percentile calculations.
The 2 methods were examined for differences in performance-based goals achieved using the following 3 comparisons: (1) indicator-level processes-of-care bundle versus indicator-level outcomes bundle, (2) patient-level processes-of-care bundle versus patient-level outcomes bundle, and (3) patient-level processes-of-care bundle versus indicator-level processes-of-care bundle. Pearson product moment correlation X2 analysis was performed. Statistical analysis was set at the 5% level. SPSS version 17.0 (SPSS Inc, Chicago, IL) was used in the calculations.
A total of 95 residency programs contributed 7333 cases of diabetes to the study. Programs contributed a maximum of 818 cases and a minimum of 20 cases, with a mean of 58 cases. The demographics of the patient sample are given in Table 3. The types of residency programs contributing data were almost evenly split, with 52.5% of cases contributed by family practice and the remainder by internal medicine. The mean age of the cohort was 56.9 years, with 56.0% of cases being female. All patients were treated with medication (by study criteria), with 64.8% of patients receiving oral hypoglycemic agents and the remainder receiving insulin or a combination of insulin and oral medication. White race/ethnicity was most frequent at 56.5%, followed by African American at 23.0%, Hispanic at 10.6%, and the remainder being other races/ethnicities or not reported.