Sarah Hudson Scholle, MPH, DrPH; Joachim Roski, PhD, MPH; Daniel L. Dunn, PhD; John L. Adams, PhD; Donna Pillitterre Dugan, MS; L. Gregory Pawlson, MD, MPH; and Eve A. Kerr, MD, MPH
Objective: To evaluate measurement of physician quality performance, which is increasingly used by health plans as the basis of quality improvement, network design, and financial incentives, despite concerns about data and methodological challenges.
Study Design: Evaluation of health plan administrative claims and enrollment data.
Methods: Using administrative data from 9 health plans, we analyzed results for 27 well-accepted quality measures and evaluated how many quality events (patients eligible for a measure) were available per primary care physician and how different approaches for attributing patients to physicians affect the number of quality events per physician.
Results: Fifty-seven percent of primary care physicians had at least 1 patient who was eligible for at least 1 of the selected quality measures. Most physicians had few quality events for any single measure. As an example, for a measure evaluating appropriate treatment for children with upper respiratory tract infections, physicians on average had 14 quality events when care was attributed to physicians if they saw the patient at least once in the measurement year. The mean number of quality events dropped to 9 when attribution required that the physician provide care in at least 50% of a patient’s visits. Few physicians had more than 30 quality events for any given measure.
Conclusions: Available administrative data for a single health plan may provide insufficient information for benchmarking performance for individual physicians. Efforts are needed to develop consensus on assigning measure accountability and to expand information available for each physician, including accessing electronic clinical data, exploring composite measures of performance, and aggregating data across public and private health plans.
(Am J Manag Care. 2009;15(1):67-72)
Measurement of physician quality performance is increasingly used by health plans as the basis for quality improvement, network design, and financial incentives.1
Still, efforts to measure physician performance face a number of challenges, in particular the need for sufficient sample size to support reliable measurement and the lack of consensus on methods for attributing patient measures to clinicians.2,3
Researchers have noted that measurement and comparison of physician quality can be hampered by sample size.4
A minimum threshold of 30 patients is a common guideline for supporting comparisons for an individual measure,5
and evidence suggests that at least 35 to 45 observations are needed to make valid comparisons.6,7
One challenge in obtaining sufficient sample size relates to the measure itself. Many quality measures describe a select group of patients and, by definition, will yield a small number of patients for any physician. Other measures apply to larger proportions of patients, but the ability to capture information on a physician’s entire panel of patients is limited (as when performance measurement relies on data from a single health plan).
A related issue in quality measurement is attribution. Which physicians should be responsible for a quality measure? Given the current focus on team-based chronic disease care and the reality that most patients receive care from multiple clinicians,8
some authors argue that the most appropriate level of accountability is not the individual physician but rather a formal or informal group of physicians.9
Healthcare organizations often attribute patient quality measures based on utilization or a specific set of services, despite the challenges in identifying which physician should be held responsible for the fulfillment (or lack of fulfillment) of a quality measure.
Efforts are needed to understand how these issues may affect the meaningfulness and soundness of physician profiling efforts. In this study, we used a data set that is typical of the information used by health plans to characterize physician performance. Using 27 well-accepted measures that can be obtained from administrative data, we evaluated (1) how many quality events were available per physician and (2) how different attribution rules affect the number of quality events.METHODSData Sources
Administrative claims and enrollment data from the Ingenix Impact Pro database10 for individuals enrolled in 9 health plans for 2003 and 2004 were available for this study. The Impact Pro database is built from deidentified health insurance claims and enrollment information contributed by different managed care organizations. Each of 9 plans selected for the study had at least 250,000 members and accounted for 15% to 50% of managed care enrollees in their markets. During the study period, 170,168 primary care physicians (PCPs) provided care to members of these plans. More details on the study methods are available elsewhere.11Selection of Measures and Attribution to Physicians
We focused on 27 measures describing acute, chronic, and preventive care activities performed by PCPs. Only measures that could be obtained through administrative claims data were included. eAppendix Table 1
(available at www.ajmc.com
) lists all quality measures used in this study, as well as the period used to attribute patients and quality events to physicians.
We identified physicians by the unique identifiers used by each health plan. Primary care physicians, including family physicians, general internists, and general pediatricians, were identified based on their specialty designated in health plan credentialing records.
In selecting an attribution approach, we considered the interactions between clinicians and patients in the course of delivering care, the kinds of services involved, the evidence of a physician’s involvement in the patient’s care, and the data sources available. For this study, we applied a measure-specific attribution logic based on administrative data. Measures were attributed to PCPs based on the outpatient visits they provided to patients during a prescribed time frame specific to each measure. Visits were defined using Healthcare Effectiveness Data and Information Set codes for preventive and ambulatory health services.5
To test a less stringent approach to attribution, a patient measure was attributed to a physician if the patient had 1 or more visits during the prescribed time frame. In addition to this “1-visit” rule, 2 more stringent rules were assessed: a PCP was attributed responsibility for a patient’s measure (1) if the patient completed at least 30% of his or her ambulatory visits with that physician (30% rule) and (2) if the patient completed at least 50% of his or her ambulatory visits with that physician (50% rule).
A quality event occurred each time a patient was eligible for a quality measure. Therefore, a single patient could contribute multiple quality events if he or she was eligible for multiple measures (eg, preventive screening and another measure).Statistical Analysis
We computed summary information describing the number and proportion of physicians attributed with quality events for eligible patients for each attribution approach. We also examined the proportion of physicians with more than 30 quality events for each individual measure and the proportion of quality events accounted for by those physicians with more than 30 quality events. More detailed results are provided in eAppendix Table 2
and eAppendix Table 3
(available at www.ajmc.com
). All analyses were conducted by staff at Ingenix and the National Committee for Quality Assurance using SAS version 9.0 (SAS Institute, Inc, Cary, NC).12RESULTS
Overall, 57% of 170,168 PCPs represented in the study claims data could be attributed responsibility for at least 1 quality event (ie, ≥1 of their patients was eligible for ≥1 of our selected quality measures). Table 1
summarizes findings based on the 1-visit rule and describes the percentage of PCPs with more than 30 quality events for a measure. Except for preventive measures, few PCPs had more than 30 observations for any given measure. However, these high-volume providers account for a larger share of quality events overall, particularly for preventive care measures. For example, only 17% of physicians had more than 30 quality events for colorectal cancer screening, but these physicians accounted for 78% of the quality events for this indicator. Only 1% of physicians had more than 30 quality events for annual glycosylated hemoglobin testing among patients with diabetes mellitus, but they accounted for 16% of the quality events for this measure.Table 2
summarizes how moving from a less stringent rule to a more stringent rule for attribution affects the number of patients available for characterizing physician performance. For example, using the 1-visit rule for the measure assessing appropriate care for upper respiratory tract infections in children, physicians on average had 14 eligible patients for that measure in the measurement year. The mean number of quality events dropped to 11 when care was attributed using the 30% rule and to 9 when care was attributed using the 50% rule. Relative to the 1-visit rule, the 50% rule reduced by about half the number of quality events per physician for a measure. Adopting a more stringent rule for a measure also reduced the number of PCPs with at least 1 quality event for that measure (data not shown).DISCUSSION
Even evaluating the large health plans included in the study and using a less stringent approach to measure attribution, few physicians had more than 30 quality events available to characterize their performance on key quality measures such as colorectal cancer screening or diabetes care. Thirty observations represent a common threshold for adequate denominan tor size in performance measurement; the number of observations needed to gain reliable measurement at the physician level may be higher (or lower) depending on the betweenphysician variation in performance on a given measure.6,7,11
Still, for many measures, physicians with a high volume of quality events account for a significant percentage of all quality events observed. Using more stringent and specific rules to assign patients to physicians (by requiring that a larger proportion of a patient’s care was managed by that physician) further decreased the number of quality events attributable to any given physician.
Our findings illustrate the challenges of benchmarking individual physician performance using available administrative data from individual health plans. Pham and colleagues8 recently noted that care for patients covered by Medicare is frequently shared among multiple providers and concluded that this dispersion of patients could limit the effectiveness of pay-for-performance initiatives because of the lack of accountability on the part of individual physicians. Our data go further to show that, even if accountability is assumed, there is limited information available to characterize physician performance on actual quality measures for single private sector health plans.
PDF is available on the last page.