Sukyung Chung, PhD

To assess quality, cost, physician productivity, and patient experience for 2 primary care physician (PCP) practice styles: the focused, who typically address only the patient’s acute problem, versus the max-packers, who typically address additional conditions also.

 Retrospective observational study using administrative data, electronic health record (EHR) data, and patient surveys. Data represent 285 PCPs (779 PCP-years) in a large, multispecialty group practice during 2011, 2012, and 2013.

 PCPs were ranked each year by their number of additional conditions addressed during acute care visits. The top one-third (max-packers) addressed 25.4% more “other problems” than expected, while focused PCPs (bottom one-third) addressed 20.3% fewer than expected. Outcomes were resource use, clinical quality metrics, patient-reported experience, physician time using the EHR, and physician productivity. All measures were risk-adjusted to account for patient mix. 

Relative to a focused pattern of care, max-packing was associated with 3.4% lower overall resource use, consistently better scores for the available clinical quality metrics, and comparable patient experience (except for worse wait time ratings). Patients of focused PCPs used 7.3% more specialist services, in terms of costs, than patients of max-packers ($1218 vs $1136; 

 <.001). Max-packers spent 40 minutes more per clinical day using the EHR. PCPs with less appointment availability and who used a mix of appointment slots were more likely to be max-packers.

 Max-packing behavior yields desirable outcomes at lower overall cost but involves more conventionally uncompensated PCP time. Alternatives to compensation just for face-to-face visits and using more flexible scheduling may be needed to support max-packing.

Relative to primary care physicians who typically focus only on the patient’s main problem during an acute care visit, those also addressing chronic issues (the max-packers) achieve:

Comparable overall patient experience scores, in spite of longer waits

Lower annual costs, due to less specialty care

See fewer patients per day with longer visits

Spend more time messaging and using the electronic health record

Are undercompensated (per clinical work hour) by traditional fee-for-service models

Have sicker panels not accounted for by simple capitation models

Cannot be readily identified by specialty, age, or sex; it is their behavior that matters

Primary care physicians (PCPs) are increasingly working in large groups, often feeling pressured to see more patients while dealing with demanding electronic health record (EHR) systems.

 Little is known about PCP responses to increasing patient loads. Some emphasize “throughput,” scheduling as many patients as possible by focusing on the specific reason for the visit. Others do the opposite, using any acute care visit to address additional problems, thereby reducing follow-up visits. The latter approach was termed 

 by the Institute for Healthcare Improvement.

 In a study unrelated to ours, a very busy PCP spontaneously explained max-packing: “So my [appointment] access is already poor and I already do…max-packing: You come in for a visit, I take care of everything I need to take care of in order to prevent you from coming back too soon. If you come in for a cold and I know you have an appointment next month to follow up on your high blood pressure and diabetes,…I’m also going to look after your high blood pressure and your diabetes at this visit so you don’t have to come back in a month.”

The value of comprehensive PCP care coordination across visits is demonstrated,

 but there has been little exploration of what occurs during visits (ie, is focused care better or worse than max-packing?). We contrast PCPs within the same system who, during acute care visits, focus on the patient’s main problem versus those who also address other conditions—the “max-packers.”

 We examine resource use, clinical quality, patient experience, physician time, and productivity metrics, along with PCP and practice characteristics associated with max-packing.

This is an exploratory observational study of PCPs treating adult (≥18 years) patients in a large multispecialty group practice (“the Clinic”) with 17 sites spread across 4 urban counties. Nearly all Clinic patients have insurance (14%, health maintenance organization [HMO; Medicare or other]; 61%, commercial preferred provider organization/fee-for-service [FFS]; 10%, Medicare FFS; 2%, Medicaid; 13%, other). PCP compensation reflects their own work relative value units (wRVUs). The Clinic also provides specialty care, ambulatory surgery, and laboratory, imaging, and inpatient physician services. All patient care is recorded in a unified EHR.

Our starting sample is 310 PCPs practicing during 2011 to 2013 (834 PCP-years). To report on “typical PCPs,” we excluded those whose clinical time was less than 20% clinical full-time equivalent (cFTE); who had fewer than 50 active adult Clinic patients, were practicing geriatrics only, were trained in osteopathic medicine, or have a concierge practice; or whose primary service location was inpatient or nursing facilities or was missing appointment scheduling information. This yielded 285 PCPs (779 PCP-years).

New Clinic patients choose a PCP and are considered that PCP’s patient unless they request a change. Patients are included on their PCP’s active patient list (APL) if they have at least 1 visit with any PCP in the previous or current calendar year or at least 1 nonvisit encounter (eg, pharmacy refill, online message) in the current year.

Depending on the metric, somewhat different relevant patient populations were used. PCPs typically see their own patients along with those of other PCPs. When categorizing PCP practice styles, we used only encounters with their own patients—about 75% of all visits. When assessing quality and resource utilization, we used all the PCP’s own patients, even those seeing other providers. When assessing how PCPs use their time, we included all encounters.

Max-packing is an observed behavior, not an inherent physician characteristic. Thus, we categorized each PCP-year as max-packer, middle, or focused, based on visits with their own patients that year (

]). We used an episode grouper (Symmetry Suite 9.2.1; Optum; Eden Prairie, Minnesota) to organize all professional, ancillary, and pharmaceutical services, assigning each (ie, a line on a bill) to 1 of 514 clinically homogenous episodes of care.

 For example, if a patient saw her PCP for acute back pain, then within 30 days saw a specialist for the same problem and the specialist ordered back imaging, those downstream services were considered part of the acute back pain episode and extended the 30-day window. Windows for acute problems are typically 30 to 180 days; chronic problems essentially continue indefinitely. If, during the initial visit, the PCP also adjusted her antihypertensive medications and ordered a glycated hemoglobin (A1C) test, the medications would be assigned to a previous hypertension (chronic) episode and the laboratory test to her chronic diabetes care. The PCP evaluation and management (E&M) cost is assigned to the primary reason for the visit (back pain). Two “phantom” records (ie, without physician costs) are generated because during the face-to-face visit, the physician did something for those conditions (hypertension and diabetes). To minimize potential documentation bias, when calculating phantoms, we ignored conditions for which patients received only “counseling.”

We expected that a PCP who primarily sees young, healthy patients would address fewer “other problems” during acute visits than a PCP with mostly elderly patients. We used regression to calculate the expected number of phantom records, given each patient’s age, sex, and mix of acute and chronic conditions (eAppendix A). Based on the ratio of observed-to-expected phantoms per acute episode with ones’ own patients, we ranked PCP-years; the top one-third in each year are labeled “max-packers” and the bottom one-third are labeled “focused.”

Billing data identify all Clinic services provided. We approximated resource use with standardized costs priced at 150% of Medicare rates, regardless of payer. Prescription drug costs were approximated using medication orders, drug-specific order fill rates, and drug-specific average insurer-based payments. Prescription costs were winsorized to address outlier values. Professional services of Clinic physicians in hospitals were included, but facility and other hospital-based charges were unavailable.

Resource use measures (costs) were mean (1) E&M costs for all acute visits by the PCP’s own patients, (2) total cost per acute episode, and (3) annual total cost per patient on the PCP’s panel. For the second measure, only episodes initially managed by the patient’s own PCP were considered, but resources ordered or provided by other clinicians were included. Following the Clinic’s practice, all resources used by the PCP’s patients were included in the panel measure, even if none were provided by that PCP. At the panel level, we also examined types of services (eg, physician services, laboratory services, imaging, prescriptions) by who provided or ordered them (ie, own PCP, other PCPs or urgent care, and specialists). All resource use measures were risk-adjusted for patient age, sex, and mix of acute and chronic conditions (

The Clinic routinely assessed clinical quality with the Integrated Healthcare Association’s 2011-2013 targeted measures

: the average of 6 diabetes management metrics (A1C testing, low-density lipoprotein cholesterol [LDL-C] screening, medical attention for nephropathy, A1C control [<8.0%], LDL-C control [<100 mg/dL], and blood pressure control [<140/90 mm Hg]) and 3 screening metrics (colorectal cancer screening for adults aged 50-75 years, breast cancer screening for women aged 52-69 years, and evidence-based cervical cancer screening for women aged 24-65 years). Patients with a new diagnosis of diabetes are typically closer to being in control and some PCPs have more such patients, so metrics were risk-adjusted for when the patient became eligible (

The Clinic routinely questioned its patients about their experiences using the Press Ganey Outpatient Medical Practice Survey.

 We focused on 5 of the survey’s care provider (CP) assessments: likelihood of your recommending your CP, amount of time the CP spent with you, friendliness/courtesy of the CP, explanations the CP gave you about your problem or condition, and concern the CP showed for your questions or worries. We also examined patients’ assessments of wait time at the Clinic. For each, a score indicates the percentage responding that it was very good (5 on 1-5 scale). We have found that some patient responses are associated with race/ethnicity in ways apparently unrelated to objective measures,

 so we adjusted for the race/ethnicity of the PCP’s own patients who responded.

In the typical Clinic encounter, the patient checks in and is then escorted from the waiting room by the medical assistant, who takes vital signs and records them in the EHR. The physician enters the exam room, opens the patient’s chart at some point, and logs out before exiting. Access logs facilitate calculating the time each PCP spends with the EHR open in an exam room, in their Clinic office, and remotely.

 An objective lower-bound estimate of time spent with patients is how long the patient’s chart was open with the physician in the exam room. The EHR also records scheduled appointment times.

PCPs are more likely to address multiple issues if there is email or phone communication with the patient prior to or after the visit,

 so we counted such communications within a week of each face-to-face visit. Because physicians may address problems without face-to-face visits, we also counted communications outside of those 1-week windows.

Usual PCP timeliness is the mean difference between appointment time and PCP chart open time for patients checking in before their scheduled appointment time.

At the Clinic, productivity is based on physician wRVUs. This can be assessed as wRVU per cFTE, appointment hours, or total hours. Alternatively, productivity can be measured by the size of the APL per cFTE, either unadjusted (simple head count), adjusted for patient age and sex, or adjusted for patient age, sex, and clinical conditions.

 For family physicians, the panel-based metric includes children and adults.

To compare observed measures of resource use, clinical quality, and patient experience, all relative to the values expected,

 tests were used. The expected values reflect patient mix based on age, sex, and clinical conditions (for cost); new versus established patients (for quality metrics); and race/ethnicity (for patient experience). For patient-focused metrics (cost, quality, experience), we compared observed and expected values for all the relevant patients associated with PCPs in each tertile. For PCP-focused metrics (eg, time spent, panel size, factors associated with behaving as a max-packer), the PCP-year is the unit of observation.

Given the exploratory nature of this work, we did not formally adjust 

 values (most are either highly significant or clearly insignificant). We present results for the middle tertile of PCPs to explore any trend across groups. SAS Enterprise Guide version 7.1 (SAS Institute; Cary, North Carolina) was used for data cleaning and management; statistical analyses were performed using Stata 14.2 (StataCorp; College Station, Texas).

This study was approved by the institutional review boards of the Palo Alto Medical Foundation and Sutter Health.

The number of “other problems” addressed during acute care visits varied markedly. With the exception of the tails, the distribution appeared quite normal, so we split the PCP-years into thirds. The top one-third, max-packers, addressed 25.4% more other problems than expected per acute episode; the bottom one-third, focused PCPs, 20.3% fewer (

 <.001). This behavior pattern is quite consistent over time, with 65% to 70% of the PCPs in the same tertile from one year to the next (eAppendix A).

 presents summary findings for risk-adjusted resource use, clinical quality, and patient experience. Max-packers had the highest relative (risk-adjusted) per-visit E&M and total per-episode costs but lowest annual total per-patient costs. The opposite pattern holds for focused PCPs (

 <.001). For the 4 clinical quality metrics, max-packers scored significantly above expectations for diabetes management, colorectal screening, and breast cancer screening (each 

 <.001); focused PCP scores were all below expectations (

None of the 5 care experience measures differed significantly from expected. For patient assessment of the wait time, however, focused PCPs received markedly better than expected scores (mean [standard error] = 1.06 [0.004]), whereas max-packers scored markedly worse (0.94 [0.004]). Observed EHR-based wait times were approximately 4 minutes longer for max-packers than for focused PCPs (data not shown).

 subsets the annual resource use measures by type of service and who ordered/provided the service. Overall, the patient’s own PCP provided or ordered 32.3% of all costs, other PCPs and urgent care accounted for about 9.4%, and specialists accounted for about 57.1%. Given the marked differences in case mix across the 3 groupings of PCPs, we first compare observed versus expected use based on a regression model with patient risk factors (eAppendix B). This allows us to calculate the percentage by which patients assigned each group have expenditures above (or below) the overall average for each type of service. The data in Figure 2 and that follow here are those normalized costs (details are in 

). With the exception of nearly identical observed and expected own PCP expenditures for patients of the focused PCPs, all other cost measures (observed-to-expected due to case mix) for focused and max-packing PCPs are significantly different (

 <.001). Patients of focused and max-packing PCPs had nearly identical amounts of services rendered and ordered by their own PCPs ($664 vs $666, respectively). Patients of focused PCPs had fewer services rendered or ordered from other PCPs/urgent care ($184 vs $199). In contrast, patients of focused PCPs had services rendered or ordered by specialists that were 7.3% higher ($1218 vs $1136). Ancillary services were comparable, but E&M, laboratory, imaging, and prescription costs were 7% to 15% higher (all 

Max-packing and focused PCPs allocated their time differently (

). Per clinical day, max-packers had 6.8% fewer scheduled appointment hours and 14.5% fewer appointments, yielding 8.2% longer appointments (all 

 <.001). Max-packers’ average time between EHR open and close was 1.47 minutes (15.6%) longer (

Modern clinical care involves more than face-to-face time: Before visits, PCPs may review laboratory results and medical history, while afterward, referrals are made and tests and drugs ordered.

 All 3 groups of PCPs had comparable EHR time in exam rooms, but max-packers logged substantially more EHR time in their offices or remotely. Overall, max-packers spent 40 minutes more EHR time per clinical day compared with focused PCPs (5.6 vs 4.9 hours; 

Max-packers averaged 0.273 non—face-to-face patient contacts (19% more than the focused PCPs) in the week before each visit and 0.269 contacts (26% more) in the week after. There was no difference in contacts per patient-year outside those peri-visit windows.

Max-packers generated slightly fewer wRVUs per cFTE (

 = .39) but 3.7% more per scheduled appointment hour (

 = .03). If, however, clinical time is calculated as appointment time plus EHR time in the office plus EHR time spent remotely, max-packers generated 5.5% fewer wRVUs per clinical hour (

 <.01). An alternative productivity metric is panel size. Per cFTE, max-packers (versus focused PCPs) had smaller panels, even if adjusted for patients’ age and sex (1947 vs 2162; 

 <.001), yet the max-packers’ risk-adjusted panel sizes when accounting for patients’ clinical conditions were larger (2066 vs 2035; 

Articles

Primary care physicians who address multiple problems during acute care visits achieve better clinical scores, comparable patient experience, and lower annual cost.

Primary Care Physician Practice Styles and Quality, Cost, and Productivity

To determine if patient clinical conditions need to be considered when assessing primary care physician (PCP) workload in the context of standardizing panel sizes.

Work resource value units (wRVUs) were used to standardize PCP panel workload. Standardized panels were created using (1) age and sex— and (2) clinical condition–based risk indicators. Billing data were used for all patients, regardless of insurance, for PCPs in a group practice (n = 190). Weighting methods were assessed

for subgroups based on PCP specialty (family medicine, internal medicine, and pediatrics) and patient age (adults vs children) and for different levels of aggregation (patient vs PCP).

Groupwide weights based on wRVUs of all primary care services delivered during the year were applied to individual patients and then aggregated to PCP panels. For age/sex weighting, only patient age and sex were taken into account. For condition-based weighting, 1275 disease categories, based on a combination of episode treatment groups (ETGs) and age and/or sex, were used.

As expected, at the patient level, condition-based weights were far more discriminative than age/sex. At the PCP level, this discrimination was less important; panel weights varied 1.9- (age/sex—based) to 2.6-fold (condition-based) across PCPs. Correlations between the 2 weighting methods were high (r = 0.93) for child panels and moderate (r = 0.71) for adult panels (all 

The heterogeneity of PCP panels should be considered when assessing PCP workload for panel management. Panel variability in workload is well captured by age/sex—based weights for children, but for adults condition-based adjustment may be necessary.

There is a wide variation in patient demographic and clinical characteristics across primary care physician (PCP) panels, even within a single group practice organization. For panel-based productivity or performance assessment, standardization of PCP panels requires adjustment for such differences.

		 Child and adult panels should be assessed separately.

		 For child panels, age/sex—based adjustment is probably sufficient.

		 For adult panels, further adjustment reflecting patient clinical conditions is warranted.

		In the setting studied, the adult and child panels of family practitioners differed from those seen in the panels of general internists and pediatricians, respectively.

		 Panel size standardization should account for unusual, but predictable, coding patterns.

Efforts to slow the healthcare expenditure growth include a potential shift from the current standard fee-for-service (FFS) or productivity-based model to compensating primary care physicians (PCPs) for the appropriate management of a patient panel.

 Many group practices and managed care organizations are exploring such panel-based payment models.

 The transition to panel-based compensation must address each physician’s concern that his or her panel is “sicker than average” and thus requires more physician time. In spite of many flaws with respect to its incentives, FFS-based payment roughly reflects physician work effort and thus is perceived by physicians as fair in this regard. An important first step in panel-based payment is determining the method, or even the need, to account for variations in case mix among physician panels. To facilitate acceptance of change, the new methods should be simple and transparent.

Historically, correcting for patient case mix has been performed using various risk-adjustment techniques. The vast risk-adjustment literature has typically focused on predicting individuals’ costs, including hospitalization and other very expensive services. Age and sex alone perform poorly in predicting individual healthcare costs,

and sophisticated risk-adjustment measures, using indicators of clinical conditions and/or prior healthcare costs, are far superior.

 Physician-level risk adjustment focuses on profiling physicians based on all costs incurred by their patients. These studies found that after including clinical condition indicators, adjustment for condition severity did not change provider ranking

 and methods of dealing with cost outliers and attributing costs to a provider did not substantially influence provider ranking.

 This literature, however, does not address what is necessary when standardizing panels for the PCP’s anticipated work effort. Panel standardization for PCP payment may be a simpler challenge because we need only focus on the (1) primary care work effort and (2) work effort aggregated to the provider panel level. This led us to ask if age/ sex-based weights are good enough for such panel size standardization.

In our study, we seek a simple and robust approach to standardize PCP panel size that is easy to implement and understandable by clinicians. Age and sex categorization meets that requirement, but may be insufficiently accurate. We therefore consider standardized workloads for a panel using 2 weighting methods: (1) finely defined condition-based weights, and (2) simple age/sex—based weights. Poorly constructed conditionbased weights might be no better than age- and sex-based weights, so we compared the 2 for both individual patients and PCP panels to see if the expected higher discrimination of condition- based weights for individuals matters as much for panels. To determine whether panel weights would need to be recalculated each year as panel composition changes, we then examined the stability of adjusted panel sizes over time to see if values change much from year to year.

We used data from 2007 to 2008 on services provided by PCPs in the Palo Alto Division of Palo Alto Medical Foundation (PAMF/PAD), California. We focused on 2008 data except for the cross-year correlations. The primary care departments were located in 7 clinics of varying sizes (number of PCPs ranged from 4 to 72). Most PAMF patients were insured and represented a multitude of plans (preferred provider organization [PPO] 57%, health maintenance organization [HMO] 26%, Medicare 10%, other or no insurance 7%). The average age of all patients with a PCP was 35.4 years, and 57.5% were female. Among adults (

18 years) (n = 281,842, 70.7%), the average age was 47.6 years and 60.1% were female; among children (<18 years), the average age was 6.1 years and 49.9% were female.

Panels were created based on each patient’s identified PCP, as determined at registration and updated whenever a patient formally changes PCPs. On average, 87% of primary care visits were made to the patient’s own PCP. Patients who used any primary care service during the year were included in the panel estimates. We used 2 panel definitions: (a) overall panel, and (b) separate adult (aged >18 years) and child (aged <18 years) panels. After excluding PCP panels of fewer than 100 patients (n = 7), 183 panels (76 family medicine, 62 general internal medicine, and 45 pediatric) were included in the analyses for overall panels. For adult/child-specific panel analyses, 141 adult panels (77 family medicine, 62 internal medicine, and 2 pediatrics) and 110 children panels (65 family medicine and 45 pediatrics) were used, after excluding panels with less than 50 adults or children. Panel size varied widely across PCPs. “Trimmed” overall panels ranged from 203 to 2220 (average 959; standard deviation [SD] = 344) and adult/child—specific panels ranged from 50 to 2126 (average 693; SD = 456).

Our focus was the standardized primary care service workload for each panel, assuming each patient received the average work resource value units (wRVUs) for “similar patients” across all PAMF/PAD PCPs. (Our key question is whether “similar patients” should be characterized by age/sex or by clinical conditions.) This essentially matched the current compensation model at PAMF in which salaries are proportional to wRVUs. Primary care wRVUs were predominantly for evaluation and management services (ie, consultation during office visits). Excluded are material or capital (ie, non-wRVU) costs associated with procedures, imaging, tests, immunizations and injections delivered in the primary care offices, or other services ordered by a PCP but delivered by non-PCP clinicians. A uniform Medicare resource-based relative value scale schedule (RBRVS) was used to eliminate differences in payer source across patients.

For each age (0-90 years, in 1-year increments, top-coded at 90) and sex, the average annual PCP wRVUs per patient in the category was calculated. The resulting age/sex-based weights were then applied to individual patients in each category and further aggregated to the panel level. The distribution of wRVUs differed noticeably by sex after adolescence, with higher PCP-generated wRVUs for females (Figure 1), although care delivered by obstetrics and gynecology specialists was excluded.

The procedure to compute condition-based weights was similar, but with many more categories. Adjustment based solely on diagnosis codes may over-reward clinicians who code more conditions. To reduce this effect, we used Symmetry’s Episode Treatment Groups (ETG version 7.5) to aggregate codes to clinical conditions and attribute wRVUs. The Symmetry ETG system creates episodes of illness, ranging from simple, short-term acute problems to long-term chronic illnesses. The ETG grouping methodology uses data on each service provided and the associated diagnosis. Based on the diagnoses, it assigns each service to 1 of a set of mutually exclusive and collectively exhaustive episodes of care regardless of provider, treatment location, or duration. The Symmetry disease classifi cation methodology is a widely used and validated system for building clinically homogeneous episodes. This is important because: (1) physicians can relate to the illness groupings, allowing for meaningful communication regarding treatment; and (2) clinical homogeneity within an ETG provides the basis for substantive comparison and detailed drill-down analysis.11 Several other systems in the market produce comparable results.

The following example illustrates the approach for a patient with diabetes who visits a PCP for a fall and laceration. During that visit an x-ray is ordered, the laceration sutured, a glycated hemoglobin (A1C) test ordered to check on the patient’s diabetes, and a flu shot given. As the primary diagnosis recorded for the visit is trauma, the evaluation and management component of the visit would be grouped into a trauma ETG. The suturing procedure would also be attributed to that episode. The x-ray cost and its interpretation by a radiologist would also be attributed to the trauma ETG, but in this analysis non-PCP costs are ignored. Likewise, although the A1C test will be assigned to a diabetes ETG, its costs are ignored because the test is provided by the laboratory rather than the PCP. The patient will probably have other visits that capture PCP offi ce visit services associated with diabetes. To examine potential misclassifi cation errors (ie, the episode cost for diabetes in this example does not capture the PCP’s effort in managing the patient’s diabetes), we repeated the entire analysis using episodes based just on the primary diagnosis in the billing data when defining condition-based weights, and found no noticeable difference in patient- and panel-level weights.

Using the 2008 data for PCP-provided services, we identifi ed about 450 outpatientfocused ETGs. We further divided these by age and sex group when there were significant differences in wRVus by age (and/or sex) within a specifi c ETG, using ANOVA, creating 1275 categories. For this step we used 14 age/sex groups defined as 0, 1, 2 to 5, 6 to 49, 50 to 69, 70 to 80, and greater than or equal to 81, multiplied by 2 for each sex. The age breaks refl ected the pattern of age-specific wRVu use observed in the data. For example, average wRVus for infants (6.4) fell after the first year, but were similar for ages 2 to 5 (range 3.0-3.4) (Figure 1). As an example of interaction effects, the “asthma” ETG was split into 14, 1 for each age/sex group defined above. ETG weights were computed for each “ETG cell” that a patient would be in and a patient could have multiple ETGs, so the weights were additive.

Assuming wRVus are a reasonable measure of PCP work effort, the expected panel management “burden” of a patient can be approximated by the average wRVu for that patient. (We are not arguing for the validity of the wRVu as a measure of PCP effort, but it is the 

 basis for most physician compensation.) We assigned each person a weight, refl ecting either age and sex or the sum of the weights associated with each ETG for that patient. To examine whether conditionbased weights added calibration power to age-sex categories, we plotted average condition-based weights across 20 categories (every fifth percentile of condition-based weight distribution) 

We aggregated weights for each patient in each PCP’s panel and compared average weights in a given year using the 2 methods and also assessed shifts in panel weights over time. Pearson’s correlation coeffi cient (r) was used to index the strength of linear association: strong if greater than 0.8, moderate if between 0.5 and 0.79, and weak if less than 0.5, and results were reported when signifi cant at 

Patient Level Variation in Age/Sex—Based and Condition-Based Weights

Primary care wRVus (per patient-year) vary substantially by age and sex (

). With the average patient accounting for 3.2 primary care wRVus per year, the weights for males varied roughly 5-fold across the age spectrum from 1.9 (aged 20 years) to 9.6 (aged 0 years). For females, the variation was 4.3-fold, from 2.1 (aged 10 years) to 9.2 (aged 0 years).

Primary care physician panels vary markedly in work effort needed. Age/sex adjustment is sufficient to account for this for children, but not for adults.

Standardizing Primary Care Physician Panels: Is Age and Sex Good Enough?

 To assess the effect of a physician-specific pay-for-performance program on quality-of-care measures in a large group practice.

 In 2007, Palo Alto Medical Clinic, a multispecialty physician group practice, changed from group-focused to physician-specific pay-for-performance incentives. Primary care physicians received incentive payments based on their quarterly assessed performance.

 We examined 9 reported and incentivized clinical outcome and process measures. Five reported and nonincentivized measures were used for comparison purposes. The quality score of each physician for each measure was the main dependent variable and was calculated as follows: Quality Score = (Patients Meeting Target / Eligible Patients) × 100. Differences in scores between 2006 and 2007 were compared with differences in scores between 2005 and 2006. We also compared the performance of Palo Alto Medical Clinic with that of 2 other affiliated physician groups implementing group-level incentives.

 Eight of 9 reported and incentivized measures showed significant improvement in 2007 compared with 2006. Three measures showed an improvement trend significantly better than the previous year’s trend. A similar improvement trend was observed in 1 related measure that was reported but was nonincentivized. However, the improvement trend of Palo Alto Medical Clinic was not consistently different from that of the other 2 physician groups.

 Small financial incentives (maximum, $5000/year) based on individual physicians’ performance may have led to continued or enhanced improvement in well-established ambulatory care measures. Compared with other quality improvement programs having alternative foci for incentives (eg, increasing support for staff hours), the effect of physician-specific incentives was not evident.

	Various well-established quality-of-care indicators demonstrated continued improvement during a trial of physician-specific quality financial incentives.

		In the context of other organization-level quality improvement efforts, physician-specific incentives seem to have some incremental effect of improving quality of care.

		These changes occurred in a practice setting relying on fee-for-service payment to physicians and with many of the quality goals unassociated with additional tests or procedures.

		Alternative foci of incentives for quality improvement (eg, increasing staff hours to assist physicians’ quality improvement activities) need to be explored.

In recent years, a growing number of health plans and payers in the United States have adopted pay-for-performance (P4P) mechanisms to encourage improvement in quality of care. Despite its promise and growing popularity, empirical evidence on the effectiveness of P4P is inconsistent. Some studies

 have demonstrated the hoped-for positive effects, but others found no effect

 or unintended negative effects.13 These inconsistent findings may in part arise from heterogeneities in the scope of targeted quality dimensions and in the design and implementation of the program. Pay-for-performance programs are increasingly targeting multiple measures rather than focusing on 1 or 2 measures. Targeted measures are often process measures but sometimes include outcome measures.

 tend to demonstrate small effects of P4P, but even these are not uniform across all measures evaluated.

Most P4P programs in published studies were designed and implemented by a payer rather than by the physicians subject to the incentives. The buy-in to such externally determined specific performance measures and associated incentives by physicians may be limited. Payerdriven incentive schemes are typically designed to work across a wide range of sites and data systems. This “least common denominator” approach may not always seem clinically applicable to those physicians who believe they have better data. Eligibility restrictions on incentive payments (eg, only applicable to enrollees of certain insurers) may lead to focused attention on just those patients or to a failure to change practice because too few patients would be involved. Studies

 typically assess the effects of a modest group-level incentive compared with no incentives. Increasingly, studies

are assessing individual physicianlevel incentives again compared with no incentives. However, it is unknown whether the target of incentives (ie, group level vs physician specific) would make a difference. The present study explores this last question.

The primary objective of this study was to assess the effects of physician-specific P4P incentives versus group-level incentives on various quality measures. The P4P program examined in this study is similar to other P4P programs, but its specific implementation was physician led rather than payer driven. It was designed via a consensus process among representatives of the participating physicians with regard to the definitions of quality measures, the inclusion and exclusion rules among patients for each measure, and the incentive formulas. Although the measures implemented are similar to those required by and reported to payers, the physicians used criteria for inclusion and exclusion and for targets of success that they believed were more relevant to their clinical practice. For example, completion of colon cancer screening in their version could be confirmed by patient self-report or by documents verifying completion in a different institution rather than just through services provided by the group. For patients with diabetes mellitus, the physicians chose a more stringent glycosylated hemoglobin target of 7.0% instead of the externally set target of 8.0%. Most important, all patients of the primary care physicians (PCPs) (regardless of insurer) were considered for the performance evaluation so that work flow could be altered for all patients. The data used in the study came from electronic health records, which generally provide more accurate and precise data on clinical procedures and outcomes than billing data.

The study was conducted at Palo Alto Medical Foundation (PAMF), a not-for-profit healthcare organization. In 2007, PAMF contracted with the following 3 multispecialty physician groups: Palo Alto Medical Clinic (PAMC), Camino Medical Group, and Santa Cruz Medical Group. Although 25% to 30% of PAMF patients are enrolled in capitated programs, the physicians in the groups contracting with PAMF were paid based on relative value units of service regardless of the patient’s coverage. All 3 physician groups, located at clinics in adjacent counties, had a roughly similar mix of primary care and specialty physicians and served patients of similar demographic composition. The physician-specific P4P incentive program was implemented at PAMC in 2007. PAMC provides coverage at 5 clinics operating within 3 counties in the San Francisco Bay Area, California, serving approximately 13% of the general population in the underlying geographic area, with low patient turnover (3% per year).

Design of the Physician Incentive Program

In 2007, all PCPs (n = 179) at PAMC practicing family medicine, general internal medicine, or pediatrics participated in the physician incentive program. The bonus amount was based on individual physicians’ performance on 15 ambulatory quality measures, with a composite score calculated using an algorithm developed by the incentive program leadership.

 In brief, the physicians set targets for each measure. Physicians received

varying points for achieving minimal, average, and stretch goals based on the percentage of their patients achieving the target. The bonus was based on the percentage of potentially achievable points actually earned. The maximum achievable bonus was $5000/year, or about 2% of the PCP annual salary. The design and implementation of the program were discussed at the physician partnership meeting and at primary care department head meetings.

Since 2003, PAMF has been participating in the P4P program sponsored by several California health plans and operated by the Integrated Healthcare Association (IHA) (http://www.iha.org). PAMF retained a portion of the IHA P4P incentive for its organization-wide quality improvement (QI) efforts, and the remainder (roughly $4000 per physician) was distributed to each of 3 physician groups. The PAMF portion went to support the considerable central organizational services provided by PAMF for QI interventions on behalf of physicians.

Three physician groups independently decided on the allocation of the remainder of the funds. Until 2006, the allocations to the physician groups were not passed on to individual physicians. In 2007, PAMC decided to distribute a portion of the IHA payments to individual PCPs based on their performance scores on measures that were internally defined by the group and had been used at least since 2005. The other 2 physician groups continued using the IHA measures and definition of eligible patient populations in assessing grouplevel or department-level performance.

Although the physician-specific financial incentive was newly implemented in 2007, monitoring and quarterly reporting of quality indicators in the PAMC system had been in place since 2003. Physicians were alerted by e-mail with an electronic link to a detailed quality score workbook with their scores, peer physicians’ scores, and rank relative to other physicians in a distribution curve for each quality measure. In these reports, physician identities are disclosed to each other.

Before the implementation of the physician incentive program at PAMC, the program leadership convened several meetings to decide on the performance measures to use, details about each measure, and target levels of performance to incentivize and then selected 15 clinical measures representing clinical outcomes and processes. The present study focuses on 9 reported and incentivized clinical outcome and process measures that had been already evaluated and reported at least since 2005. The other 6 measures, which were specific to pediatric patients, were newly adopted for the 2007 program; the present study excludes these 6 measures without the prior year data needed for identification of the program effect. The 9 incentivized measures analyzed in this study were 3 outcome measures for patients with diabetes and 6 process measures (

Several of 9 measures were similar to those used in the IHA’s P4P program (http://www.iha.org), which was developed from evidence-based practice guidelines such as the 2007 Healthcare Effectiveness and Data Information Set indicators by the National Committee for Quality Assurance. The specific definitions of the measures used in the PAMC program were somewhat different, reflecting the group’s organizational goals, high standard of quality of care, and information technology capacity. For example, it set stricter thresholds for the diabetic control indicators than did the IHA (eg, 100 vs <130 mg/dL for low-density lipoprotein cholesterol [LDL-C] control; to convert cholesterol level to millimoles per liter, multiply by 0.0259). In contrast, patients whose completed screening tests could be verified through other providers were coded as completed, whereas the IHA required that the tests be performed by the physician group.

Our analysis uses 5 comparison and nonincentivized measures reflecting similar dimensions of care reported since 2005 or earlier but not included in the physician-specific incentive program. These include the following 5 measures: 4 outcome measures (1 for patients with hypertension and 3 for patients with diabetes [the same indicators as were incentivized but with less stringent targets]) and 1 process measure focused on a different target population (patients with hypertension) (Table 1).

The quality score of each physician was calculated for each measure with denominators of 6 or more eligible patients in a physician’s panel as follows: Quality Score = Numerator/Denominator) × 100, where the numerator is the number of patients who received the recommended care, and the denominator is the number of patients who were eligible for the recommended care (definitions for each measure are given in Table 1). The purpose of the physicians setting the minimum criterion for the denominator in the incentive design was to prevent the percentage score from being dominated by a few cases; we follow their approach. With the help of a health maintenance compliance reminder system embedded in the electronic health record and diabetes and asthma registry data, physicians could easily identify their eligible patients for each measure.

The unit of analysis in the primary analysis is the physician, eliminating concerns about a changing mix of providers over time. We first analyzed time trends in quality scores over 3 years (2005-2007) for each measure. We did not have access to patient-level data, so it was impossible to control for within-category comorbidities. However, the high stability of patient panels suggests that comorbidity patterns in a physician’s panel do not vary markedly over time and that the same patients are included for multiple years for certain measures.

To take into account within-physician correlations across quality measures, we estimated physician-level random-effects models. Statistical significance was set at 

 <.05. To assess the effects of temporal trends, we descriptively compared the trend in scores among the PAMC physicians relative to those of the other 2 physician groups that had not implemented a physician-specific incentive program. For this purpose, we used the mean performance scores for the following 3 measures in the IHA P4P program that were defined identically among the 3 physician groups for 3 years (2005-2007): asthma controller prescribing, cervical cancer screening, and Chlamydia screening. The IHA scores are not available at a physician-specific level and may reflect some changes in providers across years. Because the IHA data are aggregated, we cannot perform the same types of statistical tests as were performed for the PAMC.

All PCPs practicing at 5 PAMC locations were eligible for the program. Among 179 physicians, 167 were included in the study; 12 had insufficient qualifying patients for various reasons (eg, medical leave or too few patients). Physicians were more likely to be in family medicine (42%) or general internal medicine (34%) than in pediatrics (24%). Most physicians (152 in 2005 and 169 in 2006) also had data for the equivalent measures in the previous years; 148 had data for all 3 years.

Eight of 9 reported and incentivized measures showed significant improvement in 2007 compared with 2006. The rate of improvement was significantly better than the previous year’s trend (2006 compared with 2005) for the following 3 measures: blood pressure control for patients with diabetes, colon cancer screening, and documentation of tobacco use history (

). For LDL-C level control among patients with diabetes, the change in 2006-2007 reversed the improvement seen in 2005-2006. This reverse trend at the end of 2006 was apparently due to a change in the laboratory cholesterol analyzer, which produced systematically lower LDL-C values. Accelerating improvement was observed in 1 of 5 nonincentivized measures examined, namely, blood pressure control for patients with hypertension.

Trends in the 3 IHA P4P measures assessed consistently across the 3 physician groups did not clearly demonstrate any greater improvement in 2007 at the PAMC compared with 2006 or compared with trends in the other 2 physician groups. These results are shown in the  

With an individual physician-level P4P program, we observed improvement in some reported and incentivized quality measures during the intervention period and in comparison with the previous years’ trend. However, we observed no difference in consistently measured quality indicators between PAMC and the other 2 physician groups without physician-specific incentives. However, all 3 physician groups implemented various other QI activities, and it is impossible to rule out other extraneous but site-specific factors that may have influenced these results. Nonetheless, these findings suggest that physician-specific financial incentives, among other strategies, could be effective to improve quality of care.

Our study focused on a program designed and implemented with a high level of participating physicians’ engagement and is notable for the inclusion of all patients and for a more comprehensive scope of measures than most P4P programs. The quality reporting system allowed the physicians to easily track their eligible patients and peer physicians’ performance. Given this context, the physician group may have expected positive outcomes. However, because various QI programs had already been implemented and because organization-level incentives had been in place at the study site for many years, it may have been unrealistic to expect substantial additional improvement.

The relative similarity in trends between PAMC and the other 2 physician groups may be happenstance. The other physician groups, with no internally defined P4P program and with less comprehensive data, could only focus on those patients identified by the IHA collaborating health plans, roughly 30% of the total. To the extent that some QI efforts may have involved extra effort in outreach to patients or in patient education, this means that the same incremental resources could be focused on fewer patients, leading to the expectation of a greater effect outside of PAMC.

Our findings are consistent with some recently published P4P studies. Similar to our finding of accelerated improvement in 2007 for blood pressure control among patients with diabetes, Beaulieu and Horrigan

 reported positive glycosylated hemoglobin and LDL-C outcomes among this patient population. Other studies found similar results regarding improved cancer screening5 and found no improvement in asthma controller prescribing.

 examining group-level incentives echoes our finding of improved documentation of tobacco use history. Taken together, well-designed P4P incentive programs seem to have modest positive effects on improving targeted dimensions of quality of care.

The estimated improvement due to the physician-specific incentives in our study may be conservative for at least 2 technical reasons. First, for the percentage score in a particular quarter, the denominator was the number of patients ever seen by a PAMC physician for periods ranging from the previous 6 months to the past several years, depending on the measure. If only those patients who were newly qualified to be in the denominator were included, the effect would have been substantially larger. The lack of variation due to the denominator effect might also have lessened physicians’ response to the incentive program. Second, by design, physicians already had been receiving bonuses provided to the entire group based on similar performance measures. We might have found a larger effect of physician-specific incentives if the physician group had had no financial incentives at all before the provision of physician-specific incentives. Other studies

Although we found an improvement in quality among some measures with a small bonus, the same or better improvement might have been achieved through other uses of the funds (eg, increasing staff hours and investment in additional information technology to easily track target patients). Given limited physician office hours and the increasing number and complexity of clinical guidelines and recommendations, provision of extra resources to substitute or complement physicians’ time may work better than modest monetary incentives given solely to physicians. The lack of observable difference in improvement relative to the other physician groups adopting different QI strategies also suggests potential advantages of other forms of incentives. Quality improvement may well depend on a wide range of interventions, including data infrastructure, physician and staff engagement, changes in work flow, and incentives. Future studies should investigate the effectiveness of alternative approaches to achieve QI.

Several limitations of our study merit discussion. First, we did not have a contemporaneous comparison group at the same study site receiving only performance reporting or group-level incentives; such a study design was unacceptable by the participating physicians. Instead, we descriptively compared group-level aggregate performance scores with scores of other physician groups serving similar patient populations. This limits the inferences we can make about the direct effect of the program. Second, experiences from one medical group cannot be generalized to those of other groups with different sizes, locations, information technology capacities, and “culture.” PAMC had been already providing high-quality services with the help of advanced information technology (eg, the electronic health record database, diabetes registry, asthma registry, and reporting system of comprehensive quality measures); the implementation and the effects of a similar physician-specific incentive program in other groups with different capacity might be different from what is reported herein. Third, the incentive program was discontinued after 1 year due to the merger of the 3 physician groups and the complexity of developing a uniform compensation scheme. This decision was made long before any results of the incentive program became available. Therefore, we are unable to assess how the incentive effect would change over time.

In future studies, important variations in payment schemes such as varying the amount and risk (ie, bonus vs penalty) of payment should be examined. The types of programs that may work well in a highly integrated medical group may be different from those that are best for independently operating practices. Further investigation about what is the driving force of the improvement or lack of improvement across the various incentivized measures is also needed. Specific physician and group characteristics related to responsiveness to P4P, as well as within- physician correlation across measures, should be understood to better tailor a program to varying practice settings.

In conclusion, physician-specific financial incentives (albeit small) for higher-quality care applied to well-established ambulatory care measures and implemented in the context of other ongoing organization-level QI efforts were associated with incremental improvements in aspects of quality of care that were incentivized. The effects of broader-scope performance- based physician compensation schemes and other types of incentives for QI (eg, increasing staff hours to assist physicians’ QI activities) need to be explored and compared.

	Earlier versions of this study were presented at the Second Agency for Healthcare Research and Quality Annual Conference, Washington, DC, September 7-10, 2005, the Fourth National Pay for Performance Summit, San Francisco, CA, March 9-11, 2009, the 137th Annual Meeting of the American Public Health Association, 2008 Annual Meeting and Exposition, San Diego, CA, October 25-29, 2008, and the 26th AcademyHealth Annual Research Meeting, Chicago, IL, June 28-30, 2009. We thank the participants of the meetings for their comments.

This study examined the effect of physician-specific pay-for-performance incentives on well-established ambulatory quality measures in a large group practice setting.

Sukyung Chung, PhD

Articles

Primary Care Physician Practice Styles and Quality, Cost, and Productivity

Standardizing Primary Care Physician Panels: Is Age and Sex Good Enough?

Effect of Physician-Specific Pay-for-Performance Incentives in a Large Group Practice