Diana S.M. Buist, PhD, MPH

 To measure changes in primary care physician (PCP) ordering rates for 4 global resource use measures before and after dissemination of physician feedback reports that provided peer-comparison resource use rates. We also explored whether physician practice characteristics (panel size, clinic size, and years of experience) were associated with resource use changes.

 Pre-post implementation study measuring physician resource use in an integrated healthcare system (2011-2014).

 Kaiser Permanente Washington PCPs (N = 210) were provided annual feedback reports showing their personal ordering rates compared with those of their peers. Monthly physician ordering was measured from November 2011 to September 2014 (including prereport and postreport periods). We examined 4 physician ordering rates (specialty referrals, high-end imaging, laboratory tests, and 30-day prescriptions) per 1000 patients, adjusted for patient age, gender, and clinical complexity.

 After accounting for physician practice characteristics, monthly PCP ordering rates for high-end imaging significantly decreased by 0.8 images per 1000 patients (

 <.01). In contrast, orders for laboratory tests and 30-day prescriptions significantly increased by 15.0 tests and 84.7 prescriptions per 1000 patients (both 

 <.01). We observed greater changes following feedback in physicians with fewer years of experience (≤10 years), who had 4.2 fewer specialty referrals (

 = .01) and 101.3 more 30-day prescriptions (

 <.01) compared with those with more experience (>20 years).

 Physician feedback reports may be associated with changes in physician resource use, and physicians with fewer years of experience may be more responsive to feedback reports. Better understanding of factors associated with changes in resource use is necessary for future targeted development of physician interventions.

Healthcare systems use feedback reports to document physician practice patterns and increase physician accountability for resource use, but there are few reports of whether personalized resource use feedback affects provider performance. We evaluated the association between distribution of feedback reports to primary care physicians in an integrated delivery system and changes in 4 global measures of physician resource use.

Feedback reports were associated with varied changes in resource use.

Physicians with fewer years of experience may be more responsive to feedback reports.

Better understanding of factors associated with changes in resource use is necessary for future targeted development of physician interventions.

Fueled by concerns of rising healthcare costs and the national dialogue on overuse and misuse, healthcare systems are increasingly interested in developing better tools for measuring and reporting healthcare quality and efficiency.

 Reducing variability in physician practice patterns is an opportunity for healthcare systems to improve quality and reduce costs.

 Healthcare organizations are addressing variability in care with increased internal transparency in organizations to create a culture of high-value care.

 Numerous organizations, from Medicare and Medicaid to small practices, use audit and feedback reports to support physician behavior change and performance improvement.

 Although feedback reports have been shown to effectively increase accountability,

 the literature on implementing feedback reports and their impact on physician practice patterns in healthcare systems is inconclusive.

In 2012, Kaiser Permanente Washington (KPWA; formerly Group Health Cooperative) implemented a Resource Stewardship quality improvement (QI) initiative to reduce low-value care, or care that does not improve patient outcomes and can harm patients. The initiative focused on helping KPWA physicians become better stewards of healthcare resources. Recognizing that daily decisions drive quality and cost, an internally transparent peer-comparison feedback report was developed to bring personalized information to physicians that informed their clinical decision making and facilitated internal conversations about medical resource use.

The annual resource stewardship report showed individual physician-, clinic-, and system-level ordering rates for specialty referrals, high-end imaging, laboratory tests, and 30-day prescriptions. It was designed to show variations in global service use, because improving a few specific areas was deemed unlikely to be sufficient for reducing low-value care at the healthcare-system level. Feedback was combined across multiple services to help identify areas of greater overall resource use rather than targeting single areas of utilization that may result in unintended shifts in resource utilization (eg, reducing specialty referrals may cause increases in high-end imaging).

We evaluated whether distribution of internally transparent peer-comparison feedback reports within an integrated delivery system was associated with changes in individual physician resource use. We also explored whether physician practice characteristics (panel size, clinic size, and years of experience) were associated with changes in resource use. Although evidence for the link between practice characteristics and resource use is mixed,

 better understanding of these characteristics could be used to target feedback reports to those who would benefit most.

KPWA is a mixed-model delivery system providing insurance and healthcare to approximately 710,000 patients in Washington state. Approximately 370,000 KPWA patients are cared for within the integrated delivery system (group practice); the remainder receive care through a network of contracted physicians and other healthcare providers across the state. Within the group practice, approximately 1000 salaried multispecialty clinicians practice in 25 medical clinics; approximately 300 are primary care physicians (PCPs). We included KPWA PCPs specializing in family medicine or internal medicine with a panel size of 250 or more patients during each month of the study period. This QI initiative and subsequent analyses were determined to be exempt from human subjects review in compliance with the Office for Human Research Protections (45 CFR Part 46). We followed healthcare QI reporting guidelines.

We extracted data on physician orders and characteristics from KPWA’s automated clinical and administrative data systems. We used physician orders to accurately attribute and assess physicians’ intended resource use. We created monthly measures of physician ordering from November 2011 to September 2014, including the 11 months before distribution of the reports (prereport period, November 2011-September 2012) and the 11 months after distribution (postreport period, November 2013-September 2014). We included a 13-month implementation period (October 2012-October 2013) during which 2 reports were distributed to each PCP.

Resource stewardship reports were designed to identify practice pattern variations and make resource use visible to KPWA’s PCPs. Physicians were given clinic-specific reports that included ordering rates and physician names for all PCPs in their clinic. Bar graphs of annual, physician-level, comorbidity-adjusted ordering rates in the prior calendar year for specialty referrals, high-end imaging, laboratory tests, and 30-day prescriptions per 1000 patients were displayed for all PCPs within the clinic (see 

for a sample 2015 report; some reported ordering rates changed since the 2013 report). The graphs included lines to indicate the system or clinic average and 1 SD above and below the average. Bar graphs in the second-year report (September 2013) also showed physicians’ ordering rates in the prior year to allow physicians to see personal changes over time. Reports included physician names to promote within-system transparency and give physicians the opportunity to learn from the practice patterns of other physicians within their clinic.

Feedback reports were disseminated in November 2012 and September 2013 through monthly within-clinic physician meetings and individual meetings between physicians and their local clinic chiefs. The Medical Director for Quality (M.H.) and clinic chiefs explained the Resource Stewardship initiative and the report’s purpose, presented the data, and gave physicians opportunities to discuss intraclinic variability as a group. Clinic chiefs were trained to present the reports as a chance for physicians to learn from each other rather than as a punitive tool.

We constructed 4 physician ordering measures reported in the feedback reports: specialty referrals, high-end imaging, laboratory tests, and 30-day prescriptions. Specialty referrals included all referral orders made by the physician. High-end imaging orders included computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography. Laboratory tests and prescriptions included all laboratory tests and prescriptions ordered; prescriptions were standardized to a 30-day supply. Rates were calculated as per 1000 paneled patients. Nearly all KPWA patients within the group practice select a PCP and are empaneled to the physician’s care team; care teams are accountable for the care coordination of all empaneled patients.

 Given the focus on physician stewardship and accountability, patients who were not paneled to a PCP were excluded. We adjusted ordering rates by the panel’s average Adjusted Clinical Group (ACG) score to account for diagnosis mix and comorbidity burden. The ACG system is a widely used patient case-mix—adjustment system for populations receiving outpatient services.

For each physician, we extracted 3 physician practice characteristics hypothesized to be associated with physician resource utilization: panel size, clinic size (number of physicians in the clinic), and years of experience (years since graduation from medical school).

 Median panel size was calculated using the postreport period and adjusted for full-time equivalency. We also collected physician characteristics (gender, race [white, nonwhite]) for control variables.

We calculated descriptive statistics for physicians and assessed variability across physician orders using the coefficient of variation (CV; SD divided by mean). All models to test differences in orders were performed at the physician-month level with a repeated measures random effects model, using an identity link and Gaussian error term with robust standard errors. Dummy variables for each calendar month were included to control for variations in average utilization between months. We controlled for correlation due to repeated measures within physicians and clustered physicians within clinics. The model specifications allowed us to estimate the average resource utilization rate using marginal standardization over both the prereport and postreport time periods. We tested for differences in average resource utilization rates between the prereport and postreport time periods. We then separately tested interactions between each of the physician practice characteristics with the time periods to identify physician practice characteristics associated with changes in resource use. All models controlled for physician gender and race; age was highly correlated with years of experience and thus was not included. We did not control for multiple comparisons; instead, we chose to present results from all analyses that were carried out. All analyses were conducted with Stata 13.0 statistical software (StataCorp; College Station, Texas). All hypothesis tests were performed using 2-sided α = .05.

Our study included 210 PCPs (202 family physicians and 8 general internists) (

). Physicians were balanced on gender (54% female) and were predominantly white (72%) (

). Forty-four percent had panels with 1400 to 1700 patients, 46% had practiced for more than 20 years, and 51% worked in a large multispecialty clinic with more than 20 other physicians. Most worked less than full time; mean physician full-time equivalency was 0.73 (SD = 0.18; data not shown).

Medical resource use over time is illustrated in the 

. Over the entire study period, the CV of case-mix—adjusted mean monthly rates was highest for imaging (CV, 65%; mean = 5.0 images per 1000 patients; range, 4.2-6.1) and lowest for 30-day prescriptions (CV, 19%; mean = 1049.4 prescriptions per 1000 patients; range, 942.0-1128.4). Specialty referrals and laboratory tests had similar CVs of 38% (mean = 47.5 referrals per 1000 patients; range, 38.7-54.2) and 35% (mean = 289.1 tests per 1000 patients; range, 241.3-423.5), respectively. Prereport and postreport period CVs were consistent with the CV over the entire study period and with each other (

Regression results for case-mix—adjusted orders of specialty referrals, high-end imaging, laboratory tests, and 30-day prescriptions account for physician characteristics, physician practice characteristics, and calendar month (Table 2). Compared with the prereport period, monthly physician orders for high-end imaging decreased by 0.8 images per 1000 patients, which corresponds to a –14.5% (95% CI, –20.0% to –9.1%) change. Orders of laboratory tests and 30-day prescriptions increased by 15.0 tests (5.2% change; 95% CI, 2.6%-7.8%) and 84.7 prescriptions (8.4% change; 95% CI, 5.7%-11.1%), respectively, per 1000 patients. No statistically significant change in specialty referral orders was observed.

Physician Practice Characteristics Associated With Change in Physician Orders 

Although specialty referral orders showed no overall change, stratification by years of experience revealed statistically significant changes among physicians; physicians with 10 or fewer years of experience were associated with a relatively greater decrease in referral orders (—4.2 referrals per 1000 patients; –7.7% change; 95% CI, –13.4% to –1.8%) compared with physicians with more than 20 years of experience (

). Having fewer years of experience was associated with a relatively greater increase in prescription orders (86.0 orders for physicians with 11-20 years [8.8% change; 95% CI, 2.6%-15.0%] and 101.3 for physicians with ≤10 years [11.7% change; 95% CI, 4.2%-19.2%] compared with physicians with >20 years). Practicing in a clinic with 11 to 20 physicians was also associated with a relatively greater increase in prescription orders (46.7 orders; 2.6% change; 95% CI, 0.1%-9.1%) compared with clinics with more than 20 physicians. No practice characteristics were associated with statistically significant changes in imaging or laboratory tests.

Distribution of internally transparent peer-comparison resource use feedback reports at KPWA was associated with changes in physician resource use, specifically reductions in high-end imaging orders and increases in laboratory test and 30-day prescription orders. Years of experience were associated with changes in practice patterns, with greater changes in some resource use among less experienced physicians.

This study is an important example of using physician ordering data to evaluate changes over time following physician interventions. Compared with the claims and survey data normally used to assess utilization, physician order data may more accurately reflect intended use rather than realized use (as measured in claims) or perceived use (as measured in surveys). Claims data may be confounded by patient behaviors, whereas survey data may be confounded by response biases. We previously reported physicians’ recollection and experience receiving resource stewardship reports.

 Although we could not directly measure who received reports and associated discussions, nearly four-fifths recalled receiving the report from their clinic chief. Three-quarters of these respondents participated in a discussion about their individual and group reports, and the majority found them useful for understanding their own practice patterns and provoking discussion among their peers. Notably, although more than three-fifths of all respondents reported changing their practice in each of the 4 measures, we observed important changes in 3 of the 4 measures in this study. The ability to examine physician orders through electronic health record systems allows healthcare organizations to measure their physicians’ intended resource use.

Based on current best practices for effective feedback reporting systems and our survey findings,

 we believe that clearer specific performance targets and more timely feedback with greater frequency could improve the effectiveness of our feedback reports. Like previous interventions with physician feedback reports and resource use,

 the direction of the change in resource use after receipt of the report varied by the specific measure. How feedback was presented to physicians may help explain the inconsistent direction of change in orders. Although KPWA’s and clinics’ average rates and peer physician rates were intended to be viewed as comparators, the feedback reports did not include explicit target goals or benchmarks to guide how physicians should change specific behaviors. Verstappen et al found that feedback reports reduced the use of some low-value services among PCPs.

 However, given the challenges of using utilization data to identify appropriate and inappropriate care and systemwide interest in addressing variation in global service use,

 these reports only provided overall resource use. It is likely that higher resource users have greater appropriate and inappropriate use. Physicians also could have been influenced to reduce resource utilization by other sources. For example, KPWA’s promotion of safe reduction of CT and MRI scans since 2009 and external campaigns to reduce imaging overuse may have reinforced the message to reduce overall utilization of high-end imaging.

 Similarly, KPWA physicians were sent weekly “clinical pearls” emails that occasionally highlighted low-value care themes that targeted specific conditions and therapies. Conversely, low resource users may have increased their ordering patterns after seeing how they compared with their peers. Use of global ordering rates limited our ability to ascertain how feedback reports impacted inappropriate care compared with appropriate care, and further research is needed to decompose these effects.

The frequency and timeliness of feedback data have been found to be associated with more effective feedback reports.

 Our feedback reports used annual data from the previous calendar year; the lag in the data (11 and 9 months for the first and second reports, respectively) may have reduced the reports’ effectiveness because physicians were unable to link their current performance with the data reported. Despite known best practices suggesting greater frequency and timeliness of producing feedback reports,

 we were restricted by the lack of dedicated resources. Reporting intervals likely need to be shorter because more than 20% of surveyed KPWA physicians did not remember having the report shared with them.

 Current recommendations suggest providing feedback in shorter intervals (eg, monthly) so that physicians have sufficient time to change their behavior.

Our exploratory examination of physician practice characteristics and changes in resource use found significantly greater changes in utilization among physicians with fewer years in practice. Having fewer years of experience has been associated with greater resource use (eg, higher cost, greater prescribing and imaging), which is hypothesized to be due to greater exposure to newer and costlier care, lack of experience, or lack of established practice patterns.

 Similarly, physicians with less experience may be more flexible in their practice patterns and may have changed their practice patterns to be more comparable with patterns observed in more experienced physicians. This finding, coupled with a high percentage of physicians reporting conversations about their practice patterns,

 may suggest that internally transparent reports may be prompting physicians with less experience to learn from their colleagues with more experience. These conversations among physicians may be a key element of creating a culture and environment of continuous learning that promotes high-value care.

 Conversely, feedback reports appeared to be less effective in modifying ordering rates among physicians with more experience. More active and multifaceted interventions, such as outreach programs and workshops combined with feedback reports, may be needed to change their ordering practices.

We evaluated healthcare resource utilization patterns following a systemwide feedback initiative designed to bring better information to physicians by informing their clinical decision making on the global use of resources. Most importantly, we did not have a control group to evaluate whether observed changes were reflective of historical trends or important unmeasured confounders. For example, high-end imaging rates have been slowing down nationally during this time.

 KPWA physicians are salaried in an integrated delivery system, which may limit the generalizability of our findings because working in a capitated payment system may influence physician practice patterns.

Notably, KPWA had concurrent components of this initiative and other QI interventions that may have been potential confounders for the ordering measures. The Resource Stewardship initiative included 2 additional components to promote reducing low-value care. The first component was an organization-wide effort to promote shared decision making and the second focused on evidence-based clinical improvement, in the spirit of the Choosing Wisely campaign.

 In the first, physicians were offered training in multiple shared decision-making skills, including discussing low-value care with patients. In the second, as noted previously, KPWA sent clinical emails that highlighted Choosing Wisely themes and implemented a prostate-specific antigen (PSA)/Pap smear trigger tool that electronically identified inappropriate Pap tests in women

and PSA tests in elderly men from June 2013 to May 2014. During the study period, 3 emails (1 each for Pap tests and PSA tests and 1 introducing Choosing Wisely lists) targeted PCPs, whereas other Choosing Wisely—themed emails targeted specialists. Given the focus on reducing Pap tests and PSA tests, the third component of the initiative had the greatest likelihood of impacting changes in laboratory tests. This suggests that our finding of an increase in laboratory tests may be conservative. Additionally, high-end imaging had the most robust concurrent QI initiative (described previously), and additional data from the feedback reports may have accelerated ongoing improvements. Other concurrent initiatives (numerous pharmacy initiatives to promote use of preferred agents and KPWA’s primary care medical home transformation) were less likely to change global utilization rates.

 Despite these limitations, we used a robust pre-post design to assess the impact of implementing feedback reports in a large primary care setting.

Implementation of systemwide dissemination of feedback reports across multiple medical resources was associated with modest changes in ordering behavior among PCPs. Changing physician ordering behavior through greater transparency of physician behavior may help healthcare systems move toward creating a culture of high-value care.

RTI International (EC), Waltham, MA; Kaiser Permanente Washington Research Institute (DSMB, EJ, SF, RP, GG), Seattle, WA; Washington Permanente Medical Group (MH), Seattle, WA; Trillium Health Partners, Institute for Better Health (RJR), Mississauga, ON, Canada.

 This work was funded by a Partnership for Innovation grant from the Group Health Foundation.

 The findings in this article were presented at the 2015 HMO Research Network Conference, Long Beach, CA, and the 2015 AcademyHealth Annual Research Meeting, Boston, MA.

 Dr Buist has received Robert Wood Johnson grant funding on reducing low-value care and has attended the V-BID Health Task Force on Low-Value Care. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (EC, DSMB, MH, RP, RJR); acquisition of data (DSMB, SF, RP); analysis and interpretation of data (EC, DSMB, EJ, SF, RP, RJR); drafting of the manuscript (EC, DSMB, MH, EJ); critical revision of the manuscript for important intellectual content (EC, DSMB, MH, GG, RJR); statistical analysis (EC, DSMB, EJ); provision of patients or study materials (DSMB, GG); obtaining funding (DSMB, MH); administrative, technical, or logistic support (DSMB, SF, GG); and supervision (DSMB, MH).

 Eva Chang, PhD, MPH, RTI International, 307 Waverly Oaks Rd, Ste 101, Waltham, MA 02452. Email: echang@rti.org.

1. Lee VS, Kawamoto K, Hess R, et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. 

. 2016;316(10):1061-1072. doi: 10.1001/jama.2016.12226.

Articles

Implementing systemwide dissemination of feedback reports to primary care physicians in an integrated delivery system may be associated with changes in medical resource use.

Primary Care Physician Resource Use Changes Associated With Feedback Reports

To evaluate woman-level characteristics associated with timing of follow-up after abnormal mammograms in an integrated healthcare system with an active breast health program.

 The study included women aged 40-84 years who had an abnormal mammogram (20,060 screening and 3184 diagnostic) recommended for follow-up. We compared characteristics of women who received any follow-up evaluation within <7, 8 to 14, 15 to 21, and 22 to 180 days. We estimated odds ratios (ORs) and 95% confidence intervals (CIs) using multivariate ordinal logistic regression.

 The proportion of women seeking care within 7 days was 23% for screening and 69% for diagnostic mammograms. Characteristics associated with later follow-up (>8 days vs <7 days) after an abnormal screening mammogram included being older (OR = 1.15; 95% CI, 1.04-1.26 [age 70-79 years]; OR = 1.31; 95% CI, 1.14-1.51 [age 80 years]), Asian (OR = 1.18; 95% CI, 1.04-1.33), or having a college degree (OR = 1.10; 95% CI, 1.01-1.19). Characteristics associated with earlier follow-up included family history of breast cancer (OR = 0.93; 95% CI, 0.88-0.98), symptoms at time of mammogram (OR = 0.79; 95% CI, 0.70-0.88), or extremely dense breasts (OR = 0.82; 95% CI, 0.69-0.96). For diagnostic mammograms, symptoms at time of mammogram (OR = 0.47; 95% CI, 0.39-0.56) and being obese (OR = 0.79; 95% CI, 0.65-0.98) were associated with earlier follow-up.

 Several woman-level characteristics were associated with timely follow-up after an abnormal screening exam, but only presence of symptoms and being obese was associated with timely follow-up after an abnormal diagnostic exam.

	Within an integrated healthcare system with an active breast health program to facilitate care after an abnormal mammogram, there remains woman-level characteristics associated with timely follow-up.

		Presence of symptoms was associated with timely follow-up after both abnormal screening and diagnostic mammogram.

		Women who were older, Asian, or more educated were less likely to return earlier after an abnormal screening mammogram. Efforts to understand and reduce barriers might be important in these groups of women.

	Approximately 75% of US women report having at least 1 screening mammogram to detect breast cancer.

 Abnormal mammograms, which require additional imaging or surgical follow-up, occur at a rate of almost 10% and 8% of screening and diagnostic exams, respectively.

 There are currently no published national data available for mammography facilities and practices benchmarks for the appropriate timing for receipt of follow-up evaluation after an abnormal mammogram.

	A delay in follow-up can be attributed to characteristics of the woman (ie, not making an appointment), the provider (ie, not communicating the importance of follow-up), or the healthcare system (ie, insufficient scheduling space for prompt return). Failure of 1 or all of these systems results in delayed follow-up by the patient, which may contribute to a delayed diagnosis of breast cancer, later stage disease,

 Woman-level characteristics associated with a delay or lack of follow-up after an abnormal mammogram include being older,

	Evaluating the follow-up of breast health within an HMO setting provides opportunities to monitor women after an abnormal mammographic exam when access to appropriate follow-up care should be equitable. Two prior studies have evaluated follow-up after an abnormal mammogram within an HMO setting,

 but neither study had an active program for breast health, which would identify and schedule care for women with abnormal mammograms. In our study, we investigated woman-level characteristics associated with  follow-up after an abnormal mammography examination in women enrolled in a health plan with an active breast health program.

	Women aged 40 to 84 years who received a mammogram between 1996 and 2003 in Group Health, an integrated delivery system in the Puget Sound region of Washington State, and were enrolled for at least 12 months were eligible for this study. All women had access to free mammography through the health plan, including screening and diagnostic exams. This analysis is restricted to women with an abnormal result from mammography defined as a Breast Imaging Reporting and Data System (BI-RADS)

 assessment of 0 (incomplete), 4 (suspicious abnormality), or 5 (highly suspicious of malignancy). We specifically excluded women recommended for 6-month short-interval follow-up (BI-RADS 3) because we were interested in abnormal exams that required immediate attention. Further, we restricted the sample to women who were recommended to receive additional follow-up with 1 of 4 procedures: (1) biopsy, (2) fine needle aspiration, (3) surgical evaluation, or (4) ultrasound. Women had to be continuously enrolled for 12 months after the abnormal mammogram to ensure adequate information on follow-up. We identified 25,272 and 5074 abnormal screening or diagnostic mammograms, respectively.

 Breast Cancer Screening Program at Group Health

	Since 1985, the Breast Cancer Screening Program at Group Health has sent reminder letters to women to encourage participation in mammography screening.

 To date, the program has mailed more than 500,000 letters. Breast clinic nurses were responsible for much of the follow-up after a mammogram. For 

, the radiologist interpreted the image the next working day after the mammogram. The breast clinic nurses called the patient within 24 to 48 hours after the report was issued with the patient’s result. If a patient had an abnormal result, the breast clinic nurse attempted to schedule the next available appointment for the patient. For 

, the radiologist interpreted the image the same day and reported the result of the mammogram to the woman. A patient with an abnormal diagnostic mammogram would have scheduled the next available follow-up appointment recommended by the radiologist with the breast clinic nurse. Among nonresponders for either screening or diagnostic mammograms, the breast clinic nurse would call the patient to schedule the follow-up appointment.

	At the time of the mammography visit, women completed a self-administered questionnaire,

 which collected information on age, reproductive risk factors, breast screening, medical history, and concerns regarding women’s breasts at the time of the exam. Women completed the questionnaire before knowing the results of their mammogram. The  uestionnaire served as the data source for woman-level characteristics under evaluation.

	Information about follow-up procedures was available through electronic administrative data including pathology, radiology, utilization, and claims databases. Time to follow-up care was calculated from the date of the mammogram until the first date of follow-up care with any of the 4 procedures. Women might have received a procedure that was different from the one recommended, but was still 1 of the 4 procedures of interest. We categorized women according to receipt of follow-up care within <7, 8 to 14, 15 to 21, and 22 to 180 calendar days, based on ability to seek follow-up care within sequential weeks up to 4 weeks.

	This study was approved by the institutional review board at Group Health Research Institute.

	Woman-level characteristics of interest for this analysis included the following risk factors: age, race, body mass index (kg/m2), education, family history of breast cancer in first- or second-degree relatives, self-reported symptoms at the time of mammography, biopsy history prior to the mammography examination, radiologists’ interpretation of mammographic breast density,

 and use of postmenopausal hormone therapy. Women with missing data for any of the risk factors or without follow-up within 180 days were excluded from the  analysis. The final sample size included 20,060 screening and 3184 diagnostic mammograms. Descriptive summary statistics were calculated for each variable by interval of follow-up time.

	We used a multivariate ordinal logistic regression model (ologit) for ordered outcome variables to compare differences in the follow-up timing intervals.

An ordinal logistic model is a model for multinomial data that generalizes the logistic regression model. The model allows the consideration of differences in “early” versus “later” follow-up at each of the 3 thresholds of the outcome. The interpretation of the odds ratio is the comparison of early versus late defined in our analysis as <7 days versus >8 days. All analyses included all potential risk factors and used robust methods to account for clustering for multiple records per woman. Tests for trends were evaluated by entering the ordinal term representing  the categorical variable into the ordinal model. We evaluated the contribution of each variable using a likelihood ratio test, comparing the full model with the reduced model. All hypothesis tests were 2-sided and assumed a type I error rate of alpha = .05.

	All analyses were conducted in Stata/MP 11.0 (Stata-Corp, College Station, TX).

	For women who had an abnormal screening mammogram, the median time from receipt of the exam to followup was 13 days. The proportion of women receiving care within <7, 8 to 14, 15 to 21, and 22 to 180 days was 23.3%, 36.3%, 22.0%, and 18.3%, respectively, among women with abnormal screening mammogram (

). For women who had an abnormal diagnostic mammogram, the median time to further follow-up was 5 days. Further, the proportion of women receiving care within 7, 8 to 14, 15 to 21, and >22 days was 69.1%, 10.6%, 7.0%, and 13.2%, respectively (

	Among women with an abnormal screening mammogram, those more likely to receive follow-up care after 7 days were older (70-79 and >80 years) (P-trend <.001), Asian or Pacific Islander, and college graduates (P-trend = .006). The woman-level factors associated with being more likely to receive follow-up care within 7 days included reporting symptoms at the time of the exam, having a positive family history of breast cancer, and having extremely dense breasts (P-trend = .001) (Table 1).

	Among women with an abnormal diagnostic mammogram, few woman-level characteristics were associated with earlier or later follow-up. Older women had later follow-up than younger women, though the results were not statistically significant at all levels. Women with higher body mass index P-trend = .026) and symptoms at the time of the mammogram were more likely to have follow-up care earlier rather than later.

	This is the largest study to date to investigate the timing of follow-up after an abnormal mammogram. Unique features of this analysis include the available data from an integrated healthcare system and active surveillance by nurses to encourage and assist patients in returning to evaluate the abnormality noted on the mammogram. Further, we had sufficient data to investigate follow-up for both screening and diagnostic mammograms.

	We confirmed that women of older age were more likely to seek care later rather than earlier for both screening and diagnostic mammograms.

 We also detected that having symptoms at the time of the mammogram was associated with earlier follow-up for both screening and diagnostic exams. Our finding is supported by Caplan and colleagues, who determined that women with breast symptoms were diagnosed with breast cancer 6 days earlier than women without breast symptoms.

	In this study, we found that women of Asian/Pacific Islander descent were more likely than white women to delay follow-up care after an abnormal screening mammogram. Karliner and colleagues reported that Asian women did not fully understand their physicians’ reports of their mammograms, particularly Chinese-speaking women.

 It is unclear whether language barriers were responsible for the longer delay in seeking care among Asian/Pacific Islander women in our study, but language interpreters or written materials might help reduce delay in other programs.

	Women with extremely dense breasts were more likely to seek follow-up care quickly compared with women with less dense breasts. Prior studies have not reported on breast density and delay in follow-up care. The sensitivity of mammography in women with extremely dense breasts is only 30% compared with 80% in women with fatty breasts.

 These women are also likely to have an increased risk of interval cancers,

 which likely results in their prompt follow-up.

	In our study, fewer than 1% and 4% of women with abnormal screening and diagnostic mammograms, respectively, did not receive any follow-up within 180 days (data not shown). Other studies have reported lack of follow-up in women after abnormal exams at rates between 8% and 28%.

 The difference in rates of follow-up is likely due to the ability of women to receive additional medical services within our health system at no financial burden and with the assistance of specially trained nurses working in the breast program. The use of computer monitoring to follow up with women postmammography reduces the delays in seeking care.

 Group Health’s Breast Cancer Screening Program implemented active follow-up of women with breast clinic nurses and now uses the electronic medical record to help track women.

	This study was conducted in a well-characterized defined population of women who receive healthcare through an integrated delivery system. There was thorough assessment and follow-up of mammography screening because of the Breast Cancer Screening Program at Group Health. We might have missed some clinical care that occurred at outside facilities that was not submitted for a claim. However, this is unlikely for women with their primary insurance at the health plan, because filing a claim would enable payment for outside services and women would be motivated to do so.

	Clinical guidelines for promptness in obtaining an evaluation after an abnormality is noted on mammography examination are not well established. An assessment of the National Initiative on Cancer Care Quality in the United States suggests that 90% of patients should be seen within 3 weeks of an abnormal screening mammogram for a diagnostic mammogram, within 14 days from needing an open biopsy to cancer diagnosis, and within 5 working days from breast cancer diagnosis to an appointment with the treating physician.

 While we cannot make a direct comparison to the National Initiative for Cancer Care Quality, we can report that 82% of women received the recommended follow-up care within 3 weeks of their abnormal screening mammogram. Further, 69% of women with abnormal diagnostic mammograms received recommended follow-up care within 7 days. In 2004, the National Quality Measures for Breast Centers identified 2 measures of timely follow-up: imaging timeliness of care (ie, time between screening mammogram, diagnostic mammogram, needle biopsy, surgery) and surgical timeliness (ie, time between diagnostic mammogram, open surgical biopsy/excision, initial breast cancer surgery).

 There are characteristics that influence women’s timeliness to follow-up that future interventions may address to reduce delays in breast cancer diagnosis.

 From Group Health Research Institute (KJW, EJAB, SH, DSMB), Seattle, WA; and Department of General Internal Medicine (JGE), University of Washington, Seattle, WA.

 This research was supported by the National Cancer Institute (Dr Buist, Principal Investigator, U01CA63731; and Dr Elmore, Principal Investigator, K05CA104699).

 The authors (KJW, EJAB, SH, JGE, DSMB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Woman-level characteristics such as breast symptoms at mammography are associated with earlier follow-up within an integrated health system with an active breast health plan.

Diana S.M. Buist, PhD, MPH

Articles

Primary Care Physician Resource Use Changes Associated With Feedback Reports

Timing of Follow-up After Abnormal Screening and Diagnostic Mammograms