Variation in Outpatient Antibiotic Prescribing in the United States

December 7, 2009
Michael A. Steinman, MD
Michael A. Steinman, MD

,
Katherine Y. Yang, PharmD, MPH
Katherine Y. Yang, PharmD, MPH

,
Sepheen C. Byron, MHS
Sepheen C. Byron, MHS

,
Judith H. Maselli, MSPH
Judith H. Maselli, MSPH

,
Ralph Gonzales, MD, MSPH
Ralph Gonzales, MD, MSPH

Volume 15, Issue 12

Rates of outpatient antibiotic prescribing vary widely between US commercial health plans. High-utilizing health plans may improve quality and lower costs by reducing unnecessary antibiotic use.

Objective

: To evaluate variation in outpatient antibiotic utilization among US commercial health plans and the implications of this variation for cost and quality.

Study Design and Methods

: We measured antibiotic utilization rates among 229 US commercial health plans that participated in the 2005 Healthcare Effectiveness Data and Information Set. Rates were adjusted to account for health plan age and sex distribution. To estimate antibiotic costs, we multiplied utilization data for each drug class by national estimates of intraclass distribution of drugs, duration of therapy, and median average wholesale price.

Results

: Antibiotic utilization rates varied markedly among plans, ranging from 0.64 antibiotic fills per member per year (PMPY) at the 5th percentile of plans to 1.08 fills PMPY at the 95th percentile, with a mean of 0.88 (SD ± 0.15) antibiotic fills PMPY. US census region was the strongest predictor of antibiotic utilization. Antibiotic costs averaged $49 PMPY and ranged from $34 to $63 PMPY among plans at the 5th and 95th percentiles of cost, respectively. If a health plan with 250,000 members at the 90th percentile of antibiotic costs reduced its costs to the 25th percentile, annual drug cost savings would be approximately $4.1 million.

Conclusions

: Antibiotic utilization varies substantially among commercial health plans and is not accounted for by differences in the age and sex distribution of plan members. Because reducing rates of antibiotic utilization is likely to lower costs and improve quality, high-utilizing plans may reap considerable rewards from investing in

programs to reduce the overuse of antibiotics.

(Am J Manag Care. 2009;15(12):861-868)

Outpatient antibiotic utilization varies substantially among commercial health plans in ways not explained by patient case mix. As a result:

  • Antibiotic utilization is likely to be a valuable marker of prescribing quality for health plans.
  • Health plans with high rates of antibiotic utilization may benefit from targeted quality improvement programs to reduce unnecessary antibiotic use.
  • Cost savings from reducing unnecessary antibiotic use are substantial.

Antibiotics are widely overused in ambulatory practice, particularly in the management of acute respiratory tract infections. 1,2 Although bacterial infections cause a small minority of these illnesses, it is estimated that 40% to 50% of all patients in the United States who seek medical attention because of these conditions receive antibiotics.3 The consequences of this overuse are striking. Every year, millions of people are directly exposed to the side effects of antibiotics, ranging from common, bothersome symptoms to infrequent but devastating complications such as Clostridium difficile colitis and anaphylaxis.4 Moreover, research suggests that community levels of bacterial resistance occur in proportion to the volume of community antibiotic use.5-7 Thus, perpetuation of antibiotic overuse promotes the continued evolution of ominous resistance profiles.

Since early in this decade, the European Surveillance of Antimicrobial Consumption project has been collecting country-level data on antibiotic utilization and has documented large differences in per capita antibiotic utilization between countries in Europe.8,9 Because the European Surveillance of Antimicrobial Consumption measures are not based on a particular clinical condition, an optimal rate of overall antibiotic utilization is difficult to establish. Nonetheless, clinical evidence and expert opinion strongly suggest that prescribing is most appropriate in countries at the lower end of the spectrum and that the difference between low- and high-utilizing countries to a considerable extent represents potentially unnecessary and avoidable prescriptions.8,10,11

Recently, the United States adopted a similar (albeit more limited) approach to track and report antibiotic utilization. In 2005, the National Committee for Quality Assurance (NCQA) developed and implemented a new measure to compare overall rates of antibiotic utilization between US health plans. By evaluating variation in antibiotic use within a country, these data can improve understanding of factors that contribute to variation in overall antibiotic utilization without the confounding effects of different countries’ healthcare systems. In addition, implementation of this measure in an established program to compare and improve healthcare quality can directly facilitate efforts to improve quality of care by identifying high-prescribing plans and stimulating them to investigate the factors that contribute to potentially excessive antibiotic use.7,8,12-15

In this study, we analyze antibiotic utilization rates among nonelderly members of commercial health maintenance organization (HMO) and point-of-service (POS) health plans in the United States participating in the NCQA’s Healthcare Effectiveness Data and Information Set (HEDIS) program. We quantified the degree of variation in antibiotic utilization rates across health plans, controlling for member characteristics, and estimated the cost implications of this variation.

METHODS

Data for this study were collected by commercial HMO and POS health plans in the United States that participate in NCQA’s HEDIS program.16 HEDIS is a voluntary program that collects data from health plans on various domains of effectiveness and utilization. These data are used to benchmark and compare the quality of care across health plans. Participating health plans account for more than 85% of individuals enrolled in US commercial plans. We report data from 2005, the first year plans reported data on overall antibiotic utilization for a new HEDIS measure.

We received a core set of data from 248 commercial health plans, representing 83% of commercial plans in established parts of the HEDIS program. The antibiotic utilization measure was a first-year measure. All first-year measures in HEDIS are not publicly reported in order to evaluate the measure, and, as is common, some plans choose not to report during this first year of data collection. We were able to obtain information on approximately 80% of the plans not contributing core data. Plans that did not report data had fewer members than those which did report (55% of nonreporting plans had fewer than 10,000 members vs 7% of reporting plans; P <.001).

Among 248 plans contributing core data, we excluded 4 health plans that did not provide basic data necessary for our analyses, 14 health plans whose rate of overall antibiotic utilization was extreme enough to suggest discrepancies in the way these data were collected or reported, and 1 plan that did not use an HMO or POS model of care. Our final analytic dataset thus comprised 229 health plans.

Our main outcome variable was each plan’s rate of antibiotic utilization per member per year (PMPY), as assessed by pharmacy claims billed. In calculating this rate, we counted each antibiotic prescription fill equally, regardless of drug dose or duration. Our analyses focused on people age 0 to 64 years, as most people age 65 years and older are enrolled in government-sponsored Medicare plans and those remaining in commercial plans may not be representative of the larger population of elders.

eAppendix

Each plan reported antibiotic utilization data for enrollees stratified into age-sex groups. To calculate standardized rates of antibiotic utilization, for each plan we multiplied the PMPY utilization rates for each of these age-sex strata by the proportion of the overall study cohort within each of these strata. Next, we summed each of these weighted rates to create an overall antibiotic utilization rate for all plan members age 0 to 64 years. This approach yielded a PMPY antibiotic utilization rate that adjusted for the age and sex distribution of each plan’s members. This standardized rate yielded figures similar to those of a crude rate that did not adjust for differences in the age and sex distribution of plans (see the , available at www.ajmc.com).

Next, we evaluated variation in use of specific types of antibiotics. Health plans provided utilization information for each of 15 classes of antibiotics, following a categorization scheme defined by a multistakeholder expert panel that worked with NCQA to arrive at a consensus definition that was both clinically relevant and feasible to implement. This panel further grouped each of the antibiotic classes into 2 categories: “antibiotics of concern,” agents generally considered to have broad-spectrum activity; and all other agents (“other antibiotics”), generally considered to have a narrow spectrum of antimicrobial activity (see Table 2). For purposes of clarity, we will use the terms “broad spectrum” and “narrow spectrum” to refer to these categories. We identified potential discrepancies in class-level utilization data for 20 plans. These discrepancies were most likely due to rounding errors, as plan-level data on utilization rates were provided to only 2 decimal places (eg, if the reported utilization rate for an antibiotic class was 0.04 antibiotic fills PMPY, the actual

rate could be anywhere from 0.0351 to 0.0449, an error rate of up to ±12%). We excluded these 20 plans from our analyses of antibiotic classes, plus an additional 3 plans that did not report data necessary to complete these analyses, leaving 206 plans with usable data. Because these discrepancies could affect estimates of drug expenditures, we also excluded these plans from our cost analyses, described below.

Next, we evaluated plan characteristics associated with the overall rate of antibiotic utilization. First, we conducted bivariate analyses using linear regression, where the outcome of interest was the plan’s age-sex adjusted rate of antibiotic utilization and the predictor variables were plan characteristics available from HEDIS that we hypothesized might be associated with antibiotic utilization. Next, we entered all variables into a multivariable linear regression model, on which further diagnostic testing confirmed the adequacy of model fit.

To estimate the cost of antibiotic utilization for each plan, we used several steps (see the eAppendix for details). Health plans provided data on utilization of antibiotic classes but not specific drugs within those classes. To compensate, we used data from the 2004 and 2005 National Ambulatory and National Hospital Ambulatory Medical Care Surveys to estimate the frequency of use of specific antibiotics within each antibiotic class. Next, we estimated the typical course of therapy for each antibiotic, including the most commonly used formulation, dose, dosing frequency, and duration of therapy, repeating this process separately for persons age 0 to 3 years, 4 to 9 years, and 10 to 64 years. For each antibiotic, we used data from the 2005 Red Book to assess the median price per pill (or other formulation), averaging across all manufacturers of a given drug and all bottle sizes.17 For multisource (generically available) drugs, we evaluated only the price for the generic versions. Finally, we calculated the estimated cost of a typical course of therapy for each antibiotic and then integrated this cost at the level of the antibiotic class and ultimately at the level of the health plan.

This research was approved by the Committee on Human Research at the University of California, San Francisco and the Research and Development Committee at the San Francisco VA Medical Center.

RESULTS

Table 1

Characteristics of the 229 health plans are shown in . Plans were distributed across the United States and varied widely in the size of their enrolled population. Together, these plans accounted for 42.9 million enrollees age 0 to 64 years. Children age 0 to 17 years comprised 27% of the overall member population.

Figure 1

Across all plans, the mean rate of antibiotic utilization was 0.88 prescription fills PMPY. Rates of antibiotic utilization varied widely among plans (), with a standard deviation (SD) of 0.15 fills PMPY, and 1.7-fold variation in rates of antibiotic utilization between the plans at the 5th and 95th percentiles of utilization (0.64 and 1.08 prescription fills PMPY, respectively). For a plan with 250,000 members, 89,000 fewer antibiotics would be dispensed each year if plan doctors prescribed antibiotics at the 10th percentile rate compared with the 90th percentile rate. If a similarly sized plan were to reduce its utilization from the 75th to 25th percentile, 47,000 fewer antibiotic prescriptions would be filled.

Figure 2

Variation in rates of antibiotic utilization persisted among age and sex groups (). Mean rates of antibiotic utilization differed across age-sex groups, and there was considerable variation in utilization rates within each group (SDs for each group ranged from 0.10 fills PMPY for males age 18-34 years to 0.24 fills PMPY for males age 0-9 years).

Table 2

shows variation in the distribution of types of antibiotics used among health plans. Across plans, a mean of 47% of antibiotics dispensed were broad-spectrum agents (“antibiotics of concern”). The proportion of broad-spectrum antibiotics varied among plans, ranging from 34% of antibiotic prescription fills for plans at the 5th percentile of broad-spectrum utilization to 59% for plans at the 95th percentile (SD ± 8%). There was little correlation between plans’ overall rate of antibiotic utilization and the proportion of antibiotics that were broad-spectrum agents (Pearson r = 0.12, P = .08).

Table 3

We next evaluated factors associated with a plan’s rate of overall antibiotic utilization. On bivariate analyses, plans with “excellent” accreditation status (the highest achievable rank) used more antibiotics than plans with “commendable” status (the second-highest rank), and plans with a smaller percentage of physicians who were board certified used more antibiotics than plans with a larger percentage of physicians who were board certified (). In addition, national region was strongly associated with rates of antibiotic utilization. Compared with plans in the West, plans in the South used a mean of 0.16 more antibiotic prescription fills PMPY, plans in the Midwest used 0.12 more fills PMPY, and plans in the Northeast used 0.07 more fills PMPY. As the mean rate of antibiotic utilization in the West was 0.78 fills PMPY, these differences correspond to a 21% higher rate of antibiotic utilization in the South, a 15% higher rate in the Midwest, and a 9% higher rate in the Northeast (see Table 3 footnote f for a further explanation of how these percentages were derived). On multivariable analysis, geographic region remained the most notable predictor of antibiotic utilization rates. Thirteen percent of the total variance in antibiotic utilization rates was explained by national region, while multivariable adjustment for all variables shown in Table 3 explained 26% of the total variance.

Finally, we estimated the cost implications of variation in antibiotic utilization. Across 206 plans with 38 million enrollees available for this analysis, total antibiotic expenditures totaled $1.90 billion, corresponding to a mean antibiotic cost of $49 PMPY. The SD of PMPY antibiotic costs was $9, and costs ranged from $34 PMPY for plans at the 5th percentile of costs to $63 PMPY for plans at the 95th percentile of costs. If all plans above the 25th percentile of antibiotic costs PMPY reduced their costs to this level, the mean (SD) drug cost savings would be $10 (±$6) PMPY. Across the 38 million enrollees of the plans we studied, that corresponds to $305 million in savings per year. If plans reduced their utilization to the 10th percentile, the drug cost savings would be greater, with mean (SD) savings of $14 (±$7) PMPY and total savings of $492 million per year.

DISCUSSION

In this study of 229 commercial health plans in the United States, we found substantial variation in the rate of antibiotic utilization, with high-utilizing plans dispensing nearly 70% more antibiotics per capita than low-utilizing plans. There was similarly large variation in the proportion of broad-spectrum antibiotics used by health plans, ranging from 34% of all antibiotics in plans at the lower end of the spectrum to 59% in plans at the higher end. Geographic region was strongly associated with health plans’ rate of antibiotic utilization: after controlling for other factors, plans in the southern United States used 0.16 more antibiotics PMPY than plans in the western United States, a difference larger than 1 SD of utilization rates nationally.

This variation in antibiotic utilization has substantial implications for health and healthcare costs in the United States. Overprescribing of antibiotics results in unnecessary drug side effects and promotes population-level resistance to antibiotics, which—consistent with our data—is greater in southern than in western states.18-20 Although resistance to any antibiotic is promoted by high-volume use, excessive prescribing of broad-spectrum agents is of particular concern because this overprescribing promotes resistance to agents that are commonly used to treat serious or complicated infections for which the consequences of treatment failure can be severe.2,21 Finally, unnecessary antibiotic use has important cost implications.3 Of an estimated $1.9 billion spent on antibiotics per year by health plans in our study cohort, direct drug costs could be reduced by 16% if health plans reduced their antibiotic utilization to the current 25th percentile of costs and by 26% if plans reduced their utilization to the current 10th percentile of costs. As with European antibiotic studies and other investigations into variation in the delivery of healthcare, it is difficult to determine the “correct” amount of antibiotic utilization. Nonetheless, understanding variation can provide valuable information to health plans and policymakers seeking to improve care quality and reduce unnecessary spending.12,14 Wennberg and colleagues were among the first to highlight the importance of reporting variation, pointing to large geographic differences within the United States in the use of healthcare interventions such as surgical procedures.22,23 For interventions for which an optimal rate of utilization is known, these data can stimulate benchmarking, allowing health plans and institutions to set achievable goals for their practice and to monitor progress toward these goals.24 Where an optimal rate of utilization is not known, reports of variation can help health plans and institutions understand their performance relative to that of their peers. This information can prompt plans with relatively high rates of utilization to examine why their delivery of services varies substantially from the norm, to determine whether this represents a remediable problem in quality, and if so to investigate how to improve their care quality.24

Our results suggest that overall rates of antibiotic utilization are well suited to a variation-centered approach. A number of commercial health plans are achieving far lower rates of antibiotic utilization than others. Of course, not all health plans are comparable; for example, plans whose enrollees have greater illness severity and barriers to accessing care may have legitimate reasons for prescribing more antibiotics than others.25 Thus, a high observed rate of antibiotic utilization should not be an end unto itself, but should prompt in-depth analysis to identify nonclinical factors that promote increased antibiotic use. Such analyses can be further guided by evaluation of prescribing rates within age and sex strata to determine whether certain patient subgroups have disproportionately high antibiotic utilization rates relative to a health plan’s peers. Where appropriate, local initiatives can be crafted to address the factors that promote increased antibiotic use, preferably borrowing from previous research to use active forms of clinician education and other methods proven to reduce unnecessary antibiotic prescribing.26-28 The NCQA took a variation-centered approach in creating this measure for the HEDIS program, in which overall antibiotic utilization has been included in the “use of services” domain, which tracks utilization and can be useful for comparison and identification of achievable goals without any specific performance targets.

Although we cannot definitively establish a “correct” rate of antibiotic utilization in commercial health plans, a variety of data suggest opportunities for improvement in the United States. Numerous studies have documented substantial overuse of antibiotics in the United States, and cross-national comparisons have found that Americans receive approximately 20% more antibiotics per capita than Europeans, with only 3 of 27 European countries having higher rates of antibiotic dispensing than the United States.1-3,10 Patients in the Netherlands, the lowest-prescribing European country, receive 60% fewer antibiotics than patients in the United States.10,29 Reducing antibiotic utilization is a complex endeavor and requires attention not only to clinical efficacy but also to patient satisfaction and downstream utilization of health services. Nonetheless, controlled trials to reduce inappropriate antibiotic use in the outpatient setting found no increase in subsequent health services utilization (eg, office visits, telephone calls) and little to no adverse impacts on patient satisfaction.28

Our study has several limitations. The data collected were from the first year in which this measure was implemented by HEDIS. Although plans were given detailed instructions for complying with this measure, it is possible that certain plans had not perfected their data-collection and reporting processes. In addition, data are not publicly reported for the first year of any HEDIS measure. As is common, some plans chose not to report their results during this first year, and it is difficult to know whether these plans did not participate because they expected their performance to be poor or because of other factors. (For example, more than half of nonreporting plans had fewer than 10,000 members, compared with 7% of reporting plans, suggesting the possibility that smaller plans might have had fewer resources available to put toward a first-year, non— publicly reported measure.) Nonetheless, we did receive data from 83% of plans, and all HEDIS data are audited, suggesting that our results are representative of the target population.

Other characteristics of our methods merit consideration in interpreting this study’s findings. First, the manner in which data were reported by health plans may result in slight imprecision in our calculation of overall antibiotic utilization rates. However, this imprecision is likely to be small in relation to the large variation in rates among plans. Second, our estimates of drug cost data were based on extrapolations from another national data source (National Ambulatory Medical Care Survey), combined with utilization data from the study plans, and do not precisely reflect the actual drug costs incurred by plans (which may be influenced by negotiated deals with drug suppliers, etc). Thus, our drug cost analyses should be interpreted as reasonable estimates rather than a precise accounting of real drug costs, and they do not account for downstream cost expenditures or savings associated with reduced antibiotic use. Third, we did not have access to clinical data such as comorbid conditions; thus, we could not control for interplan differences in members’ health beyond those that are correlated with patient age and sex. Finally, we collected data only on HMOs and POS plans participating in the HEDIS program. Although the strong majority of eligible commercial health plans participate in the HEDIS program, we cannot know the generalizability of our results to plans not participating in HEDIS or to persons with public insurance (eg, Medicaid), other forms of commercial insurance, or no insurance at all.

It is difficult to improve healthcare quality unless it can be measured. The substantial unexplained variation in antibiotic utilization across US health plans suggests opportunities to improve the quality and costs of antibiotic prescribing. We believe the NCQA antibiotic utilization measure should stimulate additional efforts to understand and improve antibiotic utilization at the health plan level—particularly for health plans in the higher range of antibiotic use. If successful, these efforts are likely to improve quality of care and to generate meaningful cost savings from reduced antibiotic costs—a winwin scenario for patients, payers, and the public health.

Acknowledgments

We thank Min Gayles, formerly at the National Committee for Quality Assurance, for her assistance obtaining the data and facilitating this research, and Saunak Sen, PhD, and John Boscardin, PhD, for their assistance with data analysis.

Author Affiliations: From the Division of Geriatrics (MAS), San Francisco Veterans Affairs Medical Center, San Francisco, CA; Department of Clinical Pharmacy (KYY) and Department of Medicine (MAS, JHM, RG), University of California, San Francisco, CA; and the National Committee for Quality Assurance (SCB), Washington, DC.

Funding Source: This work was supported by Career Development Transition Award 01-013 from the VA Health Services Research and Development Service (Dr Steinman), by grant K23-AG030999 from the National Institute on Aging and the American Federation for Aging Research (Dr Steinman), by the Agency for Healthcare Research and Quality (NCQA) Translating Research into Practice Program (R01 HS13915; Dr Gonzales and Ms Maselli) and by grant KL2 RR024130 from the National Center for Research Resources (Dr Yang). We submitted an advance copy of this paper to NCQA for review and comments, but that institution had no control over the analyses and interpretation of data or over the decision to publish this work. The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official view of the Department of Veterans Affairs, the National Center for Research Resources, or the National Institutes of Health.

Author Disclosures: The authors (MAS, KYY, SCB, JHM, and RG) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (MAS, RG); acquisition of data (KYY, SCB, JHM); analysis and interpretation of data (MAS, KYY, SCB, JHM, RG); drafting of the manuscript (MAS, RG); critical revision of the manuscript for important intellectual content (MAS, KYY, SCB, RG); statistical analysis (MAS, JHM); obtaining funding (MAS); and administrative, technical, or logistic support (MAS).

Address correspondence to: Michael A. Steinman, MD, San Francisco VA Medical Ctr, 4150 Clement St, Box 181-G, San Francisco, CA 94121. E-mail: mike.steinman@ucsf.edu.

1. Linder JA, Stafford RS. Antibiotic treatment of adults with sore throat by community primary care physicians: a national survey, 1989-1999. JAMA. 2001;286(10):1181-1186.

2. Steinman MA, Gonzales R, Linder JA, Landefeld CS. Changing use of antibiotics in community-based outpatient practice, 1991-1999. Ann Intern Med. 2003;138(7):525-533.

3. Gonzales R, Malone DC, Maselli JH, Sande MA. Excessive antibiotic use for acute respiratory infections in the United States. Clin Infect Dis. 2001;33(6):757-762.

4. Shehab N, Patel PR, Srinivasan A, Budnitz DS. Emergency department visits for antibiotic-associated adverse events. Clin Infect Dis. 2008;47(6):735-743.

5. Seppälä H, Klaukka T, Vuopio-Varkila J, et al. The effect of changes in the consumption of macrolide antibiotics on erythromycin resistance in group A streptococci in Finland. Finnish Study Group for Antimicrobial Resistance. N Engl J Med. 1997;337(7):441-446.

6. Austin DJ, Kristinsson KG, Anderson RM. The relationship between the volume of antimicrobial consumption in human communities and the frequency of resistance. Proc Natl Acad Sci U S A. 1999;96(3):1152-1156.

7. Goossens H, Ferech M, Vander Stichele R, Elseviers M; ESAC Project Group. Outpatient antibiotic use in Europe and association with resistance: a cross-national database study. Lancet. 2005;365(9459):579-587.

8. Elseviers MM, Ferech M, Vander Stichele RH, Goossens H; ESAC Project Group. Antibiotic use in ambulatory care in Europe (ESAC data 1997-2002): trends, regional differences and seasonal fluctuations. Pharmacoepidemiol Drug Saf. 2007;16(1):115-123.

9. Vander Stichele RH, Elseviers MM, Ferech M, Blot S, Goossens H; ESAC Project Group. European surveillance of antimicrobial consumption (ESAC): data collection performance and methodological approach. Br J Clin Pharmacol. 2004;58(4):419-428.

10. Goossens H, Ferech M, Coenen S, Stephens P; European Surveillance of Antimicrobial Consumption Project Group. Comparison of outpatient systemic antibacterial use in 2004 in the United States and 27 European countries [published correction appears in Clin Infect Dis. 2007;44(9):1259]. Clin Infect Dis. 2007;44(8):1091-1095.

11. Coenen S, Ferech M, Haaijer-Ruskamp FM, et al; ESAC Project Group. European Surveillance of Antimicrobial Consumption (ESAC): quality indicators for outpatient antibiotic use in Europe. Qual Saf Health Care. 2007;16(6):440-445.

12. Aspinall SL, Berlin JA, Zhang Y, Metlay JP. Facility-level variation in antibiotic prescriptions for veterans with upper respiratory infections. Clin Ther. 2005;27(2):258-262.

13. Smith S, Smith GE, Heatlie H, et al. Reducing variation in antibacterial prescribing rates for ‘cough/cold’ and sore throat between 1993 and 2001: regional analyses using the general practice research database. Public Health. 2006;120(8):752-759.

14. O’Connor GT, Quinton HB, Traven ND, et al. Geographic variation in the treatment of acute myocardial infarction: the Cooperative Cardiovascular Project. JAMA. 1999;281(7):627-633.

15. The Dartmouth Atlas Working Group. The Dartmouth Atlas of Health Care. http://www.dartmouthatlas.org/. Accessed June 11, 2008.

16. National Committee for Quality Assurance. HEDIS & quality measurement. http://www.ncqa.org/tabid/59/Default.aspx. Accessed June 11, 2008.

17. Fleming T, ed. 2005 Redbook: Pharmacy’s Fundamental Reference (Red Book Drug Topics). 109 rev ed. Montvale, NJ: Thomson Healthcare; May 30, 2005.

18. Denys GA, Koch KM, Dowzicky MJ. Distribution of resistant grampositive organisms across the census regions of the United States and in vitro activity of tigecycline, a new glycylcycline antimicrobial. Am J Infect Control. 2007;35(8):521-526.

19. Whitney CG, Farley MM, Hadler J, et al; Active Bacterial Core Surveillance Program of the Emerging Infections Program Network. Increasing prevalence of multidrug-resistant Streptococcus pneumoniae in the United States. N Engl J Med. 2000;343(26):1917-1924.

20. Centers for Disease Control and Prevention (CDC). Geographic variation in penicillin resistance in Streptococcus pneumoniae—selected sites, United States, 1997. MMWR Morb Mortal Wkly Rep. 1999;48(30):656-661.

21. Mainous AG 3rd, Hueston WJ, Davis MP, Pearson WS. Trends in antimicrobial prescribing for bronchitis and upper respiratory infections among adults and children. Am J Public Health. 2003;93(11):1910-1914.

22. Lu-Yao GL, McLerran D, Wasson J, Wennberg JE. An assessment of radical prostatectomy. Time trends, geographic variation, and outcomes. The Prostate Patient Outcomes Research Team. JAMA. 1993;269(20):2633-2636.

23. Wennberg DE. Variation in the delivery of health care: the stakes are high. Ann Intern Med. 1998;128(10):866-868.

24. Kiefe CI, Weissman NW, Allison JJ, Farmer R, Weaver M, Williams OD. Identifying achievable benchmarks of care: concepts and methodology. Int J Qual Health Care. 1998;10(5):443-447.

25. Zaslavsky AM, Hochheimer JN, Schneider EC, et al. Impact of sociodemographic case mix on the HEDIS measures of health plan quality. Med Care. 2000;38(10):981-992.

26. Butler CC, Rollnick S, Pill R, Maggs-Rapport F, Stott N. Understanding the culture of prescribing: qualitative study of general practitioners’ and patients’ perceptions of antibiotics for sore throats. BMJ. 1998;317(7159):637-642.

27. Poses RM, Wigton RS, Cebul RD, Centor RM, Collins M, Fleischli GJ. Practice variation in the management of pharyngitis: the importance of variability in patients’ clinical characteristics and in physicians’ responses to them. Med Decis Making. 1993;13(4):293-301.

28. Ranji SR, Steinman MA, Shojania KG, Gonzales R. Interventions to reduce unnecessary antibiotic prescribing: a systematic review and quantitative analysis. Med Care. 2008;46(8):847-862.

29. Cars O, Mölstad S, Melander A. Variation in antibiotic use in the European Union. Lancet. 2001;357(9271):1851-1853.