Currently Viewing:
Supplements The Aligning Forces for Quality Initiative: Early Lessons From Efforts to Improve Healthcare Quality
Creating and Sustaining Change: Early Insights From Aligning Forces
Claire B. Gibbons, PhD, MPH; and Anne F. Weiss, MPP
Getting the Structure Right for Communitywide Healthcare Improvement
Gordon Mosser, MD
Lessons for Reducing Disparities in Regional Quality Improvement Efforts
Scott C. Cook, PhD; Anna P. Goddu, MSc; Amanda R. Clarke, MPH; Robert S. Nocon, MHS; Kevin W. McCullough, MJ; and Marshall H. Chin, MD, MPH
The Imperative to Promote Collaborative Consumer Engagement: Lessons From the Aligning Forces for Quality Initiative
Debra L. Ness, MS
That Was Then, This Is Now
Lisa A. Simpson, MB, BCh, MPH, FAAP
Regional Health Improvement Collaboratives Needed Now More Than Ever: Program Directors' Perspectives
Randall D. Cebul, MD; Susanne E. Dade, MPA; Lisa M. Letourneau, MD, MPH; and Alan Glaseroff, MD, ABFM
The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2012
Dennis P. Scanlon, PhD; Jeff Beich, PhD; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Romana Hasnain-Wynia, PhD; Megan C. McHugh, PhD; and Jessica N. Mittler, PhD
Barriers and Strategies to Align Stakeholders in Healthcare Alliances
Larry R. Hearld, PhD; Jeffrey A. Alexander, PhD; Jeff Beich, PhD; Jessica N. Mittler, PhD; and Jennifer L. O’Hora, BA
The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2012 - eAppendix
Midterm Observations and Recommendations From the Evaluation of the AF4Q Initiative
Jeffrey A. Alexander, PhD; Dennis P. Scanlon, PhD; Megan C. McHugh, PhD; Jon B. Christianson, PhD; Jessica N. Mittler, PhD; Romana Hasnain-Wynia, PhD; and Jeff Beich, PhD
Currently Reading
Producing Public Reports of Physician Quality at the Community Level: The Aligning Forces for Quality Initiative Experience
Jon B. Christianson, PhD; Karen M. Volmar, JD, MPH; Bethany W. Shaw, MHA; and Dennis P. Scanlon, PhD
Approaches to Improving Healthcare Delivery by Multi-stakeholder Alliances
Megan C. McHugh, PhD; Jillian B. Harvey, MPH; Dasha Aseyev, BS; Jeffrey A. Alexander, PhD; Jeff Beich, PhD; and Dennis P. Scanlon, PhD
Evaluating a Community-Based Program to Improve Healthcare Quality: Research Design for the Aligning Forces for Quality Initiative
Dennis P. Scanlon, PhD; Jeffrey A. Alexander, PhD; Jeff Beich, PhD; Jon B. Christianson, PhD; Romana Hasnain-Wynia, PhD; Megan C. McHugh, PhD; Jessica N. Mittler, PhD; Yunfeng Shi, PhD; and Laura J. B
Using Websites to Engage Consumers in Managing Their Health and Healthcare
Jessica N. Mittler, PhD; Karen M. Volmar, JD, MPH; Bethany W. Shaw, MHA; Jon B. Christianson, PhD; and Dennis P. Scanlon, PhD
Participating Faculty: The Aligning Forces for Quality Initiative: Early Lessons From Efforts to Improve Healthcare Quality at the Community Level
Letter From the Guest Editor
David Blumenthal, MD, MPP
Samuel O. Thier Professor of Medicine and Professor of Health Care Policy Massachusetts General Hospital/Partners HealthCare System and Harvard Medical School, Boston

Producing Public Reports of Physician Quality at the Community Level: The Aligning Forces for Quality Initiative Experience

Jon B. Christianson, PhD; Karen M. Volmar, JD, MPH; Bethany W. Shaw, MHA; and Dennis P. Scanlon, PhD
Objectives: To describe the approaches used by the Aligning Forces for Quality (AF4Q) alliances in producing community-based reports of physician quality and to assess the contribution of these reports to existing physician performance information.
Study Design: The study included semi-structured interviews with alliance stakeholders and tracking of the number and content of physician performance reports in 14 AF4Q initiative communities and 7 comparison communities.
Methods: The study used qualitative analysis of interview data and systematic tracking of the number and content of physician performance reports over time.
Results: Report production occurred in several stages including initiation, measure selection/specification, measure construction, and dissemination. The measure selection/specification process was often the first major act undertaken by alliances under the AF4Q initiative grant. Alliances utilized nationally endorsed performance measures and made a strategic decision to gain buy-in with physicians. Alliances have experienced greater difficulty in producing buy-in for patient experience measures. The primary decision point for measure construction was the use of administrative claims data or physician-provided medical records data. Overall, AF4Q alliances have contributed to an increase in physician performance information in their communities.
Conclusions: Our findings suggest that the AF4Q initiative has accelerated the development and content of physician performance measures in AF4Q communities.

(Am J Manag Care. 2012;18:S133-S140)


The public reporting of physician performance has been supported by the Bush and Obama administrations as an important element of healthcare reform.1,2 Since 2006, the Robert Wood Johnson Foundation (RWJF) has encouraged public reporting efforts by community healthcare coalitions (ie, alliances, the multi-stakeholder partnerships receiving funding through the Foundation’s Aligning Forces for Quality [AF4Q] initiative) chosen to participate in its AF4Q initiative3 as one way to address deficiencies in quality of care.4 The value of locally produced versus national-level physician performance reports is based on several premises: (1) local efforts are more effective in reporting physician performance measures salient to the community and add to the information available to community residents when selecting providers; (2) physicians view the results as more credible because they have played a role in report development, leading them to engage in more effective, targeted quality improvement efforts; and (3) locally developed reports receive more local media attention, enhancing visibility and credibility with consumers and increasing the likelihood that they will use the information in making healthcare decisions.

Nevertheless, there is reason to be skeptical that coalitions of diverse stakeholders can produce credible physician performance reports, as previous local, voluntary collaborative efforts to implement health system change strategies have had mixed success.5-7 Also, because data submission by providers or health plans is voluntary, the process of producing credible comparative performance reports is fraught with political and technical challenges for even well-funded, historically effective community healthcare coalitions.

With the exception of recent work by Young,8 relatively little has been written about the community-based reporting process and whether resulting reports add significantly to the amount and relevance of physician performance information available to consumers. This article contrasts the approaches that different AF4Q alliances have taken in producing community-based reports with clinical quality and patient experience measures, and assesses the contribution of these reports to existing physician performance information.


The AF4Q initiative provides guidance, technical assistance, and funding to community coalitions, in the hope that this will accelerate their provision of physician performance information, which will change consumer and provider behaviors, thereby improving healthcare quality (see eAppendix A at, AF4Q Initiative: Public Reporting Logic Model). When the AF4Q initiative began, only 3 of the initial 14 AF4Q alliances were publicly reporting ambulatory quality data. Nationally, there were relatively few local or state-level efforts to measure and report physician performance. Upon joining the AF4Q initiative, alliances were given a goal to publicly report ambulatory quality measures for over 50% of primary care providers in their communities within 3 years; it was implied that funding could be discontinued if this target was not met.9 The AF4Q initiative provided technical assistance, including webinars on the selection and construction of ambulatory quality of care measures. Later, the AF4Q initiative required the alliances to add patient experience measures to their physician performance data, along with expanding the scope of the reports to include hospital quality and provider efficiency measures. But, due to the urgency attached to the early reporting of physician performance data, and the fact that alliances had more experience in this area than in hospital reporting, this article focuses only on reporting of physician performance measures by the 14 original alliances (data from additional alliances that joined in 2009-2010 are not included because of their limited experience as AF4Q participants).

Some alliance leaders believed that the AF4Q initiative placed disproportionate emphasis on achieving its early public reporting target relative to the attention given to the program’s other core areas—quality improvement and consumer engagement—and that its reporting timeline was overly ambitious. Representing this viewpoint, one alliance leader said, “…the real push is this public reporting piece which is the endgame for AF4Q.” Additionally, some alliance leaders did not share the AF4Q initiative’s view of the potential value of physician performance reports. Their skepticism was expressed in comments such as: “You want to engage consumers, but…with the quality data, it’s interesting and it’s sexy but ‘what the hell are they [consumers] supposed to do with it?’”; and, “…I think the original belief was that all you need to do is report, and things will magically get better …” On the other hand, many alliance leaders felt that the AF4Q initiative’s ambitious reporting target helped move stakeholders from general support for the alliance and its mission to specific actions. In the end, all but 1 alliance was successful in disseminating a public report by the 3-year target date covering at least 50% of primary care physicians in the community (Figure); the reporting efforts of the sole unsuccessful alliance were delayed by state-level legal issues related to uses of health plan data. The sections that follow assess how alliances produced their reports and the contributions of the reports to the existing physician performance information in their communities.

Data Sources

The analyses are based on 2 data sources: semi-structured in-person and telephone interviews that provided information about various aspects of physician performance reporting, such as goals, strategies, and processes for measure construction,10 and the ongoing tracking of contents of alliance reports and the reports produced by other organizations in alliance communities. Interview responses were transcribed from audio recordings, and text files were read and coded. Codes related to performance measurement and public reporting included topics such as challenges/barriers to measurement and reporting; clinical quality and patient experience measurement; data aggregation; and data collection. The coded text was entered into Atlas.ti, a software package for qualitative analysis. These data were used primarily in assessing the processes used by AF4Q alliances to construct reports of physician performance.

Data were collected on the presence and content of public reports in areas served by the 14 original alliances. Without knowledge of their public reporting history, 7 areas that were similar to AF4Q communities in location, population size, and demographics also were selected. Each year, beginning in 2007, we reviewed the websites of hospitals and medical associations, healthcare coalitions, quality improvement organizations, state departments of health, and the AF4Q alliances to document public reporting activities. In addition, websites for the 5 largest commercial health plans, which included national plans operating in the AF4Q communities, were examined. In communities where there were fewer than 5 significant plans, websites for plans with membership that together constituted approximately 75% or more of the total private sector health plan enrollment in the area were reviewed. In all (AF4Q and other) communities, we collected information from organizations that sponsor public reports to verify the search findings and gather further details regarding measure sources and construction. We used this information to determine if alliance public reporting efforts contributed to the type and amount of physician performance information available to consumers in AF4Q communities, how the availability of this information compared with other communities, and if alliance reporting changed over time in ways consistent with AF4Q initiative expectations.


Producing Public Reports

Alliance report production occurred in several stages, including initiation, measure selection, measure construction, and dissemination (see eAppendix B, Alliance Public Reporting Process). In this article, we examine the AF4Q initiative efforts in selecting and constructing physician performance measures (dissemination activities are addressed by Mittler et al in this supplement11). For convenience, we discuss these 2 stages separately, but they overlap at points. For instance, while alliances chose measures in areas where care deficiencies have been documented by national studies, the selection of specific measures was guided by early judgments about what types of data were likely to be available for measure construction. In the Table, we summarize the physician performance measures selected by the alliances and the results of 2 key decisions made in constructing those measures.

Measure Selection

Prior to the AF4Q initiative, 8 alliances did not have communitywide processes in place to select and construct ambulatory care quality measures. After joining the AF4Q initiative, all 14 original alliances developed these processes, and all met AF4Q initiative expectations that their reports include nationally endorsed measures. For alliances that were not previously reporting physician performance, AF4Q initiative participation was seen as critical to measure development. One respondent observed that the AF4Q initiative gave the alliance “…the ability to create the infrastructure to bring the physicians together, to create the agreement upon the measures, and to actually get them up there [measures reported]. That would never have happened without Aligning Forces.”

All alliances considered obtaining physician “buy-in” to reporting an essential first step and a necessary precursor to measure selection. The alliances went to great lengths to garner physician support. One alliance leader observed that “…we have been to every venue we could possibly think of in the last 6 months; talking to physician groups…trying to engage them about the measures and do these make sense, do they not and trying to explain to them about the rationale about picking the measures…” The alliances typically established physician-led work groups charged with recommending measures in different areas. The measure recommendations were then reviewed by steering committees that had broader representation. Prior to making final selections, some alliances distributed proposed measure specifications communitywide for feedback, and then made further modifications if necessary.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up