Currently Viewing:
Supplements The Aligning Forces for Quality Initiative: Summative Findings and Lessons Learned From Efforts to Improve Healthcare Quality at the Community Level
The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2015
Dennis P. Scanlon, PhD; Jeff Beich, PhD; Brigitt Leitzell, MS; Bethany W. Shaw, MHA; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Diane C. Farley, BA; Jessica Greene, PhD; Muriel Jean-Jacques,
Summative Evaluation Results and Lessons Learned From the Aligning Forces for Quality Program
Dennis P. Scanlon, PhD; Jeffrey A. Alexander, PhD; Megan McHugh, PhD; Jeff Beich, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Brigitt Leitzell, MS; Yunfeng Shi,
The Longitudinal Impact of Aligning Forces for Quality on Measures of Population Health, Quality and Experience of Care, and Cost of Care
Yunfeng Shi, PhD; Dennis P. Scanlon, PhD; Raymond Kang, MA; Megan McHugh, PhD; Jessica Greene, PhD; Jon B. Christianson, PhD; Muriel Jean-Jacques, MD, MAPP; Yasmin Mahmud, MPH; and Jeffrey A. Alexande
Currently Reading
Reporting Provider Performance: What Can Be Learned From the Experience of Multi-Stakeholder Community Coalitions?
Jon B. Christianson, PhD; Bethany W. Shaw, MHA; Jessica Greene, PhD; and Dennis P. Scanlon, PhD
From Rhetoric to Reality: Consumer Engagement in 16 Multi-Stakeholder Alliances
Jessica Greene, PhD; Diane C. Farley, BA; Jon B. Christianson, PhD; Dennis P. Scanlon, PhD; and Yunfeng Shi, PhD
Lessons Learned About Advancing Healthcare Equity From the Aligning Forces for Quality Initiative
Muriel Jean-Jacques, MD, MAPP; Yasmin Mahmud, MPH; Jaime Hamil, MPH; Raymond Kang, MA; Philethea Duckett, MPA; and Juliet C. Yonek, MPH, PhD
Aligning Forces for Quality Multi-Stakeholder Healthcare Alliances: Do They Have a Sustainable Future
Jeffrey A. Alexander, PhD; Larry R. Hearld, PhD; Laura J. Wolf, MSW; and Jocelyn M. Vanderbrink, MHA
Evaluating a Complex, Multi-Site, Community-Based Program to Improve Healthcare Quality: The Summative Research Design for the Aligning Forces for Quality Initiative
Dennis P. Scanlon, PhD; Laura J. Wolf, MSW; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Megan McHugh, PhD; Yunfeng Shi, PhD; Brigitt Leitze
Participating Faculty
Letter From Donald M. Berwick, MD, MPP, Guest Editor
Donald M. Berwick, MD, MPP
The View From Aligning Forces to a Culture of Health
Carolyn E. Miller, MSHP, MA, and Anne F. Weiss, MPP
Leading Multi-sector Collaboration: Lessons From the Aligning Forces for Quality National Program Office
Katherine O. Browne, MBA, MHA; Robert Graham, MD; and Bruce Siegel, MD, MPH
Healthcare Reform Post AF4Q: A National Network of Regional Collaboratives Continues Healthcare Reform From the Ground Up
Elizabeth Mitchell and Dianne Hasselman, MSPH

Reporting Provider Performance: What Can Be Learned From the Experience of Multi-Stakeholder Community Coalitions?

Jon B. Christianson, PhD; Bethany W. Shaw, MHA; Jessica Greene, PhD; and Dennis P. Scanlon, PhD
Objectives: This analysis assessed the evolution of public reporting of provider performance in Aligning Forces for Quality (AF4Q) alliances, contrasted alliances that stopped reporting with those that plan to continue, and drew insights from alliance public reporting efforts for the national transparency movement.

Methods: Combined with document review, qualitative research methods were used to analyze interview data collected, over a nearly 10-year period, from the 16 participating alliances.

Results: AF4Q alliances made their greatest contributions to provider transparency in reporting ambulatory quality and patient experience measures. However, reporting ambulatory cost/efficiency/utilization measures was more challenging for alliances. Alliances contributed the least with respect to measures of inpatient performance. Six alliances ceased reporting at the end of the AF4Q program because of their inability to develop stable funding sources and overcome stakeholder skepticism about the value of public reporting. Insights provided by alliance leaders included the need to: focus on provider, rather than consumer, responses to public reports as the most likely avenue for improving quality; address the challenge of funding the reporting infrastructure from the beginning; explore collaborations with other entities to increase public reporting efficiency; and develop a strategy for responding to efforts at the national level to increase the availability of information
on provider performance.

Conclusion: The AF4Q initiative demonstrated that a wide variety of voluntary stakeholder coalitions could develop public reports with financial and technical support. However, the contents of these reports varied considerably, reflecting differences in local environments and alliance strategies. The challenges faced by alliances to maintain their reporting efforts were substantial, and not all alliances chose to report. Nevertheless, there are potential roles for alliances going forward in contributing to the national transparency movement.

Am J Manag Care. 2016;22:S382-S392
In this article, we examine the efforts of voluntary stakeholder coalitions (ie, alliances) to measure and publicly report provider performance as part of the Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) initiative. (Details regarding these alliances and the communities they served are outlined in the article by Scanlon et al located in this supplement.1) Public reporting of provider performance was a key component of the overall strategy of the AF4Q initiative to improve the quality of care in alliance communities (see logic model in eAppendix A). Transparent quality measures could encourage consumers to choose higher-quality providers, enhance patient interactions with providers, and stimulate providers to undertake quality improvement activities to avoid losing patients and the stigma that might be attached to poor performance on quality measures. Health plans could use these measures to develop benefit designs that distinguish higher-quality from lower-quality providers (possibly rewarding consumers for choosing higher-quality providers) and implement pay-for-performance programs. This view of the possible benefits of public reporting was similar to that expressed by national advocates of greater provider performance transparency.2

Consistent with this emphasis, alliances were selected to participate in the AF4Q program in part because they had experience with, or expressed a willingness to implement, public reporting.3 However, our analysis (further detailed below) found substantial variation in the performances of alliances with respect to AF4Q public reporting. Some met the reporting goals and timelines established by the AF4Q program and planned to continue reporting after the program’s conclusion, others were less successful, and some stopped reporting entirely once the AF4Q program funding ended. Overall, in a separate analysis, we found that consumers in alliance communities had access to more provider performance information than available to residents of comparison communities4 and this remained true throughout the AF4Q initiative.5

To place the AF4Q program public reporting efforts in context, we begin by discussing national performance measurement and public reporting efforts prior to, during, and at the end of the AF4Q initiative. We summarize expectations for measurement and public reporting from the AF4Q program and how they changed over the course of the initiative. After describing our methods for data collection and analysis, we address 3 questions: (1) Across AF4Q communities, how did public reporting evolve and what factors drove that evolution? (2) At the end of the AF4Q program, what distinguished alliances that continued to report from those that stopped reporting? (3) What insights can be drawn from the alliance public reporting experience to inform national efforts to increase provider performance transparency?


National Reporting Landscape

Reporting health plan quality measures began in the early 1990s6 under the auspices of the National Committee for Quality Assurance (NCQA). The federal Health Care Financing Administration (now CMS) and several states (eg, New York, Pennsylvania, and California) began reporting hospital quality measures during this time period, as well. By 2000, large employers were encouraging the reporting of physician performance as part of an overall healthcare reform strategy.7 Many health plans responded by reporting a limited number of provider quality measures to their members, or by “tiering” providers based on quality measures, but this information usually was not available to the general public. Private firms also entered the reporting arena (eg, Healthgrades and WebMD), with consumers typically paying for full access to their provider performance measures.

The National Quality Forum (NQF) was established in 1999 as a public–private partnership to “create a foundation for consistent data reporting and collection.”8 Incorporating measures developed by NCQA or endorsed by NQF subsequently became the “gold standard” for public reporting. In 2001, under the auspices of the Leapfrog Group, large employers began to publish information on hospital patient safety practices, using data voluntarily submitted by some hospitals. The Ambulatory Care Quality Alliance (the American Academy of Family Physicians, American College of Physicians, America’s Health Insurance Plans, and the Agency for Healthcare Research and Quality), established in 2004, produced a “starter set” of physician quality measures, which it pilot-tested in 2006. In the public sector, Hospital Compare was created by CMS in 2002, and in 2005 the first process of care measures were displayed on the Hospital Compare website, with patient experience measures added soon after.9

Clearly, by the time the first alliances were selected for the AF4Q program in 2006, there was considerable momentum for reporting provider performance, especially quality measures. Subsequently, several additional public-sector reporting efforts were initiated, beginning with the establishment of the Chartered Value Exchange (CVE) program by the Bush administration in 2008. Under this program, community organizations could apply for CVE designation, which entailed a commitment to publish provider quality information. In return, CVEs were to receive access to Medicare performance measurement results, along with technical assistance through a peer-learning network. Of the 24 organizations that received the CVE designation by 2012, 11 were AF4Q alliances.

Support for public reporting expanded under the Obama administration’s Affordable Care Act (ACA), which required CMS to share Medicare data with “qualified” local entities for use in reporting provider performance.10 In addition, advocacy groups encouraged the reporting of physician quality measures on ACA health insurance exchanges. Meanwhile, Medicare’s Physician Compare effort provided data on individual physician characteristics: in 2015, patient assessments of care and clinical quality measures became available for practices and groups. This effort also gave consumers access to information on whether individual providers or medical groups participated in various quality programs, including the Physician Quality Reporting System, the Million Hearts initiative, and the Electronic Health Records Incentive Program.11 In addition to its focus on physicians, CMS initiated public reporting for skilled nursing facilities and home healthcare.

Alongside these federal efforts, 43 states have developed, or plan to develop, all-payer claims databases, providing a resource that could be used in producing provider performance reports.12 Subsequent to the establishment of the AF4Q program, private-sector purchasers increased their support for the reporting of measures of cost and efficiency, in addition to quality measures,13 with the hope that consumers would consider “value” (defined as quality relative to dollars spent) when making their choices and that providers would be encouraged to compete based on the value of the services they provided.14

AF4Q Program Initiatives

Two of the first 4 alliances selected to participate in AF4Q were already publicly reporting provider performance, and a third was in the process of measure construction. However, there were still relatively few community coalition public reporting efforts nationally at the onset of the AF4Q program, although some coalitions were actively involved in Leapfrog reporting efforts.15 There were several aspects of locally produced reports that appeared promising (ie, adding value to national reporting efforts) and deserving of support in the AF4Q initiative. Locally produced reports could increase the number of different sources of performance information available in the community, raising the likelihood that consumers would become aware of provider performance measures. It was hoped that the reports would draw significant attention, because they were “locally developed,” and that providers would view them as credible, especially when they were involved in measure selection or development. Finally, to the degree that local reporting efforts were guided by input from community stakeholders, their measures might be more salient to community residents than those in national reports.3

When they joined the AF4Q program, alliances were charged with publicly reporting (within 3 years) measures of ambulatory quality in the treatment of chronic conditions for at least 50% of primary care physicians in their communities. Alliance leaders felt that their continued funding could very well depend on complying. Almost all alliances reached this goal, although report content varied considerably.3 Subsequently, expectations for alliances were expanded to include reporting patient experience measures for ambulatory care, hospital quality and patient experience measures, and measures of resource use, charge, price, cost and/or efficiency in inpatient and outpatient settings. Alliances were encouraged to make their reports “consumer -friendly” and to pursue different methods of disseminating reports. By the end of the AF4Q program in 2015, the alliances were expected to produce public reports that facilitated consumer selection of “high-value” providers.

In summary, the reporting goals for AF4Q alliances were very ambitious. The hope was that alliances would play a leading role in the emerging transparency movement by providing “models” for public reporting that could be adopted in other communities. During the nearly 10-year AF4Q initiative, the number of NQF and NCQA performance measures of all types expanded considerably, which aided alliances in selecting and constructing measures for their reports. In addition to funding, the AF4Q program provided alliances with technical assistance to support their reporting efforts.

Data Sources

We used 3 data sources in our analyses. We tracked the contents of alliance reports throughout the AF4Q program and constructed a longitudinal dataset for each alliance that contained information on the types of measures reported, start and stop dates for reporting of measures, level of reporting (eg, medical group or medical practice), data sources used to construct each measure (eg, claims, medical records, surveys), and frequency with which measures were reported (eg, annually, biannually). Using this dataset, we examined how reporting evolved across alliances and over time.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up