Currently Viewing:
Supplements The Aligning Forces for Quality Initiative: Summative Findings and Lessons Learned From Efforts to Improve Healthcare Quality at the Community Level
The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2015
Dennis P. Scanlon, PhD; Jeff Beich, PhD; Brigitt Leitzell, MS; Bethany W. Shaw, MHA; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Diane C. Farley, BA; Jessica Greene, PhD; Muriel Jean-Jacques,
Summative Evaluation Results and Lessons Learned From the Aligning Forces for Quality Program
Dennis P. Scanlon, PhD; Jeffrey A. Alexander, PhD; Megan McHugh, PhD; Jeff Beich, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Brigitt Leitzell, MS; Yunfeng Shi,
The Longitudinal Impact of Aligning Forces for Quality on Measures of Population Health, Quality and Experience of Care, and Cost of Care
Yunfeng Shi, PhD; Dennis P. Scanlon, PhD; Raymond Kang, MA; Megan McHugh, PhD; Jessica Greene, PhD; Jon B. Christianson, PhD; Muriel Jean-Jacques, MD, MAPP; Yasmin Mahmud, MPH; and Jeffrey A. Alexande
Reporting Provider Performance: What Can Be Learned From the Experience of Multi-Stakeholder Community Coalitions?
Jon B. Christianson, PhD; Bethany W. Shaw, MHA; Jessica Greene, PhD; and Dennis P. Scanlon, PhD
Currently Reading
Improving Care Delivery at the Community Level: An Examination of the AF4Q Legacy
Megan McHugh, PhD; Jillian B. Harvey, MPH, PhD; Jaime Hamil, MPH; and Dennis P. Scanlon, PhD
Lessons Learned About Advancing Healthcare Equity From the Aligning Forces for Quality Initiative
Muriel Jean-Jacques, MD, MAPP; Yasmin Mahmud, MPH; Jaime Hamil, MPH; Raymond Kang, MA; Philethea Duckett, MPA; and Juliet C. Yonek, MPH, PhD
Aligning Forces for Quality Multi-Stakeholder Healthcare Alliances: Do They Have a Sustainable Future
Jeffrey A. Alexander, PhD; Larry R. Hearld, PhD; Laura J. Wolf, MSW; and Jocelyn M. Vanderbrink, MHA
Evaluating a Complex, Multi-Site, Community-Based Program to Improve Healthcare Quality: The Summative Research Design for the Aligning Forces for Quality Initiative
Dennis P. Scanlon, PhD; Laura J. Wolf, MSW; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Megan McHugh, PhD; Yunfeng Shi, PhD; Brigitt Leitze
Participating Faculty
Letter From Donald M. Berwick, MD, MPP, Guest Editor
Donald M. Berwick, MD, MPP
The View From Aligning Forces to a Culture of Health
Carolyn E. Miller, MSHP, MA, and Anne F. Weiss, MPP
Leading Multi-sector Collaboration: Lessons From the Aligning Forces for Quality National Program Office
Katherine O. Browne, MBA, MHA; Robert Graham, MD; and Bruce Siegel, MD, MPH
Healthcare Reform Post AF4Q: A National Network of Regional Collaboratives Continues Healthcare Reform From the Ground Up
Elizabeth Mitchell and Dianne Hasselman, MSPH

Improving Care Delivery at the Community Level: An Examination of the AF4Q Legacy

Megan McHugh, PhD; Jillian B. Harvey, MPH, PhD; Jaime Hamil, MPH; and Dennis P. Scanlon, PhD
Objective: Aligning Forces for Quality (AF4Q) was the Robert Wood Johnson Foundation’s nearly 10-year, multicomponent initiative to create meaningful and sustainable change in 16 communities. Our purpose was to describe the likely legacy of the care delivery component of AF4Q among participating communities and the factors that influenced the legacy.

Methods: We used a multiple-case study approach. Our analysis relied on 3 key documents for each community, based on key informant interviews conducted between 2006 and 2015: (1) a summary of the community’s care delivery activities under AF4Q, (2) a summary of the community’s experience in the AF4Q program, and (3) a summary of the characteristics of each community and the multi-stakeholder alliance that led local efforts under AF4Q. We used a team-based consensual approach to analysis.

Results: We identified 3 types of legacies: (1) in 3 communities, there appear to be sustained infrastructures or wide-reaching activities attributable to AF4Q; (2) in 5 communities, AF4Q participation was used to advance preexisting activities; and (3) in 8 communities, the care delivery legacy is likely to be limited, because the local alliance focused on performance measurement instead of care delivery or the care delivery activities had limited reach and sustainability. Community contextual factors (eg, availability of other grant support) and alliance characteristics (eg, areas of expertise) greatly influenced the AF4Q care delivery legacy.

Conclusion: AF4Q appears to have created meaningful and sustained change in care delivery in half of the participating communities. Among the other communities, the considerable financial support and technical assistance provided by RWJF was not enough to overcome some of the contextual barriers that often hamper quality-improvement efforts.

Am J Manag Care. 2016;22:S393-S402
“Helping health professionals get better at improving care” was one of the long-standing pillars of the Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) program, RWJF’s signature effort to help 16 diverse communities improve the strength, resilience, and quality of their healthcare systems.1 When pursued alongside performance measurement, public reporting, and consumer engagement, RWJF posited that improved care delivery would lead to meaningful and sustainable change within communities. To that end, RWJF provided multi-stakeholder alliances—collaborative groups of payers, purchasers, providers, and consumers—with funding and technical assistance to support improvements in care delivery. Alliances were given a fair amount of discretion in terms of how to pursue improved care delivery within their communities, but they were expected to meet certain requirements that evolved during the course of the program and report on their progress. The program was in operation from 2006 to 2015.2

RWJF’s multi-stakeholder approach under the AF4Q program followed the recognition that siloed, organizational-level attempts to improve quality resulted in only modest change3-5 and was aligned with statements from prominent healthcare institutions and leaders that multi-stakeholder approaches may be superior.6-8 Although conceptually appealing, evidence of the effectiveness of multi-stakeholder–led quality improvements (QIs) was (and remains) limited. A logic model, developed by the AF4Q evaluation team, depicts assumptions about how multi-stakeholder alliances under the AF4Q program could drive improvements in care delivery and health outcomes (Figure). Specifically, after assessing the needs of the community and developing a QI vision or strategy, the AF4Q alliance would create or build upon the QI infrastructure within the community (eg, procure and distribute QI resources, raise awareness about the need for QIs), leveraging the technical assistance and funding provided by RWJF. The infrastructure would provide a platform from which QI activities could be created or enhanced, aligned with AF4Q’s other programmatic areas (eg, consumer engagement, performance measurement), and spread throughout the community. The model also reflects that the alliances vary significantly in terms of history and market structure, and are influenced by factors in the external environment not directly related to the AF4Q initiative.

As part of the evaluation of AF4Q, we tracked alliances’ efforts to improve care delivery (herein referred to as QI) communitywide and reported our findings in a number of publications (Sidebar).9-17 In sum, we found that although alliances were slow to establish a QI infrastructure and launch QI activities, all alliances eventually implemented QI activities by establishing the activities themselves or delegating the task to close partners. There was considerable overlap in the focus of the QI activities across alliances (eg, most alliances encouraged adoption of patient-centered medical home [PCMH] processes) and the approaches employed (eg, most alliances established learning collaboratives). However, the quantity and quality of the activities pursued, or the “dose” of the AF4Q program QI interventions, varied considerably across communities. As a result, for a majority of patient care and patient outcomes prespecified at the start of the program and tracked under the evaluation, AF4Q communities did not experience greater improvement than non-AF4Q communities.

Given the large investments in multi-stakeholder alliance–led initiatives to improve quality in the public and private sectors,18 our goal in this paper was to provide additional insight about the AF4Q program’s QI legacy in the participating communities—insights that would be overlooked if one focused solely on the underwhelming quantitative results. Specifically, based on our qualitative data, we sought to describe the legacy that the QI component of the AF4Q program is likely to leave in each participating community and the factors that influenced the AF4Q program’s QI legacy. Findings from our analyses may provide RWJF staff and policy makers with insight about the cross-community results of RWJF’s investment in the area of QI and guide future efforts.



We used a multiple-case study approach to examine the AF4Q program’s QI intervention within its real-life context across the 16 participating communities.19 Although multiple-case studies typically include fewer than 5 cases due to the complexity of the data,20 our analysis included all 16 AF4Q participating regions, as this analysis represents a component of the summative evaluation of the AF4Q program.

Data Collection and Coding

We conducted 1100 semi-structured key informant interviews during 4 rounds of site visits to AF4Q communities between 2006 and 2016, and 10 rounds of telephone interviews with alliance leaders between 2007 and 2014. Interviews were conducted with a number of individuals in each community during the site visits, including alliance directors, who oversee the work of the alliance; project directors, who are responsible for implementation of the AF4Q initiative; individuals who led or planned alliances’ activities in various programmatic areas (eg, QI, public reporting, and consumer engagement); and representatives from each of the targeted community stakeholder groups, typically consumers, physicians, hospital leaders, healthcare plans, and employers.21 The telephone interviews were typically conducted with 1 or more of the following individuals: the alliance director, project director, or individual identified by the alliance or project director as being responsible for the alliance’s QI activities (“QI leaders”).

During the interviews, we asked respondents about a number of topics, including the alliances’ structure, vision, goals, strategies, and decision making; details about specific QI activities; characteristics of the alliances’ markets and partnerships; and external factors that affected their decisions and activities. All interviews were audio-recorded, transcribed, and uploaded to Atlas.ti, a qualitative software package, for analysis.

Using a multistaged coding process, we first used deductive high-level (global) categories corresponding to the AF4Q initiative’s main programmatic areas and major concepts that are relevant across all communities (eg, alliance participation, resources, and structure). Next, all data that were globally coded were read for emerging themes. The transcripts were reviewed until no new themes emerged. This inductive process resulted in a final list of codes representing the key concepts and themes related to the QI programs. A more thorough description of our interviews and coding process can be found elsewhere.21


Because of the large number of cases, our analysis relies on 3 key documents created for each AF4Q community, based on the interview data. First, we created a 2- to 3-page summary of the communities’ major QI activities under the AF4Q program. In 2013, the QI leaders verified and/or made corrections to our summaries. Second, we created a more comprehensive (14-20 page) summary of each community’s experience in the AF4Q program, which included a description of program governance, activities in the various programmatic areas, and contextual factors that influenced their activities, challenges, and successes. Third, we created a document that lists characteristics of each alliance (eg, the alliances’ history and origin, stakeholder dominance, and area[s] of expertise prior to joining the AF4Q program) and community (eg, health provider market competition and community size).

We used a team-based consensual approach to analysis.22,23 To ensure that the influence of the unique context of each case was sufficiently considered, each case was examined independently before any attempt was made to triangulate findings across cases. For each community, 2 authors reviewed the 3 key documents and drafted memos addressing 2 questions: (1) What is likely to be the legacy of the AF4Q program regarding improved care delivery? and (2) What factors influenced that legacy?

Patterns and themes across sites were identified through 4 weekly discussions. In several instances, the authors reviewed the interview transcripts to resolve disagreement, identify illustrative quotes, and further explore contextual factors that may have contributed to differences in legacy across AF4Q communities. Finally, we provided a profile of each of the AF4Q communities, including alliance and community characteristics, a measure of the “dose” of the AF4Q program’s QI intervention, and the percentage of patient experience measures that improved between 2008 and 2012.


Our results revealed 3 different types of legacies that the AF4Q program is likely to leave across participating communities. We describe these legacies, highlighting specific AF4Q communities as examples, and the factors that may have contributed to those legacies. A profile of the communities can be found in Table 1.

New Infrastructure Legacy

In 3 communities, the AF4Q program’s QI legacy is a new infrastructure or advancement in care that has a broad reach in the community, and its existence or spread can be attributed specifically to the AF4Q program. Although the long-term sustainability of these infrastructures and advancements is uncertain, the alliances continued to engage in the QI work beyond the end of the AF4Q grant.

Under the AF4Q program, the alliance in Cincinnati began a series of multipayer PCMH pilot programs involving approximately 10 physician practices each. Partnering with TransforMED, an affiliate of the American Academy of Family Physicians, to build the curriculum, the alliance led the pilot program, recruited health plans to contribute a $20 per-member-per-month payment to participating practices, and created a multi-payer database to evaluate the pilots. This work was largely the impetus for the region being selected to participate in the Centers for Medicare & Medicaid Services’ (CMS) Comprehensive Primary Care initiative (CPCI), which expanded the number of payers and practices involved. This work, in turn, laid the groundwork for Ohio’s State Innovation Model award, which also focused on PCMH adoption. One respondent noted: “I see the evolution from Aligning Forces to CPCI to a statewide model for PCMH as one continuous flow.”

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up