Currently Viewing:
Supplements The Aligning Forces for Quality Initiative: Summative Findings and Lessons Learned From Efforts to Improve Healthcare Quality at the Community Level
The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2015
Dennis P. Scanlon, PhD; Jeff Beich, PhD; Brigitt Leitzell, MS; Bethany W. Shaw, MHA; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Diane C. Farley, BA; Jessica Greene, PhD; Muriel Jean-Jacques,
Summative Evaluation Results and Lessons Learned From the Aligning Forces for Quality Program
Dennis P. Scanlon, PhD; Jeffrey A. Alexander, PhD; Megan McHugh, PhD; Jeff Beich, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Brigitt Leitzell, MS; Yunfeng Shi,
The Longitudinal Impact of Aligning Forces for Quality on Measures of Population Health, Quality and Experience of Care, and Cost of Care
Yunfeng Shi, PhD; Dennis P. Scanlon, PhD; Raymond Kang, MA; Megan McHugh, PhD; Jessica Greene, PhD; Jon B. Christianson, PhD; Muriel Jean-Jacques, MD, MAPP; Yasmin Mahmud, MPH; and Jeffrey A. Alexande
Reporting Provider Performance: What Can Be Learned From the Experience of Multi-Stakeholder Community Coalitions?
Jon B. Christianson, PhD; Bethany W. Shaw, MHA; Jessica Greene, PhD; and Dennis P. Scanlon, PhD
Improving Care Delivery at the Community Level: An Examination of the AF4Q Legacy
Megan McHugh, PhD; Jillian B. Harvey, MPH, PhD; Jaime Hamil, MPH; and Dennis P. Scanlon, PhD
Currently Reading
From Rhetoric to Reality: Consumer Engagement in 16 Multi-Stakeholder Alliances
Jessica Greene, PhD; Diane C. Farley, BA; Jon B. Christianson, PhD; Dennis P. Scanlon, PhD; and Yunfeng Shi, PhD
Aligning Forces for Quality Multi-Stakeholder Healthcare Alliances: Do They Have a Sustainable Future
Jeffrey A. Alexander, PhD; Larry R. Hearld, PhD; Laura J. Wolf, MSW; and Jocelyn M. Vanderbrink, MHA
Evaluating a Complex, Multi-Site, Community-Based Program to Improve Healthcare Quality: The Summative Research Design for the Aligning Forces for Quality Initiative
Dennis P. Scanlon, PhD; Laura J. Wolf, MSW; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Jessica Greene, PhD; Muriel Jean-Jacques, MD, MAPP; Megan McHugh, PhD; Yunfeng Shi, PhD; Brigitt Leitze
Participating Faculty
Letter From Donald M. Berwick, MD, MPP, Guest Editor
Donald M. Berwick, MD, MPP
The View From Aligning Forces to a Culture of Health
Carolyn E. Miller, MSHP, MA, and Anne F. Weiss, MPP
Leading Multi-sector Collaboration: Lessons From the Aligning Forces for Quality National Program Office
Katherine O. Browne, MBA, MHA; Robert Graham, MD; and Bruce Siegel, MD, MPH
Healthcare Reform Post AF4Q: A National Network of Regional Collaboratives Continues Healthcare Reform From the Ground Up
Elizabeth Mitchell and Dianne Hasselman, MSPH

From Rhetoric to Reality: Consumer Engagement in 16 Multi-Stakeholder Alliances

Jessica Greene, PhD; Diane C. Farley, BA; Jon B. Christianson, PhD; Dennis P. Scanlon, PhD; and Yunfeng Shi, PhD
Objective: A key component of the Aligning Forces for Quality (AF4Q) program was engaging consumers in their health and healthcare. We examined the extent to which the alliances embraced 4 areas of consumer engagement: self-management, consumer friendliness of reports of healthcare provider quality, involvement of consumers in alliance governance, and the integration of consumers into quality improvement teams.

Methods: We used a largely qualitative approach. The evaluation team conducted 1100 in-depth interviews with alliance stakeholders. Two authors reviewed the consumer engagement data for each alliance to assess its level of embrace in the 4 consumer engagement areas. For consumer friendliness of public reporting websites, we also assessed alliance public reports for reading level, technical language, and evaluable displays. Population-level effects were also examined for self-management and public reporting.

Results: Consumer engagement was new to most alliances, and few had staff with consumer engagement expertise or existing consumer constituencies. For each area of consumer engagement, some alliances enthusiastically embraced the work, other alliances made a concerted but limited effort to develop programs, and a third group of alliances did the minimum work required. Integrating consumers into governance was the area most often embraced, followed by making public reports consumer friendly. Two alliances strongly embraced both self-management and integrating patients into quality improvement efforts. The AF4Q program did not have greater population level effects from self-management or public reporting than were those observed in a national comparison sample.

Conclusion: The AF4Q program sparked a few alliances to develop robust consumer engagement programming, while most alliances tried consumer engagement efforts for the first time and developed an appreciation for integrating consumer perspectives into their work.

Am J Manag Care. 2016;22:S403-S412
The concept of consumer engagement has received a lot of attention in recent years, being described as “the blockbuster drug of the century” and a key factor for achieving the Triple Aim.1,2 The idea that consumers can help improve the efficiency of healthcare delivery by being more involved in their health and making more informed choices about treatments and providers has widespread, bipartisan appeal.3,4 However, what exactly is meant by the term has been questioned by many, including Munro who wrote: “How we define it will determine whether it’s truly a miracle drug—or just another variant of age-old snake oil.”5-7

Because of the ambiguity in definition, 2 recent papers have sought to capture consumer engagement’s dimensions in a conceptual framework.8,9 Carman and colleagues, who served as technical assistance providers for the Aligning Forces for Quality (AF4Q) program, described 3 levels at which patients can be engaged: the direct care level, in which individuals become engaged in their own health and healthcare; the level of organizational design and governance for healthcare organizations; and finally, the policy-making level, when consumers are involved with federal, state, and local healthcare policy.8 At each level, the consumer role is viewed on a continuum, from consultation with consumers at the low end to partnership and shared leadership at the high end.

Mittler and colleagues, who were part of the AF4Q program evaluation team, further differentiated engagement activities at the direct-care level into 4 components.9 Two relate to health behaviors: self-management behaviors, which focus on management of chronic disease, and healthy behaviors, which are general health-promoting behaviors like healthy eating and engaging in physical activity. The other 2 relate to managing healthcare: healthcare encounter behaviors relate to effectively communicating with healthcare providers and shopping behaviors include making informed choices to select high-quality (or high-value) healthcare providers and treatments.

This article examines how consumer engagement was implemented in the AF4Q program’s 16 multi-stakeholder coalitions (alliances). Specifically, we examine the approaches to implementing efforts in the 4 key areas of consumer engagement in the AF4Q program, which include examples of consumer engagement at the direct-care and healthcare organization governance levels, but not at the policy-making level. At the direct-care level, we examined efforts to improve (1) self-management and (2) shopping; at the healthcare organizational level, we examined efforts to involve individual consumers in (3) alliance governance and (4) healthcare quality improvement teams. The first 3 areas were required as part of the AF4Q program, while the fourth was voluntarily developed by several alliances. We examined how and to what degree the alliances embraced each of these 4 areas of consumer engagement, the factors related to the level of embrace of work in the areas of consumer engagement, and, for the direct-care consumer engagement efforts, we also examined the extent to which the interventions had population-level effects.


With the AF4Q initiative, the Robert Wood Johnson Foundation (RWJF) was early to embrace the concept of consumer engagement. The AF4Q program was premised on the idea that to improve the quality of patient care, consumers needed to be involved with providers and purchasers. Consumer engagement was seen as one of the key drivers for improving healthcare quality in the United States, along with performance measurement, public reporting, and quality improvement.10,11

Although RWJF viewed consumer engagement as essential to the AF4Q program, what exactly consumer engagement would look like within the initiative was less clear.12 The initial call for proposals required “substantial and credible” consumer representation in the alliance leadership.10 It also emphasized consumers’ use of information to make care decisions. To clarify how that would be operationalized, early in the AF4Q program, RWJF created a consumer engagement learning community in which alliances met, received technical assistance, and shared ideas. As part of the learning community, each alliance was expected to develop plans in 2 areas of direct-care consumer engagement: self-management and consumer use of public reports of provider quality.7 RWJF staff later wrote, “We let a thousand flowers bloom, believing that there was too little evidence about what works in consumer engagement to dictate one approach.”13

The AF4Q alliances’ early experience with consumer engagement was challenging. According to early evaluation findings, “Developing a concerted, coherent consumer engagement strategy has taken much more time than originally anticipated in virtually all of the sites.”7 The challenges were attributed to differing levels of enthusiasm for the work within alliances, a lack of existing evidence-based strategies, and difficulty reaching consensus on strategies within alliances.

In response to the challenges, RWJF began reworking its approach to consumer engagement in late 2008. Instead of supporting a “wide breadth of activities,” the Foundation decided to provide alliances with more structure and narrower consumer engagement expectations.14 In March 2009, the Foundation sent the alliances a memo clarifying that the principal goal for consumer engagement was raising consumers’ awareness and use of comparative healthcare performance information.14 Specifically, alliances were tasked with providing consumer friendly public reports of healthcare provider performance information and using a range of strategies to connect consumers to access and use the information to facilitate consumer shopping. The memo also reiterated the Foundation’s desire for consumer involvement in alliance governance. It further explained that the new consumer engagement expectations could be viewed as a “floor of activities” and not the entirety of what alliances could do. Thus, alliances could continue to implement programs in self-management or other areas of consumer engagement as long as they also disseminated consumer-friendly public reports on provider quality and involved consumers in governance.

Soon after the memo’s release, the alliances were required to report specifically on metrics related to the 2 main consumer engagement goals. For shopping, alliances had to report on the number of visits and page views for their online public reports of ambulatory provider quality performance. For integrating consumers into alliance governance, alliances had to document that at least 1 consumer (not a consumer advocate) was represented on the alliance’s leadership team or workgroup. As the grant reporting requirements became more process-oriented, the alliances were asked to describe major activities (eg,“to promote consumers’ use of health and comparative performance information in healthcare decision making” and “engaging individual consumers and consumer advocates in the work of [the] AF4Q [program]”).


We used a qualitative approach to examine the alliances’ experiences implementing programs in the 4 areas of consumer engagement. In addition, for consumer shopping for high-quality healthcare providers, we assessed the consumer friendliness of the alliances’ public reporting websites.

For the direct-care levels of consumer engagement, self-management and shopping, we also examined population-level changes over 4 early years of the AF4Q program (2007-2008 and 2011-2012) using 2 rounds of the AF4Q Consumer Survey. (For New Mexico and Boston, which joined the AF4Q program later, the round 1 survey was conducted in 2010 and the round 2 survey in 2013/2014.) Details of the data, analysis, and results are presented in the online eAppendix. It is notable that a third round of the consumer survey had been part of the evaluation team’s research design, but it was not, in the end, funded. Without a third round, the analysis of population-level effects was limited to approximately the first half of the AF4Q initiative. However, our qualitative results of the AF4Q program’s self-management efforts do not suggest that there would be a greater impact on patient activation later in the program. Similarly, we did not observe a substantial change in consumer friendliness of the AF4Q public reporting websites after 2012, as is detailed in the results section later in this paper.

Qualitative Component

As part of the AF4Q program evaluation, the evaluation team conducted 1100 qualitative interviews with alliance stakeholders over the program’s almost 10-year history. The qualitative interviews were conducted through a combination of periodic in-person site visits and annual or semi-annual telephone interviews. In each round of interviews, alliance stakeholders were asked about their consumer engagement work, and there were 2 rounds of interviews that largely focused on consumer engagement.

The interviews were audio-recorded, transcribed, and coded in Atlas.ti for the key AF4Q programmatic areas (eg, consumer engagement, quality improvement, and public reporting). For each alliance, data related to consumer engagement were retrieved and reviewed by 2 authors, and a detailed summary of how the alliance approached the 4 areas of consumer engagement (and any additional areas) was developed. This summary included qualitative data on alliance stakeholders’ attitudes and perceptions of the various areas of consumer engagement. The 2 authors then characterized each alliance’s level of embrace for each of the 4 areas of consumer engagement, as: (1) high embrace, in which the alliance made the work a key priority; (2) medium embrace, in which alliances made a concerted effort to develop programming, but in a more limited way; or (3) low embrace, in which alliances did the minimum required or “checked the box” in that area. We also identified themes that covered all 4 areas of consumer engagement related to developing consumer engagement programming within the AF4Q initiative.

Assessment of the Consumer Friendliness of Public Reports

To assess the consumer friendliness of the alliances’ public reporting websites, we examined the diabetes quality metrics for ambulatory care at 3 time points: (1) 2010, summer; (2) 2013, summer; and (3) 2015, fall. Diabetes metrics were selected because they were most consistently publicly reported by alliances.15 We included the 14 alliances that reported ambulatory diabetes metrics in each of the 3 time periods (excluding West Michigan and Western New York, which last reported in 2011 and 2014, respectively). We analyzed Wisconsin’s consumer-oriented public report rather than the alliance’s main reporting website.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up