Dennis P. Scanlon, PhD; Jeff Beich, PhD; Jeffrey A. Alexander, PhD; Jon B. Christianson, PhD; Romana Hasnain-Wynia, PhD; Megan C. McHugh, PhD; and Jessica N. Mittler, PhD
There is considerable interest in using multi-stakeholder alliances and regional coalitions to coordinate quality improvement (QI) efforts across providers and organizations.1,2
Proponents of this approach hypothesize that the coordinated efforts of health plans, purchasers, providers, and consumers will yield better and more sustainable outcomes than activities undertaken independently.3
The Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) initiative is the largest privately funded community-based QI initiative to date, with an expected investment of more than $300 million over the life of the project.4
“The premise of Aligning Forces is that no single person, group or profession can improve health and healthcare throughout a community without the support of others. Aligning Forces for Quality seeks to drive QI by aligning key players in local communities.”5
In 2006, the RWJF began providing grants and technical assistance (TA) to 4 alliances (multi-stakeholder partnerships in each AF4Q community), launching a program that expanded over time and now includes 16 communities across the country. Funding for the alliances is expected to continue through 2015. The AF4Q alliances are not-for-profit community organizations, either preexisting or established specifically for the AF4Q grant, with representatives from the payer, provider, and consumer sectors. While the RWJF plays an active role in strategy development and program oversight, they have delegated the day-to-day program implementation to a National Program Office (NPO). The authors of this article are a team of investigators from Penn State University, the University of Michigan, the University of Minnesota, and Northwestern University contracted by the RWJF to conduct an independent program evaluation.
The AF4Q initiative is a complex and dynamic program, addressing multiple leverage points in the healthcare system. The overarching goal for the program is summarized by the RWJF as follows: “The [AF4Q] program works to improve healthcare by engaging patients in their care, publicly reporting the performance of physicians and hospitals, and improving the quality of care delivered in each community. The Foundation hopes to provide models that will help propel national reform by providing resources, expertise, and training to help providers, payers and consumers all do their part.”4
Rather than being a fully developed program at its inception, the AF4Q initiative has expanded its scope and has made several mid-course adjustments as the program has evolved. In this article, we describe the program’s evolution and the range of AF4Q-related activities undertaken by participating alliances. Information for this article was obtained through program document review, meeting participation, observation of alliance activities, and interviews with RWJF staff, TA providers, and alliance stakeholders. Readers interested in additional information about our research design and data sources may refer to the article by Scanlon et al in this supplement.6
We begin by describing the theory of change underlying the initiative, depicted graphically in a logic model. Subsequently, we describe the evolution of the program from its inception through 3 distinct phases, covering programmatic expectations for the alliances, as well as the guidance and TA provided to support their efforts. Finally, we summarize key activities undertaken by the alliances in each of the programmatic areas that comprise the AF4Q initiative.Theory of Change: The AF4Q Logic Model
A key step in program evaluation is to articulate the initiative’s theory of change—that is, the underlying assumptions and expectations regarding how program interventions will lead to the expected outcomes, and in what time frame. The Figure
depicts a logic model, developed by our evaluation team with input from RWJF staff, the NPO, TA providers, and key alliance stakeholders. “A logic model helps to focus an evaluation by making a program’s assumptions and expectations explicit, and increases stakeholders’ understanding about the program or initiative.”7
As depicted on the right side of the model, the objective of the AF4Q initiative is improvement in key community and population health outcomes such as health status and quality of care received; these outcomes are broad and ambitious, and are envisioned to take time to realize. More proximate outcomes, such as increased transparency about provider quality and price, and improved care coordination, are depicted as intermediate outcomes. The left-hand side of the Figure illustrates how the program is envisioned to achieve these outcomes. Specifically, within the community, the AF4Q initiative starts with a multi-stakeholder community alliance, either new or preexisting. The alliance is responsible for establishing a leadership team and organizational structure to support program activities. Leadership is responsible for formulating the alliance’s vision and QI strategy within their community. In order to achieve this vision, the alliance develops and implements interventions, which are activities targeted at facilitating changes in the programmatic areas germane to the AF4Q initiative. They may sponsor these activities directly, or collaborate with other community organizations.
At a minimum, the alliances must address 5 main programmatic areas: (1) measurement and public reporting of healthcare quality and efficiency for ambulatory physician practices and hospitals; (2) efforts to engage consumers as partners in their care (consumer engagement); (3) the adoption and spread of effective QI strategies to improve care; (4) ensuring the equitable receipt of healthcare; and (5) integration of alliance activities with payment reform initiatives. In addition to aligning stakeholders around a common vision, the AF4Q initiative targets alignment of programmatic areas, depicted by connectors in the interventions box.
Across the top of the Figure, we indicated that the RWJF provides TA through multiple organizations or individuals with expertise in key programmatic areas to assist the alliances in strategy development and implementation. The model also reflects that the alliances vary significantly in terms of history and market structure and are influenced by factors in the external environment not directly related to the AF4Q initiative. Since the RWJF’s objective is to sustain the alliances’ activities beyond the anticipated conclusion of the grants in 2015, an important long-term program goal is to build collaborative capacity within the community. This may be accomplished through continuation of the alliance or through alternative models. Finally, as noted on the bottom of the diagram, the alliances’ activities and the impact of the AF4Q initiative are expected to evolve over a period of time, with necessary adjustments based on feedback from experiences in program implementation.
While the logic model provides a succinct view of the overall program, it is not sufficiently detailed to guide our evaluation. Accordingly, our team also developed individual models for each programmatic area. This has proved to be a challenging task. For example, the literature does not include a consistent definition of consumer engagement, nor do the existing conceptual models address the array of consumer engagement approaches considered in the AF4Q initiative.8 Consequently, we developed a consumer engagement framework targeting 4 categories of behavior: (1) healthy behaviors—the activities individuals perform to maintain health and prevent illness; (2) self-management behaviors—daily activities performed to control or reduce the impact of chronic illness on health status; (3) shopping behaviors—actions targeted at becoming more effective purchasers and consumers of healthcare; and (4) healthcare encounter behaviors—activities undertaken to become more effective self-advocates when interacting with healthcare providers. While not discussed in detail here, our consumer engagement framework illustrates the dynamic nature of behavior change and considers its relationship to individual, group, and community characteristics, including patient activation and the 4 engaged behavior categories. This framework and the other programmatic logic models are available in the online eAppendix
. More information about our consumer engagement framework appears in the article by Mittler et al.8Evolution of the AF4Q Initiative
Under the AF4Q initiative, the RWJF provides funding and TA to participants; in turn, the alliances are expected to meet specified goals and objectives. While the program’s initial scope was substantial, it has expanded through enhancements to existing programmatic areas and the addition of new ones. Since the alliances have been charged with a formidable task, the RWJF has made a significant commitment to the provision of TA: it is estimated that TA for each alliance will exceed several million dollars. In this section, we describe the evolution of the initiative over 3 distinct phases, including the program’s scope, goals, and expectations, and TA offerings.Phase I
The overarching goal of the initiative’s first phase was to help communities substantially improve the quality of healthcare provided in ambulatory care settings for persons with chronic diseases. Phase I targeted 3 programmatic areas believed to be key drivers of quality (depicted in the interventions box of the logic model): (1) performance measurement and public reporting of performance data; (2) QI in primary care physician practices; and (3) consumer engagement. The RWJF chose to invite 4 communities with a history of stakeholder collaboration on healthcare quality—Detroit, Michigan; Memphis, Tennessee; Minnesota; and Puget Sound, Washington—to serve as the initial communities for the program. Funding began in July 2006 and the Center for Health Improvement (Sacramento, CA) was selected as the NPO. An additional 10 communities (Cincinnati, Ohio; Cleveland, Ohio; Humboldt County, California; Kansas City, Missouri/Kansas; Maine; south central Pennsylvania; West Michigan; Western New York; Willamette Valley, Oregon; and Wisconsin) were added in February 2007 through a competitive grant process. A complete listing of the alliances and their websites is provided in Table 1
Alliances were to publicly report ambulatory care performance information for the community’s primary care providers by the end of 3 years, using local multi-payer data and nationally endorsed quality measures. They were given a road goal for consumer engagement, which was to transform the physician-patient relationship, in part, by helping patients become better informed and more activated. Alliances were directed to make “…substantial progress toward engaging community consumers to demand improved ambulatory, chronic illness care quality, including better public information about that care; and motivating those consumers to act in that publicly reported care information.”9 Rather than offering a specific model for consumer engagement, the RWJF chose to encourage the alliances to innovate and “Let a thousand flowers bloom.” In addition, the alliances were expected to have substantial and credible consumer representation on their leadership teams. While the RWJF identified QI as a core component of the initiative, no formal requirements were established in this phase. However, alliances were encouraged to engage in activities associated with patient-centered medical homes (PCMHs) and other ambulatory QI programs.
Early TA was built around webinars on topics related to the core programmatic areas, but mainly focused on performance measurement and public reporting. All 14 AF4Q alliances became part of the Consumer Engagement Learning Collaborative (CELC), the primary vehicle through which consumer engagement TA was provided. The CELC provided a structured framework, using a combination of TA consultants and meetings of the alliances, to assist in the development of consumer engagement strategies.10
The RWJF and the NPO also began sponsoring semiannual national meetings of the AF4Q alliances, providing educational sessions and workshops on programmatic areas, as well as offering opportunities for shared learning among the alliances. Specific progress measures for the alliances were limited during this phase; instead, each alliance developed its own work plans and goals based on general guidance from the RWJF and the Center for Health Improvement.Phase II
In May 2008, all 14 alliances were funded for the next phase of the program. During this phase, the RWJF more clearly defined its expectations for the alliances in the programmatic areas established in phase I. It also added new programmatic areas: (1) assisting hospitals and other inpatient healthcare facilities with improving quality; (2) focusing and strengthening the role of nurse leaders and frontline nurses in QI initiatives; and (3) using performance measures to capture patient experience of care and reduce gaps in quality for patients of different race, ethnicity, or primary language spoken (REL). During phase II, George Washington University’s Center for Health Care Quality became the new NPO. Other changes instituted during this phase included the introduction of systems for measuring and reporting alliance progress on each programmatic area, and reorganization and expansion of the TA program. Three additional communities—Albuquerque, New Mexico; Central Indiana; and Boston, Massachusetts—were added during this phase (see Table 1 for information about these alliances and their website addresses).
During phase II, goals for consumer engagement became more specific, focusing on activities to facilitate consumer use of performance reports for making healthcare decisions and tailoring alliance websites, the primary medium for reporting this information, to be more consumer-friendly. Performance measurement and public reporting requirements were expanded to include reporting of nationally recognized measures of quality, patient experience, efficiency, and prices for hospital inpatients. Alliances also had to demonstrate a plan for achieving the standardized collection of self-reported data about patients’ REL for all healthcare providers, and integrating this information into their measurement, reporting, and QI activities.
To address the new inpatient QI expectations, the NPO and TA providers established 3 QI learning collaboratives open to hospitals in AF4Q communities. The collaboratives included (1) Transforming Care at the Bedside, targeting development of patient-centered care in nursing units; (2) the Equity Quality Improvement Collaborative, focusing on improving the quality of care delivered to cardiac patients, while reducing racial and ethnic disparities; and (3) the Language Quality Improvement Collaborative, which aimed to improve care to non-English-speaking patients. The collaboratives employed a combination of in-person meetings, webinars, and monthly conference calls. Alliances were expected to assist with recruiting hospitals within their communities for participation in the collaboratives. Subsequently, in 2010, the collaboratives were replaced by 3 peer-to-peer hospital quality networks addressing the following topics: reducing readmissions, increasing patient throughput, and improving language services.
Relative to QI in the ambulatory sector, alliances were required to inventory regional QI needs and resources and incorporate this information into plans for sustainable ambulatory QI infrastructures in their communities.11 The NPO also established 2 ambulatory peer-to-peer quality networks in 2010; one was focused on the PCMH, and the other concentrated on implementing a regional learning collaborative. Both the inpatient and ambulatory peer-to-peer networks afforded alliances opportunities to share and learn from each other’s experiences and provided access to online resources and tool kits, consultation with QI leaders, and data analysis and feedback.
The NPO introduced new systems for measuring and reporting alliance progress in meeting grant expectations for each programmatic area. Starting with 14 indicators in 2009, the list expanded to 33 indicators, reflecting refinement and expansion of the program. Selected examples of indicators in the area of performance measurement and public reporting are listed in Table 2. TA offerings were greatly expanded and the CELC was disbanded by the NPO. Rather than employing the “one-size-fits-all” approach used in phase I, alliances were given latitude to select from a wide-ranging list of AF4Q-sponsored TA providers covering the main programmatic areas, alliance governance issues, and communication strategies. TA was delivered through a combination of webinars, telephone conference calls, learning collaboratives, workshops, special reports, and direct consulting with TA vendors. Topics addressed in AF4Q programmatic areas included making the alliances’ websites more consumer-friendly; physician benchmarking and attribution; capturing and reporting patient experience data; communicating with physicians about performance measurement; standardized collection of self-reported data on patient REL; and the use of consumer decision points in healthcare. Governance topics focused primarily on alliance leadership development and formulating strategies to sustain alliance activities beyond the conclusion of the grant.Phase III
In 2010, all 17 AF4Q alliances applied for funding for the next phase of the project, which began in May 2011 and continues through April 2013; 16 alliances were selected to continue with the program. The Central Indiana alliance was not selected, because according to the RWJF, the alliance’s stakeholders were unable to agree on a strategy for publicly reporting their performance data, a mandatory AF4Q requirement. New goals for the alliances included (1) setting and achieving explicit, measurable goals around specific clinical conditions (eg, diabetes, acute myocardial infarction) and sites of care (eg, physician practices, emergency departments); (2) focusing efforts on the selected conditions and sites of care to improve quality, cost, and value; (3) experimenting with payment reform; and (4) leveraging federal, state, and local health information technology (HIT) efforts. The new goals reflected the RWJF’s desire to expand the initiative’s focus beyond quality to include cost and efficiency.12
They also saw the potential of the alliances to capitalize on opportunities stemming from recent legislation, including the Health Information Technology for Economic and Clinical Health Act and the Patient Protection and Affordable Care Act (PPACA).
Alliances were also afforded the opportunity to create direct peer exchanges with fellow grantees. In late 2010, all 8 alliances that applied for the first offering of the peer-to-peer exchange program received funding to visit and learn from other AF4Q alliance leaders. For example, in summer 2011, leaders from the Cleveland alliance visited Maine to learn about the Maine alliance’s success in integrating consumers into their activities.
Finally, AF4Q alliances were also encouraged to take advantage of opportunities associated with healthcare reform legislation. Several alliances applied for and received funding through the Center for Medicare & Medicaid Innovation (CMMI), established through the PPACA. For example, the Cincinnati alliance will participate in the CMMI-sponsored Comprehensive Primary Care initiative. A key objective for this initiative is to establish a system in which Medicare, commercial, and state health insurance plans pay bonuses to primary care doctors who better coordinate patient care.The Future: Phase IV
The final program phase is projected to run from May 2013 to June 2015. A request for proposals has been issued to the 16 participating alliances, but applications are not due until October 2012. Rather than introducing new programmatic areas, the fourth phase will emphasize development of long-term strategies for sustaining the alliances and/or their activities. Alliances will be charged with developing a “strategic plan for sustainable high value care” for their communities that provides specific goals for improved quality, reduced cost, and a reduction in disparities. We also expect to see realization of some of the long-term outcomes identified in the left side of our logic model.AF4Q Programmatic Areas
While we have described the program in theory, what has actually occurred as a result of the AF4Q initiative? In the following sections, we briefly describe the range of activities undertaken by the alliances in each of the main programmatic areas. More detailed descriptions of program activities, as well as their short-term impacts, can be found in other papers in this issue.13-16
In terms of the logic model, the activities described would appear in the interventions box. Results, impact, and lessons learned are described in the article by Alexander et al in this supplement.17Performance Measurement and Public Reporting Activities
From its inception, a central theme of the AF4Q initiative has been transparency through public reporting of healthcare providers’ performance.18 Alliance leaders have noted that performance measurement and reporting has been given more emphasis than other AF4Q programmatic areas, particularly in the early stages of the initiative. According to an alliance leader, “If we look at the dashboard … the real push is the public reporting piece.” Two key objectives for this strategy are to encourage consumers to use performance information in making healthcare decisions, such as selecting a healthcare provider or preparing for a physician visit, and to motivate providers’ QI efforts through comparisons of their performance with peers and other benchmarks.
Prior to the AF4Q initiative, 4 of 16 AF4Q communities were reporting physician quality measures and 3 were reporting inpatient quality measures. To date, all 16 AF4Q alliances have released at least 1 report with physician quality measures, with 15 releasing multiple iterations. Fourteen alliances have released at least 1 report on inpatient quality, with most consisting of reformatted performance measures obtained from the Hospital Compare program. Indicators used for reporting physician quality are predominantly based on the Healthcare Effectiveness Data and Information Set (HEDIS) process measures modified for the ambulatory practice setting. The principal medium for reporting performance information has been the alliances’ websites (website addresses are listed in Table 1), often incorporating an interactive format that allows individuals to search for information by clinic or provider name, location, or other parameters, such as specialty care or system affiliation. Over time, alliances have expanded both the number of conditions and the number of measures in their reports. They have also begun to include measures of patient experience in their reports, with 8 alliances reporting on patient experience in physician office practices.
Data for performance reports have been obtained from aggregated insurance claims across multiple payers and purchasers and directly from providers. While the alliances have relied extensively on claims data, the lack of clinical data (eg, laboratory results) has constrained reporting primarily to process measures. Consequently, many alliances have targeted electronic health record (EHR) information as a better data source. However, limited penetration of EHRs in many of the AF4Q communities has imposed limits on this approach. Some alliances have employed parallel strategies, reporting process measures from claims data for all physicians and reporting outcome measures for physicians with EHRs. Recently, alliances have been working to incorporate cost/ efficiency results into their public reports, such as appropriate use of back pain imaging and generic prescribing rates. Exactly how to measure cost and efficiency by episode, time, clinical condition, and other factors is an area that lacks broad agreement nationally.12
On the inpatient side, 9 alliances are currently reporting measures related to hospital readmissions. All alliances have inventoried their communities’ current approaches to REL data collection. Using this information, alliances are addressing the program’s REL collection and reporting component in different ways. Some alliances are working on the standardized collection of REL data (at the ambulatory or hospital level) whereas others, as a first step, are planning to stratify results based on categories of insurance, using Medicaid as a proxy measure of REL.Consumer Engagement Activities
PDF is available on the last page.