Publication
Peer-Reviewed
Population Health, Equity & Outcomes
Author(s):
This article presents a case study of how stakeholders in one state came together to integrate practice and research that is a step beyond a learning health care system.
ABSTRACT
Objectives: The cost and quality problems of health care in the US have been aggravated by separate silos for research and care delivery that limit the pragmatic value of research questions and delay the implementation and spread of what is learned. We describe a variation on the learning health system concept that engaged various stakeholders in a single state to work together on simultaneous knowledge generation and dissemination built on research questions that arose from the user community.
Study Design: Observational study.
Methods: We identified the 12 strategies used by the leaders of this project to develop and operationalize an observational study in a large sample of primary care clinics in Minnesota that had implemented care coordination as part of attaining certification as health care homes.
Results: The collaboration included the state health department, a research institute embedded in a health system, the 5 main payers, a measurement/reporting organization, 42 care systems with 316 primary care clinics, patient partners, and national consultants. This community developed a research proposal for an observational study about how to improve care coordination in primary care. We describe how this collaborative implemented and disseminated the study findings.
Conclusions: By employing 12 strategies to answer questions that arose from the health care community, we opened a door between the 2 halves of the
house of medicine.
Am J Manag Care. 2025;31(Spec. No. 10):SP691-SP697. https://doi.org/10.37765/ajmc.2025.89795
The US has had problems with the cost and quality of its health care system for many years, highlighted by the Institute of Medicine (now the National Academy of Medicine) in the 2001 Crossing the Quality Chasm report.1,2 There have been similar concerns about our huge clinical trial enterprise’s costs and outcomes, with slow and incomplete translation of an inadequate number of useful research findings into practice.3 In response to these problems, in 2007, the Institute of Medicine’s Roundtable on Evidence-Based Medicine proposed “a wholly different approach to the development and application of evidence for health care.”4 The roundtable named that approach a learning health care system (LHS), which it defined as “designed to generate and apply the best evidence for the collaborative health care choices of each patient and provider; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in health care.”4 In that report and many subsequent papers on the topic, various LHS examples have been described.5-8
However, widespread use of the LHS terminology does not seem to have solved the translation to practice problem, as reflected in a recent special communication from the JAMA Summit on Clinical Trials Participants.9,10 The authors noted that medicine remains “a house divided,” with separate silos for clinical trials and care delivery that seriously compromise medicine’s ability to best serve society. They believe that fundamental problems resulting from this separation are that “RCTs [randomized clinical trials] are often poorly relevant and responsive to the needs of patients and those responsible for care delivery” and “health care delivery systems are often disengaged from clinical trials and fail to rapidly incorporate knowledge derived from RCTs into practice.”9
Hutchinson et al have found that only 26% of clinical trials even have the potential to inform patient care.11 In 2009, the National Academy of Medicine had set a goal that by 2020, “90% of clinical decisions will be supported by accurate, timely, and up-to-date evidence.”12 Yet, as Simon et al observed in 2020, clinicians and patients still lacked high-quality evidence to guide the majority of common consequential decisions.13 There are few studies, much less examples, of how LHSs have fostered research questions more relevant to key stakeholders or facilitated more translation of answers into patient care.14
Farr and Spertus have suggested that there are several challenges to operationalizing the LHS within a single hospital or health system.15 These include a limited ability to include data beyond what is in the electronic health record to better understand the relevant factors and processes, a lack of alignment with other stakeholders in the area (eg, large payers and diverse care systems), and little ability or motivation to spread what has been learned beyond organizational boundaries. They suggest that what is really needed in a world moving to value-based care is a learning health care community, one that leverages regional expertise and includes a wide variety of local stakeholders.
We have recently had experience with such a learning health care community and describe it here so that others can adapt or replicate the approach. This unique collaboration was initiated by Health Care Homes (HCH), a Minnesota Department of Health (MDH) program that certifies primary care clinics as patient-centered medical homes. Care coordination is one of the main standards for certification, but clinics were implementing it in diverse ways due to limited evidence for best practices.16 HCH was deeply interested in better understanding how care coordination works and which approaches are most effective. The community interested in this problem expanded to include a local health system research institute, the 5 main payers in Minnesota, a measurement and reporting organization (MN Community Measurement [MNCM]), 316 primary care clinics from 42 diverse care systems, patient partners, and national consultants. This community conducted an observational study designed to help all participants improve their approach to care coordination, a project supported by a large national funder (Patient-Centered Outcomes Research Institute [PCORI]).
METHODS
This example began in 2018 when the director of the HCH program contacted a clinical researcher (L.I.S.) at a research institute embedded in a local health system with an integrated medical group and health plan.17 She noted that PCORI had released a funding announcement that could support HCH’s interest in learning how clinics could provide the most effective care coordination for their high-cost, high-need patients. Care coordination was 1 of 5 main requirements for certification as a health care home, but it was being implemented by clinics in various ways due to lack of evidence for a best approach. A literature search confirmed this lack of specific usable evidence, despite numerous studies and at least 50 systematic reviews.18-26
Strategies
Bringing together all the participants needed for a multimethod observational study required that we make use of strategies that are not customarily part of research studies, as follows:
Start with care system user problems or questions, not ones suggested by a researcher. A central tenet of the LHS is that research questions need to be generated from and relevant to clinical care.4 In a learning health care community, the same motivations apply: Research will have real-world impact when it addresses real-world problems. These questions are best identified by community stakeholders (especially care systems and patients) in partnership with experienced researchers, rather than by researchers themselves.
Build on existing personal and organizational relationships. Establishing effective collaborations requires aligned goals, complementary roles, and all-important trust.27 Leveraging relationships can establish a strong foundation for partnerships and facilitate nimble solutions to challenges or needs.
Recruit all organizations at the senior leadership level. Having individual partners from within an organization is helpful, but to gain an entire organization as a partner, it is necessary to have support at the highest leadership levels.
Consider innovative incentives to support recruitment. Even when a research question is strongly aligned with real-world clinical challenges and organizational priorities, limited bandwidth and resources can undermine cooperation. Financial incentives are helpful, but innovative incentives that support organizational initiatives or indirectly reduce burdens can be as important.
Seek adequate funding support for all participants, not just for the researchers. Meaningful engagement with community partners requires adequate funding to support participation. One of PCORI’s foundational expectations for partnerships in research is to have “dedicated funds for engagement and partner compensation.”28
Take advantage of existing organizational, legal, and operational structures and expertise. The Health Insurance Portability and Accountability Act, privacy laws, and organizational policies combine to protect patients and their data. However, these regulations can also introduce significant logistical challenges to collecting data across a diverse community of stakeholders. Taking advantage of existing data-sharing relationships and infrastructure can mitigate these logistical challenges.
Incorporate and respect input from diverse stakeholders in decision-making and implementation. True engagement and partnership require listening to stakeholders, acting on their input, and involving them in decision-making (also 1 of PCORI’s 6 foundational expectations).28 Health care professionals and patients should be integrated into project committees and advisory groups.
Precede development of quantitative data collection with qualitative interviews. If stakeholder perceptions and concerns are to be incorporated into data collection, the team will need to use mixed methods and conduct interviews with the relevant stakeholders to modify existing surveys and data collection methods.
Require care systems and clinics to participate in needed data collection. Setting clear expectations about the need for complete data collection—and the rationale—when recruiting care systems and clinics can ensure that essential data are collected at high response rates.
Involve partners and target audience representatives in analysis and dissemination. Effective dissemination of research findings requires that they be understandable and relevant to stakeholders and target audience.29 Achieving that requires stakeholders to be actively engaged in the process.
Encourage participation by stakeholders in publications and presentations. As an extension of strategy 10, participation of stakeholders in publications and public presentations about research findings enhances the findings’ credibility and relevance to the broader community.
Disseminate relevant findings to participants at all levels to foster their use. For major changes in care processes or policies, it is essential that leaders have been engaged and buy into those changes. It is equally important that those who will use the new information understand and want to apply it.
RESULTS
These 12 strategies were all important to the success of the collaboration, keeping it grounded in reality and maintaining participant interest. The 12 strategies were implemented or adapted for this collaboration as follows:
Start with care system user problems or questions.This proposal began when the director at MDH asked a local researcher to help them develop a proposal that would answer their pragmatic questions: What factors contribute to better patient outcomes from care coordination, and does the inclusion of a social worker make a difference?
Build on existing personal and organizational relationships. The original contact was made because the researcher (L.I.S.) had worked with the previous director at MDH-HCH. He was also well known at MNCM and had conducted many studies in local care systems and with some of the payers. He could also draw on personal connections with various national experts as consultants. Both MDH and MNCM have had long-term relationships with each other and with care systems and payers.
Recruit all organizations at the senior leadership level. In Minnesota, most primary care practices are owned by larger care systems that usually contain subspecialists, various resources, and, often, hospitals. Such clinics are usually unable to make independent decisions about participating in external projects or studies, but when care system leaders decide to participate, all their clinics will be included.
Consider innovative incentives to support recruitment. The HCH clinic certification process required a recertification evaluation every 3 years. To make participation more attractive, MDH offered to eliminate the upcoming recertification (not permanently) for clinics that fully participated in this study. Care systems saw this as a significant benefit.
The study was funded just as the COVID-19 pandemic began in 2020, so conversations with care system leaders were used to determine when that turmoil had settled down. We recruited 42 diverse care systems and 316 clinics of the 72 care systems with 415 clinics that were certified as health care homes (64% of eligible care systems and 83% of the eligible clinics). Sixteen care systems were small (1-2 clinics), 15 were medium (3-9 clinics), and 11 were large (10 or more clinics). Nearly half (48%) were in rural areas of the state.
Seek adequate funding support for all participants. With broad input from stakeholders, we applied for funding from PCORI in 2019 and received an award in early 2020 (IHS-2019C1-15625). Although the research institute was the recipient and manager of all grant funds, it provided substantial financial support by contracting for the important roles of MDH and MNCM partners. It also allowed us to reimburse the costs for each payer’s data contributions and provided nominal reimbursements to participating care systems. Study funds also allowed us to compensate 4 patient coinvestigators and provide incentives to patients participating in study interviews, surveys, and focus groups.
Take advantage of existing organizational, legal, and operational structures and expertise. The original plan was to maintain confidentiality of patients by asking participating payers to serve as collectors of clinical, utilization, and survey data and then to anonymize the data before sending it for analysis. However, the payers felt that these tasks would be challenging and duplicative to implement. Instead, we recognized that our partner MNCM already had the technical infrastructure and existing legal and data use agreements with payers and care systems to facilitate data aggregation and transfer. Thus, MNCM took on these roles.
Incorporate and respect input from diverse stakeholders in decision-making and implementation. From the first contact through proposal development and study implementation, all members of this learning community had opportunities for input through membership on study committees and work groups. At least 1 of our 4 patient coinvestigators was on every committee. The HCH program also had a large standing advisory committee of diverse relevant stakeholders, with an update and input gathering at quarterly meetings throughout the study.
Precede development of quantitative data collection with qualitative interviews. Answering our research questions required collecting quantitative data about the specific ways that each clinic was currently conducting care coordination. With no existing questionnaires for this information, we combined established questions with newly created ones. This was greatly facilitated by first interviewing patients and care system staff familiar with the process.
In developing our data collection surveys, we conducted and recorded semistructured interviews with 19 care coordinators, 18 clinicians and clinic leaders, and 32 patients. MDH staff and care system leaders recruited volunteers for interviews, so research staff could simply conduct interviews of people who had volunteered, limiting confidentiality issues.16,30,31 Information from these analyzed interviews was used to develop and conduct surveys of leaders, a care coordinator for each clinic, and patients.
Require care systems and clinics to participate in data collection needed for inclusion in the analysis. To be included in analyses, each care system needed to provide a list of care coordination patients in each clinic, a completed survey of how each care system implemented care coordination overall, and a completed coordination survey for each individual clinic; 100% did so.
Involve partners and target audience representatives in analysis and dissemination. In planning dissemination, we conducted group discussions with 21 steering committee members, 14 care system liaisons, 9 care coordinators, and 11 patients. We asked what they regarded as key points to stimulate these group discussions and surveys to learn their reactions and recommendations. This input was used to plan dissemination.
Encourage participation by stakeholders in publications and presentations. Each publication and most presentations included relevant stakeholders. We used our partners’ existing channels, such as newsletters and websites, and presented findings at both national meetings and annual regional conferences organized by MNCM and MDH.
Disseminate relevant findings to participants at all levels to foster their use. We will send a brief lay summary of lessons learned to the patients who completed our patient survey and provided their contact information. We are also developing separate reports for each payer about the findings for their insured members, a webinar summary of learnings for individual health care professionals, and a news release that could be used by clinics and care systems. We have published 7 papers in peer-reviewed journals and have 4 in review or preparation.16,30-35
To better understand what we learned from this study and to continue this learning health care community, we submitted a proposal for a new grant to conduct a long-term follow-up study of outcomes for the same patients and with the same partners. Nearly all the participating organizations wanted to participate, but unfortunately, it was not funded.
DISCUSSION
This learning health care community developed organically due to public health department questions that were of broad interest throughout the region and fit a particular funder’s priorities. Although those grant funds were necessary to allow all the participants to put staff time into the work, the perceived shared ownership among community members made the collaboration successful during a pandemic-induced stressful time for health care. The participants’ commitment to continue working together to disseminate the results after the grant speaks further to the shared ownership and commitment to turn the learnings into actions.
In their original article about a learning health care community in Kansas City, Missouri, Farr and Spertus focused mainly on implementation/quality improvement guided by a multistakeholder strategic advisory board.15 Their coalition included an academic health center, commercial and public payers, employers, researchers, community-based organizations and foundations, the health department, state hospital associations, 14 hospitals, and patients. They described their key challenges as needing expertise in implementing evidence, having contractual mechanisms to share data, and creating bidirectional engagement at each participating hospital.
Although the Kansas City example engaged a broad spectrum of stakeholders in one community to focus on collaborative quality improvement, our example aimed to reunite the divided house of medicine (as described by the JAMA Summit on Clinical Trials).9 That was possible for us because our region has benefited from a long-standing quality improvement collaborative (the Institute for Clinical Systems Improvement) and HCH learning collaboratives.36 Those experiences largely solved the challenges Kansas City was facing so we could work on learning from our data and making care system changes across the community.
The international Multi-Stakeholder Engagement Consortium identified 10 important factors, including commitment and capacity, communication skills, expertise or experience, and research-relevant values.37 However, their focus was on identifying stakeholder individuals rather than organizations.
Another way of conceptualizing the qualities needed by an LHS was developed as an “action framework” by a Canadian group.38 Built on previous LHS frameworks, they identified 5 key “learning gears” needed by an LHS production engine: (1) advanced analytics and population insights; (2) evidence curation and synthesis; (3) patient, caregiver, and provider codesign; (4) implementation and reach; and (5) rapid cycle evaluation, feedback, and adaptation.
Each community is different enough that no single list can suffice for all, but these strategies will be important for all.
CONCLUSIONS
We recognize limitations in this approach, including that our learning health care community will require continuing financial support to maintain the infrastructure we have built through this first large grant. We will also need to find ways to identify new questions that arise from those who will use the lessons learned so we don’t evolve back into a research/funder-driven association. We also need more time to assess the degree to which participating organizations implement what has been learned.
One promising facilitator is the recent decision by the US Agency for Healthcare Research and Quality to establish a $1 billion, 10-year Healthcare Extension Service for state-based solutions to health care improvement.39 Modeled on the regional extension service of the US Department of Agriculture and public universities, this service would assist states in convening state and community stakeholders to “identify urgent health care goals and align their activities to achieve them”; provide relevant information, strategies, and tools; and train an implementation workforce. If this mechanism or others (see the Collaborative Approach to Public Goods Investments for a self-funding possibility)40 can support learning health care communities such as ours, medicine’s divided house may yet be reunited.
Author Affiliations: HealthPartners Institute (LIS, SPD), Minneapolis, MN; Health Care Homes Program, Minnesota Department of Health (DK), St Paul, MN; MN Community Measurement (EC), Minneapolis, MN; The Dartmouth Institute (GE), Hanover, NH.
Source of Funding: This work was supported through a Patient-Centered Outcomes Research Institute (PCORI) Project Program Award (IHS-2019C1-15625). All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the official views of the PCORI, its Board of Governors, or its Methodology Committee.
Author Disclosures: Dr Solberg was and Dr Dehmer is employed by HealthPartners, a health plan and care system, and received funding for this work from PCORI. Mr Kurtzon is employed by the Minnesota Department of Health and received funds via the principal investigators, HealthPartners Institute, to support the research described in this work. Ms Cinqueonce is president and CEO of MN Community Measurement, which received a subaward from HealthPartners Institute for the MNCARES study funded by PCORI Award IHS-2019C1-15625. Dr Elwyn is a professor at the Dartmouth Institute for Health Policy and Clinical Practice and reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (LIS, EC, SPD); acquisition of data (LIS, EC, SPD); analysis and interpretation of data (LIS, DK, EC, SPD); drafting of the manuscript (LIS, GE, SPD); critical revision of the manuscript for important intellectual content (LIS, DK, EC, SPD); provision of study materials or patients (LIS); obtaining funding (LIS, SPD); administrative, technical, or logistic support (LIS, DK, EC); advice (GE); and supervision (LIS, GE, SPD).
Send Correspondence to: Steven P. Dehmer, PhD, HealthPartners Institute, PO Box 1524, Mail Stop 21112R, Minneapolis, MN 55440-1524. Email: steven.p.dehmer@healthpartners.com.
REFERENCES
Stay ahead of policy, cost, and value—subscribe to AJMC for expert insights at the intersection of clinical care and health economics.