Currently Viewing:
The American Journal of Managed Care September 2015
Do Patient or Provider Characteristics Impact Management of Diabetes?
Erin S. LeBlanc, MD, MPH; A. Gabriela Rosales, MS; Sumesh Kachroo, PhD; Jayanti Mukherjee, PhD; Kristine L. Funk, MS; and Gregory A. Nichols, PhD
The Utility of Cost Discussions Between Patients With Cancer and Oncologists
S. Yousuf Zafar, MD, MHS; Fumiko Chino, MD; Peter A. Ubel, MD; Christel Rushing, MS; Gregory Samsa, PhD; Ivy Altomare, MD; Jonathan Nicolla, MBA; Deborah Schrag, MD; James A. Tulsky, MD; Amy P. Abernethy, MD, PhD; and Jeffery M. Peppercorn, MD, MPH
Building Upon the Strong Foundation of National Healthcare Quality
Charles N. Kahn III, MPH, President and CEO, Federation of American Hospitals
Improving Partnerships Between Health Plans and Medical Groups
Howard Beckman, MD, FACP, FAACH; Patricia Healey, MPH; and Dana Gelb Safran, ScD
Innovative Approach to Patient-Centered Care Coordination in Primary Care Practices
Robin Clarke, MD, MSHS; Nazleen Bharmal, MD, PhD; Paul Di Capua, MD, MBA; Chi-Hong Tseng, PhD; Carol M. Mangione, MD, MSPH; Brian Mittman, PhD; and Samuel A. Skootsky, MD
Private Sector Risk-Sharing Agreements in the United States: Trends, Barriers, and Prospects
Louis P. Garrison, Jr, PhD; Josh J. Carlson, PhD; Preeti S. Bajaj, PhD; Adrian Towse, MA, MPhil; Peter J. Neumann, ScD; Sean D. Sullivan, PhD; Kimberly Westrich, MA; and Robert W. Dubois, MD, PhD
Currently Reading
Developing Evidence That Is Fit for Purpose: A Framework for Payer and Research Dialogue
Rajeev K. Sabharwal, MPH; Jennifer S. Graff, PharmD; Erin Holve, PhD, MPH, MPP; and Robert W. Dubois, MD, PhD
Reply to "Transforming Oncology Care": Advancing Value, Accessing Innovation
Rebecca Paradis, MPA
Payer Source Influence on Effectiveness of Lifestyle Medicine Programs
Joseph Vogelgesang, BS; David Drozek, DO; Masato Nakazawa, PhD; Jay H. Shubrook, DO
High-Risk Centers and the Benefits for Lower-Risk Transplants
Schelomo Marmor, PhD, MPH; James W. Begun, PhD; Jean Abraham, PhD; and Beth A. Virnig, PhD, MPH
Targeting a High-Risk Group for Fall Prevention: Strategies for Health Plans
Lee A. Jennings, MD, MSHS; David B. Reuben, MD; Sung-Bou Kim, MPhil; Emmett Keeler, PhD; Carol P. Roth, RN, MPH; David S. Zingmond, MD, PhD; Neil S. Wenger, MD, MPH; and David A. Ganz, MD, PhD
Socioeconomic Disparities Across Ethnicities: An Application to Cervical Cancer Screening
Brendan Walsh, PhD; and Ciaran O'Neill, PhD

Developing Evidence That Is Fit for Purpose: A Framework for Payer and Research Dialogue

Rajeev K. Sabharwal, MPH; Jennifer S. Graff, PharmD; Erin Holve, PhD, MPH, MPP; and Robert W. Dubois, MD, PhD
A decision-making framework that can be used to harmonize the evidence payer's desire for coverage and formulary decisions with the evidence generated by researchers.
Objectives: Matching the supply and demand of evidence requires an understanding of when more evidence is needed, as well as the type of evidence that will meet this need. This article describes efforts to develop and refine a decision-making framework that considers payers’ perspectives on the utility of evidence generated by different types of research methods, including real-world evidence. 
Study Design: Conceptual framework development with subsequent testing during a roundtable dialogue.

Methods: The framework development process included a literature scan to identify existing frameworks and relevant articles on payer decision making. The framework was refined during a stand-alone roundtable in December 2013 hosted by the research team, which included representatives from public and private payers, pharmacy benefit management, the life sciences industry, and researchers. The roundtable discussion also included an application of the framework to 3 case studies. 

Results: Application of the framework to the clinical scenarios and the resulting discussion provided key insights into when new evidence is needed to inform payer decision making and what questions should be addressed. Payers are not necessarily seeking more evidence about treatment efficacy; rather, they are seeking more evidence for relevant end points that illustrate the differences between treatment alternatives that can justify the resources required to change practice. In addition, payers are interested in obtaining new evidence that goes beyond efficacy, with an emphasis on effectiveness, longer-term safety, and delivery system impact. 

Conclusions: We believe that our decision-making framework is a useful tool to increase dialogue between evidence generators and payers, while also allowing for greater efficiency in the research process.

Am J Manag Care. 2015;21(9):e545-e551
Take-Away Points
Key insights into when new evidence is needed to inform payer decision making and what issues should be addressed include: 
  • Payers are not necessarily seeking more evidence about treatment efficacy. Rather, they are seeking more evidence for relevant end points. 
  • Payers are interested in obtaining new evidence that goes beyond efficacy with an emphasis on effectiveness, longer-term safety, and delivery system impact. 
  • In an era focused on care coordination, quality of care, and risk sharing, evidence to understand and improve outcomes associated with real-world use take on increasing importance.
A number of efforts are underway to improve the quantity, quality, and capacity for evidence to inform decision making. These efforts include funding from the Patient Centered Outcomes Research Institute for specific studies and research infrastructure (eg, PCORnet),1,2 increased electronic health record availability, and research collaborations between the life sciences industry and payers (eg, AstraZeneca and WellPoint’s HealthCore).3 Additionally, tools such as the GRADE framework4 and the CER Collaborative5 seek to increase the capacity for and acceptance of real-world evidence, while activities like the Green Park Collaborative6 seek to clarify the demand for evidence in specific therapeutic areas.

Often, the supply of evidence may not meet the demand. Payers frequently note the gap between the available evidence versus the evidence desired to increase the certainty of coverage decisions,7 including evidence to: 1) confirm study results among typical patients in routine care; 2) assess outcomes associated with care delivery, such as adherence and total cost of care, and 3) compare existing care alternatives. Beyond what is available, less than half of the health care recommended in care guidelines or delivered in practice are supported by existing evidence.8,9 Subsequently, this raises the question of whether investments in evidence should be made for all decisions, and the extent to which the evidence will be applied in care decisions.

Matching evidence supply and demand requires an understanding of not only when more evidence is needed, but also what types of evidence will meet this need. To raise the likelihood that the evidence generated is “fit for purpose” (ie, it responds to the specific needs of the intended user), we developed and tested a decision-making framework. This framework considers payers’ perspectives on the utility of evidence generated by different types of research methods, including real-world evidence. Our hope is that the framework will help to harmonize the evidence that payers desire for coverage and formulary decisions with the evidence received from researchers, as well as help to guide researchers on what types of evidence need to be developed in the future.

Previous attempts to identify the types of research needed to address particular questions have included translation tables10,11 and collaborative dialogues. However, these efforts differ because they emphasize the perspectives of multiple stakeholder groups who often have different questions of interest based on the patient population, outcomes, or comparators being targeted. In contrast to other stakeholders, payers often seek effectiveness information comparing treatment alternatives and outcomes associated with clinical and economic efficiency. Our focus on a single stakeholder group allows for a narrowing of the comparators, patient populations, and outcomes of interest. Further, given the increased availability of real-world evidence and changing perceptions regarding its applicability and limitations, we aim to understand payers’ interest and use of such evidence.

Payer decision making has typically focused on evidence from randomized control trials (RCTs), with less utility for real-world evidence.12 This is due in part to the traditional evaluation of evidence based on hierarchies that often place RCTs at the top, with other methods—such as observational studies and indirect treatment comparisons—at the bottom. Application of the framework also allows us to test the notion brought forth by Sir Michael Rawlins and others, that rather than focusing on evidence hierarchies, a variety of approaches should be accepted and included in the evidential base.13

Fit for Purpose Framework

To help inform the framework development process, we began with a literature scan to identify existing frameworks, articles, and elements relevant to payer decision making and payer use of different evidence types. Our scan included 48 primary articles comprising payer surveys, case studies, and commentaries, and identified 29 elements of interest to payer decision making. While organizing the elements identified in the literature review, we found that David Eddy’s commentary on clinical decision making provided a useful structure. The commentary describes 2 critical steps for a decision: 1) the scientific-focused analysis of evidence and 2) the personal judgments or preferences that one applies to the corresponding evidence analysis. Similar to Eddy’s “Anatomy of a Decision,” 14 our original framework began with a general technical assessment; however, we determined the technical assessment and type of new evidence would be contingent upon the clinical question or scenario. This led us to include both a review of available evidence, as well as potential contextual factors as key elements in the framework. Using 2 clinical cases, we then vetted the draft framework with 4 experts (1 state Medicaid director, 1 pharmacy director, and 2 medical directors) via telephone. Based on these interviews, we clarified the key outcomes (eg, differentiating between long-term and short-term outcomes) and added additional elements to the contextual factors (eg, current formulary, current market share, and where and how a therapy fits into clinical management).

Our framework was subsequently vetted and refined during a full-day Fit for Purpose Roundtable in Washington, DC, hosted by the research team, which included 12 representatives from public and private payers, pharmacy benefit managers, the life sciences industry, and researchers (eAppendix, available at The discussion focused on clarifying and/or adding new elements within the final framework construct and applying the framework to 3 clinical cases. The framework (Figure) includes 3 main components: 1) the “Clinical Scenario,” which establishes the research questions of interest, the potential impact of an intervention or therapy on individual patients, as well as the health system, and a determination of value; 2) the “Assessment,” which comprises a review of the available evidence and a consideration of the contextual factors; and 3) the “Coverage Decision,” which is the ultimate output of the payer decision-making process.

A number of elements are considered within each of the framework components. Together, the 3 components help determine the extent to which evidence is fit for purpose with questions such as: Is the current evidence sufficient for that clinical scenario and coverage decision? If not, what types of new evidence questions need to be answered and which research methods would bring more certainty to or change the decision?

The framework is not intended to be used as a checklist where a determination is made for each element. Rather, for a given clinical scenario, one can determine which elements are most applicable and which should be included in evidence review. In addition, we acknowledge that individual elements can, and often should, be viewed across the different components of the framework. For example, an element such as site of care may have an impact on outcomes, care management processes, and the total cost of care; these elements should be considered collectively as one.

Application of the Framework

During the roundtable, participants applied the framework to 3 clinical cases. A clinical monograph and information relevant to the framework elements (Table) were provided in advance of the meeting. These cases included: 1) determining coverage for a new hepatitis C virus (HCV) therapy (sofosbuvir); 2) confirming coverage for oral anticoagulants (rivaroxaban, dabigatran, apixaban); and 3) limiting utilization for a multiple sclerosis (MS) therapy (natalizumab) with known safety concerns.

The review and discussion of the cases and existing evidence was designed to elicit additional considerations and factors related to the evidence needs of payers rather than reaching consensus on specific coverage decisions. Roundtable participants were asked to identify any missing components, confirm the overall organization of the framework, and identify which elements were of greater weight or importance for their coverage decision. Each case was reviewed and each participant rated via a hand-held voting device (Turning Point 2008 software) if the current evidence was sufficient for the decision, if lingering concerns existed, and which additional type(s) of evidence would be desired and required. Meeting transcripts were coded and analyzed for thematic responses leading to the discussion of key findings below.

The discussion of the new HCV therapy occurred just as the product was approved for use by the FDA. This new treatment represents a new method of administration that is hypothesized to have greater adherence. The long-term adverse effects, clinical practice acceptance, and utilization, however, are unknown in practice. Most participants indicated that the available evidence regarding efficacy and safety was sufficient for them to make a decision regarding coverage; however, additional evidence from prospective observational studies that focused on comparisons to existing treatment alternatives, long-term efficacy, safety, and adherence would help to confirm their coverage decision. The desire for observational studies was driven in part by uncertainty regarding the tolerability compared with existing treatment with interferon and the ability for a new oral agent to translate into better treatment outcomes in practice. In this case example, evidence on different clinical outcomes from the registrational trials was desired. Research designs that could assess these outcomes were viewed as having potentially high value, contributing to coverage decisions.

The class review asked participants to confirm coverage and tier placement of oral anticoagulants (either Factor Xa or direct thrombin inhibitors) for the prevention of stroke in atrial fibrillation. In practice, participants and their organizations had already made coverage decisions (albeit varied decisions) and few had plans to alter these decisions. Compared with the HCV scenario, fewer participants felt additional evidence was required, as their clinical experience with the existing agents limited their uncertainty. In this case example, additional evidence on the net benefit was unlikely to be of sufficient magnitude to overcome the existing contextual factors and to subsequently alter the coverage decision.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up