Currently Viewing:
The American Journal of Managed Care May 2013
Well-Child Care Visits and Risk of Ambulatory Care-Sensitive Hospitalizations
Jeffrey O. Tom, MD, MS; Rita Mangione-Smith, MD, MPH; David C. Grossman, MD, MPH; Cam Solomon, PhD; and Chien-Wen Tseng, MD, MPH
Differences in the Clinical Recognition of Depression in Diabetes Patients: The Diabetes Study of Northern California (DISTANCE)
Darrell L. Hudson, PhD, MPH; Andrew J. Karter, PhD; Alicia Fernandez, MD; Melissa Parker, MS; Alyce S. Adams, PhD; Dean Schillinger, MD; Howard H. Moffet, MPH; Jufen Zhou, MS; and Nancy E. Adler, PhD
Variability in Resource Use: Diagnosing Colorectal Cancer
F. Douglas Srygley, MD; David H. Abbott, BA, MS; Steven C. Grambow, PhD; Dawn Provenzale, MD, MS; Robert S. Sandler, MD, MPH; and Deborah A. Fisher, MD, MHS
Can Cancer Patients Seeking a Second Opinion Get Better Care?
Hui-Ru Chang, PhD; Ming-Chin Yang, DrPH; and Kuo-Piao Chung, PhD
Medical Costs Associated With Type 2 Diabetes Complications and Comorbidities
Rui Li, PhD; Dori Bilik, MBA; Morton B. Brown, PhD; Ping Zhang, PhD; Susan L. Ettner, PhD; Ronald T. Ackermann, MD; Jesse C. Crosson, PhD; and William H. Herman, MD
Patient Experience Over Time in Patient-Centered Medical Homes
Lisa M. Kern, MD, MPH; Rina V. Dhopeshwarkar, MPH; Alison Edwards, MStat; and Rainu Kaushal, MD, MPH
A Predictive Model of Hospitalization Risk Among Disabled Medicaid Enrollees
John F. McAna, PhD; Albert G. Crawford, PhD; Benjamin W. Novinger, MS; Jaan Sidorov, MD; Franklin M. Din, DMD; Vittorio Maio, PharmD; Daniel Z. Louis, MS; and Neil I. Goldfarb, BA
Currently Reading
Pragmatic Clinical Trials: US Payers' Views on Their Value
Jonathan Ratner, PhD; C. Daniel Mullins, PhD; Don P. Buesching, PhD; and Ronald A. Cantrell, PhD
Emergency Department Use Associated With Primary Care Office Management
Robert M. Goodman, DO, MHSA
Medicare Prescription Drug Plans as Perceived by Public Health Providers
Lenny Lok Shun Chan, PharmD; and Gina Ko, PharmD

Pragmatic Clinical Trials: US Payers' Views on Their Value

Jonathan Ratner, PhD; C. Daniel Mullins, PhD; Don P. Buesching, PhD; and Ronald A. Cantrell, PhD
Payers like key traits of pragmatic clinical trials, but are wary of pharmaceutical companies and plan to carefully scrutinize this new, appealing type of evidence.
Background: Randomized controlled trials (RCTs) reflect priorities established by regulators. Recently, pragmatic clinical trials (PCTs) have begun to attract interest. Unlike RCTs, PCTs aim to better inform post-regulatory decision making by using head-to-head comparisons of alternative treatments, diverse patient populations, and outcomes meaningful to patients, prescribers, and payers.

Objectives: To describe how US insurers and public payers perceive the value of PCTs for assessment of new prescription drugs.

Study Design: Criterion-based sample of US insurers and public payers.

Methods: We gathered qualitative evidence from intensive interviews with formulary decision makers at 15 payers, representing 10 major types of US payers. Prior literature and exploratory interviews informed our question selection.

Results: Payers viewed PCTs favorably despite wariness of drug company–sponsored trials. Payers would accept results from PCTs as part of payers’ synthesis of multiple sources of evidence. Payers were enthusiastic about 2 PCT features—a diverse population (compared with the more homogeneous populations typical of RCTs) and an active comparator drug (not placebo). Payers did not anticipate that PCTs would displace their own analyses of internal data. Pharmaceutical companies’ financial interest in obtaining trial results that favor their own drugs reduces PCTs’ perceived value and dampens their appeal to payers; nonetheless, payers would seek PCT results and review them carefully, as they do other evidence.

Conclusions: Recommendations to trial designers based on payers’ views include tailoring different types of PCTs to different disease conditions, building in head-to-head comparisons in phase IIIb PCTs, and designing phase IV PCTs to include broader populations.

Am J Manag Care. 2013;19(5):e158-e165
US payers desire both scientific rigor in clinical trial design and real-world applicability of trial results. Payers find pragmatic clinical trials (PCTs) appealing because PCTs provide:
  • Information that allows evaluation of the comparative value of drugs against active comparators.

  • Opportunities to triangulate PCTs’ evidence of effectiveness from real-world settings with data from regulatory RCTs and internal claims data sets.

  • Health outcomes for patients who more closely reflect the diversity of patients they enroll (eg, patients with comorbidities, minorities), as well as meaningful data on subpopulationsfor which a drug works.
Interest in comparative-effectiveness research (CER) suggests a need to address both regulatory requirements and requests from post-regulatory decision makers (eg, payers, patients) for evidence of the relative value of comparator treatments. Pragmatic clinical trials (PCTs) have been proposed as potential means for satisfying both regulators and post-regulatory decision makers.1 Although the definition of PCTs is emerging, a recent definition is indicative:

The RCTs whose explicit purpose is to be most informative to decision makers are called pragmatic or practical clinical trials (PCTs). . . . Common elements of such trials include clinicallyeffective comparators, study patients with common comorbid conditions and diverse demographic characteristics, and providers from community settings. Primary and secondary outcomes are patient-centered, chosen to reflect what matters most to patients and clinicians.1(p208)

Examples of PCTs include head-to-head comparisons of different pharmacologic agents for treatment of various conditions such as hypercholesterolemia2 and acute bipolar mania.3

Randomized controlled trials (RCTs) have long been considered the highest level of evidence on clinical efficacy.4 However, regulatory trials historically have focused on narrow patient populations and used placebo comparisons. In contrast, PCTs aim to answer meaningful scientific questions about investigational drugs via study designs with (1) broad patient populations, (2)  active comparators, or both. Traits of PCTs may also include (3) outcomes of interest to patients, prescribers, and payers5; (4) usual care (in contrast to a strict clinical protocol); and (5) fewer data collection elements than RCTs have (to reduce burden of  data collection). Our study defines a PCT as a randomized trial with at least 2 of these 5 “pragmatic” traits; we used this working  definition to create a shared frame of reference with our interviewees. Others, including the pragmatic-explanatory continuum indicator summary (PRECIS) group, have further discussed the extent of pragmatism in PCTs.6

Pragmatic clinical trials are seen as a bridge between RCTs (randomized) and observational studies (using real-world data).7 However, according to some observers, payer decision makers often perceive several factors as barriers to their accepting PCTs (Table 1).8,9

Moreover, because payers’ awareness and acceptance of PCTs have been understudied, uncertainty exists about whether and how US payers, key stakeholders for trials, would use results from PCTs to guide coverage and formulary decision making.


We gathered qualitative evidence to address our primary study question: Assuming that PCTs are randomized, how would payers view a PCT’s strength of evidence, compared with an RCT on the one hand and an observational study on the other? We pursued specific questions that explored which facets of PCTs payers find more compelling or more problematic than others (See Appendix for interview questions). We conducted structured, in-depth interviews of formulary decision makers and senior staff  members at private health insurers and public payers. This study was funded by Eli Lilly and Company.


We selected a criterion-based (purposive) sample10 of 15 payers and related organizations drawn from 10 categories of payers  located throughout the United States (Table 2). In selecting payers, our main criterion was that the sample reflect the diversity of US payer types—from managed care organizations offering comprehensive medical services and prescription drugs to pharmacy benefit managers; from national insurers to regional insurers; and from commercial insurers with nonelderly enrollees to public payers serving seniors.These organizations reflect most major types of US healthcare payers.

Our use of a nonprobability sample restricts the results’ generalizability. But unlike a conventional survey, the study’s purposive, interview-based approach offers a feasible method of obtaining reliable, credible, and detailed information about the use, perceived merits, and perceived difficulties of PCT evidence within US payers. Our method ensured that respondents were formulary decision makers (not junior staff assigned to fill out a website survey). Moreover, we believe our interviewing method reduced miscommunication.

Interview Format and Protocol Development

The 1-hour interviews typically included 1 to 3 formularydecision makers, drawn from the following: vice president for pharmacy, director of pharmacy, chief medical officer, other physician, vice president for drug evaluation and pharmacoeconomics, and pharmacy and therapeutics committee chair. The interviews followed a protocol of questions, although interviewees’ own logic dictated occasional changes in the order of topics. The protocol permitted interviewees to convey both interest in and concerns about PCTs. After exploratory interviews at 5 payers, we sharpened the protocol’s focus on important, under-researched topics. Moreover, early in each interview, we defined a PCT as a randomized, prospective controlled trial with the traits listed above. This established a common terminology for interviewees and interviewers.

Protocol Features

The study protocol posed questions that led interviewees to explore a decision tree (Figure). To elicit preferences between complex alternatives, we generally separated each alternative into constituent elements. Decision makers were asked to compare element A with element B, and then to explain the considerations that informed their choice. This process was repeated for various features of a PCT and for comparisons between a PCT and other types of studies. Our approach overcame 1 difficulty decision makers displayed in the initial interviews: describing their preferences regarding alternatives, each of which could involve multiple features of a trial or other type of evidence.

In addition, the protocol elicited each payer’s global assessment of PCTs as a type of evidence. This question guarded against misinterpreting a payer’s whole view of PCTs as being the sum of responses to piecemeal questions. Payers answered our questions in terms of clinical trials that met our definition of a PCT and typically did not refer to a particular PCT that they had reviewed, because their own experience with PCTs was limited.

Data Analysis/Synthesis

To develop findings, the 2 investigators who conducted the interviews independently reviewed their own notes, identified interviewees’ main points, and classified them as “consensus” or as representing 2 or more interviewee groupings. These  investigators (who were not pharmaceutical company employees) then reviewed each other’s readings of the main points and  their classification, resolved differences by reexamination of their notes, and produced findings that synthesized the interviews.


Theme 1: Payers’ Views of Place of Pragmatic Clinical Trials in Chain of Evidence

Exposure to Pragmatic Clinical Trials. All formulary decision makers consider PCTs to be rare (to date). Some decision makers did cite a head-to-head trial funded by a drugcompany as an example of a PCT, but most struggled to recall PCTs they had used in decision making. Even “classic” PCTs, the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) and Clinical Antipsychotic Trials in Intervention Effectiveness (CATIE),11,12 did not come to mind for many. We could not determine whether the issue was faulty memory (decision makers could not recall these trials at all) or imperfect labeling (decision makers did not link these trials to the term “PCT” because their practical features did not stand out). Segal11 explains why these 2 trials can be considered pragmatic. Payers’ impression that PCTs have been rare is consistent with a review that found only 22 PCTs out of the many clinical trials with reports published from January 1996 through September 2010.9

Evidentiary Chain. Once comfortable with the PCT concept as we defined it, payer decision makers stated that PCTs would be part of the continuum of evidence they consider in assessing new drugs. As our interviewees explained, when gathering evidence about a new drug, their default mode is triangulation. Rather than focus on a single type of evidence, decision makers assemble several types—for example, RCTs, analyses of their own claims database, and observational studies of postlaunch data on populations other than their own enrollees. After weighing the different sources’ conclusions, limitations, and relevance to their own enrollees, decision makers triangulate (ie, draw inferences from multiple evidence sources about the relative effectiveness of a new drug). Accordingly, decision makers stated that PCTs would become one more link in the chain of evidence they consider.

Overall, among interviewees, the concept of triangulation across RCTs, PCTs, registry data, internal claims-based studies, and observational studies was widely shared. In an alternative analogy, evidence forms a multidimensional crossword puzzle: each dimension (type of evidence) provides a clue to the answer.

External Validity. Formulary decision makers find PCTs appealing because they improve the external validity of the evidence base. Payers generally view explanatory RCTs as exemplars of internal validity. Nonetheless, decision makers stated they want evidence of external validity: that the results apply not just to the RCT’s population but also to the payer’s own enrollees. Decision makers noted that patients excluded from many RCTs often resemble payers’ own patient populations more than the   RCTs’ restricted populations. Consequently, payer decision makers generally favor PCTs over RCTs because PCTs’ broad  population holds greater relevance to payers. Furthermore, payers recognize that more external validity (a wider array of subgroups studied) often entails less internal validity (small subgroup samples and less statistical power to detect a treatment effect). Among decision makers interviewed, the consensus was that external validity trumps internal validity—perhaps with caveats; for example, when a drug is the first in a therapeutic class or when evidence of effectiveness of comparators is weak.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up