Pragmatic Clinical Trials: US Payers' Views on Their Value

Payers like key traits of pragmatic clinical trials, but are wary of pharmaceutical companies and plan to carefully scrutinize this new, appealing type of evidence.
Published Online: May 10, 2013
Jonathan Ratner, PhD; C. Daniel Mullins, PhD; Don P. Buesching, PhD; and Ronald A. Cantrell, PhD
Background: Randomized controlled trials (RCTs) reflect priorities established by regulators. Recently, pragmatic clinical trials (PCTs) have begun to attract interest. Unlike RCTs, PCTs aim to better inform post-regulatory decision making by using head-to-head comparisons of alternative treatments, diverse patient populations, and outcomes meaningful to patients, prescribers, and payers.

Objectives: To describe how US insurers and public payers perceive the value of PCTs for assessment of new prescription drugs.

Study Design: Criterion-based sample of US insurers and public payers.

Methods: We gathered qualitative evidence from intensive interviews with formulary decision makers at 15 payers, representing 10 major types of US payers. Prior literature and exploratory interviews informed our question selection.

Results: Payers viewed PCTs favorably despite wariness of drug company–sponsored trials. Payers would accept results from PCTs as part of payers’ synthesis of multiple sources of evidence. Payers were enthusiastic about 2 PCT features—a diverse population (compared with the more homogeneous populations typical of RCTs) and an active comparator drug (not placebo). Payers did not anticipate that PCTs would displace their own analyses of internal data. Pharmaceutical companies’ financial interest in obtaining trial results that favor their own drugs reduces PCTs’ perceived value and dampens their appeal to payers; nonetheless, payers would seek PCT results and review them carefully, as they do other evidence.

Conclusions: Recommendations to trial designers based on payers’ views include tailoring different types of PCTs to different disease conditions, building in head-to-head comparisons in phase IIIb PCTs, and designing phase IV PCTs to include broader populations.

Am J Manag Care. 2013;19(5):e158-e165
US payers desire both scientific rigor in clinical trial design and real-world applicability of trial results. Payers find pragmatic clinical trials (PCTs) appealing because PCTs provide:
  • Information that allows evaluation of the comparative value of drugs against active comparators.

  • Opportunities to triangulate PCTs’ evidence of effectiveness from real-world settings with data from regulatory RCTs and internal claims data sets.

  • Health outcomes for patients who more closely reflect the diversity of patients they enroll (eg, patients with comorbidities, minorities), as well as meaningful data on subpopulationsfor which a drug works.
Interest in comparative-effectiveness research (CER) suggests a need to address both regulatory requirements and requests from post-regulatory decision makers (eg, payers, patients) for evidence of the relative value of comparator treatments. Pragmatic clinical trials (PCTs) have been proposed as potential means for satisfying both regulators and post-regulatory decision makers.1 Although the definition of PCTs is emerging, a recent definition is indicative:

The RCTs whose explicit purpose is to be most informative to decision makers are called pragmatic or practical clinical trials (PCTs). . . . Common elements of such trials include clinicallyeffective comparators, study patients with common comorbid conditions and diverse demographic characteristics, and providers from community settings. Primary and secondary outcomes are patient-centered, chosen to reflect what matters most to patients and clinicians.1(p208)

Examples of PCTs include head-to-head comparisons of different pharmacologic agents for treatment of various conditions such as hypercholesterolemia2 and acute bipolar mania.3

Randomized controlled trials (RCTs) have long been considered the highest level of evidence on clinical efficacy.4 However, regulatory trials historically have focused on narrow patient populations and used placebo comparisons. In contrast, PCTs aim to answer meaningful scientific questions about investigational drugs via study designs with (1) broad patient populations, (2)  active comparators, or both. Traits of PCTs may also include (3) outcomes of interest to patients, prescribers, and payers5; (4) usual care (in contrast to a strict clinical protocol); and (5) fewer data collection elements than RCTs have (to reduce burden of  data collection). Our study defines a PCT as a randomized trial with at least 2 of these 5 “pragmatic” traits; we used this working  definition to create a shared frame of reference with our interviewees. Others, including the pragmatic-explanatory continuum indicator summary (PRECIS) group, have further discussed the extent of pragmatism in PCTs.6

Pragmatic clinical trials are seen as a bridge between RCTs (randomized) and observational studies (using real-world data).7 However, according to some observers, payer decision makers often perceive several factors as barriers to their accepting PCTs (Table 1).8,9

Moreover, because payers’ awareness and acceptance of PCTs have been understudied, uncertainty exists about whether and how US payers, key stakeholders for trials, would use results from PCTs to guide coverage and formulary decision making.


We gathered qualitative evidence to address our primary study question: Assuming that PCTs are randomized, how would payers view a PCT’s strength of evidence, compared with an RCT on the one hand and an observational study on the other? We pursued specific questions that explored which facets of PCTs payers find more compelling or more problematic than others (See Appendix for interview questions). We conducted structured, in-depth interviews of formulary decision makers and senior staff  members at private health insurers and public payers. This study was funded by Eli Lilly and Company.


We selected a criterion-based (purposive) sample10 of 15 payers and related organizations drawn from 10 categories of payers  located throughout the United States (Table 2). In selecting payers, our main criterion was that the sample reflect the diversity of US payer types—from managed care organizations offering comprehensive medical services and prescription drugs to pharmacy benefit managers; from national insurers to regional insurers; and from commercial insurers with nonelderly enrollees to public payers serving seniors.These organizations reflect most major types of US healthcare payers.

Our use of a nonprobability sample restricts the results’ generalizability. But unlike a conventional survey, the study’s purposive, interview-based approach offers a feasible method of obtaining reliable, credible, and detailed information about the use, perceived merits, and perceived difficulties of PCT evidence within US payers. Our method ensured that respondents were formulary decision makers (not junior staff assigned to fill out a website survey). Moreover, we believe our interviewing method reduced miscommunication.

Interview Format and Protocol Development

The 1-hour interviews typically included 1 to 3 formularydecision makers, drawn from the following: vice president for pharmacy, director of pharmacy, chief medical officer, other physician, vice president for drug evaluation and pharmacoeconomics, and pharmacy and therapeutics committee chair. The interviews followed a protocol of questions, although interviewees’ own logic dictated occasional changes in the order of topics. The protocol permitted interviewees to convey both interest in and concerns about PCTs. After exploratory interviews at 5 payers, we sharpened the protocol’s focus on important, under-researched topics. Moreover, early in each interview, we defined a PCT as a randomized, prospective controlled trial with the traits listed above. This established a common terminology for interviewees and interviewers.

Protocol Features

The study protocol posed questions that led interviewees to explore a decision tree (Figure). To elicit preferences between complex alternatives, we generally separated each alternative into constituent elements. Decision makers were asked to compare element A with element B, and then to explain the considerations that informed their choice. This process was repeated for various features of a PCT and for comparisons between a PCT and other types of studies. Our approach overcame 1 difficulty decision makers displayed in the initial interviews: describing their preferences regarding alternatives, each of which could involve multiple features of a trial or other type of evidence.

In addition, the protocol elicited each payer’s global assessment of PCTs as a type of evidence. This question guarded against misinterpreting a payer’s whole view of PCTs as being the sum of responses to piecemeal questions. Payers answered our questions in terms of clinical trials that met our definition of a PCT and typically did not refer to a particular PCT that they had reviewed, because their own experience with PCTs was limited.

Data Analysis/Synthesis

To develop findings, the 2 investigators who conducted the interviews independently reviewed their own notes, identified interviewees’ main points, and classified them as “consensus” or as representing 2 or more interviewee groupings. These  investigators (who were not pharmaceutical company employees) then reviewed each other’s readings of the main points and  their classification, resolved differences by reexamination of their notes, and produced findings that synthesized the interviews.


Theme 1: Payers’ Views of Place of Pragmatic Clinical Trials in Chain of Evidence

Exposure to Pragmatic Clinical Trials. All formulary decision makers consider PCTs to be rare (to date). Some decision makers did cite a head-to-head trial funded by a drugcompany as an example of a PCT, but most struggled to recall PCTs they had used in decision making. Even “classic” PCTs, the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) and Clinical Antipsychotic Trials in Intervention Effectiveness (CATIE),11,12 did not come to mind for many. We could not determine whether the issue was faulty memory (decision makers could not recall these trials at all) or imperfect labeling (decision makers did not link these trials to the term “PCT” because their practical features did not stand out). Segal11 explains why these 2 trials can be considered pragmatic. Payers’ impression that PCTs have been rare is consistent with a review that found only 22 PCTs out of the many clinical trials with reports published from January 1996 through September 2010.9

Evidentiary Chain. Once comfortable with the PCT concept as we defined it, payer decision makers stated that PCTs would be part of the continuum of evidence they consider in assessing new drugs. As our interviewees explained, when gathering evidence about a new drug, their default mode is triangulation. Rather than focus on a single type of evidence, decision makers assemble several types—for example, RCTs, analyses of their own claims database, and observational studies of postlaunch data on populations other than their own enrollees. After weighing the different sources’ conclusions, limitations, and relevance to their own enrollees, decision makers triangulate (ie, draw inferences from multiple evidence sources about the relative effectiveness of a new drug). Accordingly, decision makers stated that PCTs would become one more link in the chain of evidence they consider.

PDF is available on the last page.
Recommended Articles
Telemedicine could increase access to best-practice care for older adults with major depression who are facing barriers of mobility, stigma, and geographical isolation, and a study published in The Lancet Psychiatry compared treatment response delivered via telemedicine with same-room care.
Proposed mergers of Aetna and Humana, and Anthem and Cigna, raise questions of whether consumers will continue to see competition in health insurance markets.
Patient characteristics and product mix play an important role in determining the overall use and costs of comparable insulins across different manufacturers.
Urologists at Cancer Research UK have identified 5 distinct genomic signatures in prostate cancer that can have important implications on treatment decisions.
Fifty years ago today, President Lyndon B. Johnson signed the law that created Medicare and Medicaid, setting in motion not only the greatest change in healthcare in the nation's history at that point, but also a lasting change for society.