Comparative effectiveness research and pragmatic clinical trials are valued methods to address the limitations of traditional randomized trials, answer questions of cost-effectiveness or noninferiority, and inform data-driven dialogue and decision making by stakeholders.
Published Online: May 10, 2012
Sarah J. Lowry, MPH; Elizabeth T. Loggers, MD, PhD; Erin J. A. Bowles, MPH; and Edward H. Wagner, MD, MPH
This article was published as part of a special joint issue and also appears in the Journal of Oncology Practice.
Objectives: Although much effort has focused on identifying national comparative effectiveness research (CER) priorities, little is known about the CER priorities of community-based practitioners treating patients with advanced cancer. CER priorities of managed care–based clinicians may be valuable as reflections of both payer and provider research interests.
Methods: We conducted mixed methods interviews with 10 clinicians (5 oncologists and 5 pharmacists) at 5 health plans within the Health Maintenance Organization Cancer Research Network. We asked, “What evidence do you most wish you had when treating patients with advanced cancer?” and questioned participants on their impressions and knowledge of CER and pragmatic clinical trials (PCTs). We conducted qualitative analyses to identify themes across interviews.
Results: Ninety percent of participants had heard of CER, 20% had heard of PCTs, and all rated CER/PCTs as highly relevant to patient and health plan decision making. Each participant offered between 3 and 10 research priorities. Half (49%) involved head-to-head treatment comparisons; another 20% involved comparing different schedules or dosing regimens of the same treatment. The majority included alternative outcomes to survival (eg, toxicity, quality of life, noninferiority). Participants cited several limitations to existing evidence, including lack of generalizability, funding biases, and rapid development of new treatments.
Conclusion: Head-to-head treatment comparisons remain a major evidence need among community- based oncology clinicians, and CER/PCTs are highly valued methods to address the limitations of traditional randomized trials, answer questions of cost-effectiveness or noninferiority, and inform data-driven dialogue and decision making by all stakeholders.
(Am J Manag Care. 2012;18(5 Spec No. 2):SP77-SP83)
Evidence of cancer treatment effectiveness comes primarily from randomized controlled clinical trials. Although these studies are used extensively to evaluate the efficacy of new treatments, meet US Food and Drug Administration requirements, and determine clinical guidelines, their findings may not be generalizable to community practice.1,2 This may result from implementation in academic settings and use of complicated protocols with stringent inclusion and exclusion criteria, which may not be relevant to community practices.1-4 These factors lead to greater uncertainty in decision making by doctors, patients, and policy makers, as well as in evaluations of cost-effectiveness, in these settings.
Pragmatic, or practical, clinical trials (PCTs), a form of comparative effectiveness research (CER), are an alternative to traditional trials. PCTs compare 2 or more clinically relevant interventions, recruit a population that is more representative of the target population, and assess a broad range of clinically relevant health outcomes to aid decision making for a variety of stakeholders1,3,5 (Figure). This form of research has been recognized as a potential solution to concerns regarding generalizability,1 and significant effort has gone into identifying the national CER agenda.6 Unfortunately, little research has investigated the oncology-specific CER or PCT priorities of community-based clinicians treating patients with advanced cancer who would be the recipients and users of these CER results. Rising healthcare costs, increasing intensity of care, an aging population, and the fact that cancer is the second-leading cause of death in the United States7-9 make it all the more important to achieve effective and efficient care in advanced cancer.
We interviewed 1 oncologist and 1 pharmacist from each of 5 Health Maintenance Organization (HMO) Cancer Research Network (CRN) health plans to understand their knowledge and perceptions of CER and PCTs and what evidence they wish they had when treating patients with advanced cancer. The perspectives of this clinician population are of interest because, as salaried employees of their respective health plans, their preferences for evidence and/or research priorities may reflect both provider and payer perspectives.
A single trained interviewer (EJAB), a nonclinician researcher, conducted structured telephone interviews (between December 2010 and March 2011) with 2 clinicians (1 oncologist and 1 pharmacist) from each of 5 health plans within the CRN (Group Health in Washington State; the Northwest, Northern California, Colorado, and Georgia regions of Kaiser Permanente). Delivery system pharmacists were included because they may be involved in developing formularies, approving off-label chemotherapy use, and addressing appeals representing the payer perspective. Patients were selected from the 5 locations participating in an NCI-funded grant entitled, “Building CER capacity: Aligning CRN, CMS, and state resources to map cancer care.” All 10 participants who were invited to participate agreed to do so. Participants were selected for the study on the basis of their roles as clinical leaders at each HMO Research Network site; most were the heads of their respective practices or lead pharmacists, and all had a major role in decision making in their departments. After being given a brief description of the study via e-mail, participants were contacted to schedule an interview. The study was approved by the Kaiser Permanente Colorado institutional review board.
We used a mixed methods approach, collecting both quantitative and qualitative data, to gain a more thorough understanding of participants’ responses.10,11 After querying participants on their familiarity with CER and PCTs, we emailed published definitions of CER and PCT to them in real time; these definitions (Figure) were then used as the standard for the rest of the interview. We asked participants about the relevance of CER to patient and health plan decision making as well as their self-reported likelihood of changing their practice or advocating for health plan policy change on the basis of evidence from various types of studies on a scale from 1 to 10 (1 = “extremely unlikely” and 10 = “extremely likely”). We asked open-ended questions about what evidence participants wished they had when treating patients with advanced cancer and about what CER studies or PCTs they felt were the most important to conduct at this time for patients with advanced lung, colorectal, breast, or prostate cancer. These cancers were identified a priori as having the greatest potential for impact because they are the 4 most commonly diagnosed cancers in the United States, and they have many treatment-related challenges. We also collected data on participants’ demographic characteristics.
We calculated mean values and ranges for all quantitative data. We used ethnographic software (Atlas.ti; GmbH, Berlin, Germany) to classify and analyze qualitative data from interview transcripts. Qualitative data included all answers to open-ended questions, including what evidence participants wished they had when treating patients with advanced cancer as well as their thoughts on CER, PCTs, and how they compared with traditional methods of research. Two investigators (EJAB and SJL) independently reviewed the same 2 randomly chosen transcripts to identify codes and then discussed and refined this list. Both investigators then independently reviewed the remaining 8 transcripts, identifying codes, and then discussed them and summarized major themes. All participants’ names and institutions were kept anonymous.
Participants ranged in age from 30 to 62 years (mean, 47 years); 8 of 10 were male. Participants had been in practice for an average of 17.5 years; 6 had previously practiced in a comprehensive or university-based cancer center, and all oncologists had completed a fellowship. Four participants reported that 5% to 15% of their patients were currently enrolled onto clinical trials; the rest reported less than 5%. All but 1 participant had heard of CER, whereas only 2 had heard of PCTs. After reviewing the definitions of CER and PCTs (Figure), half considered CER more relevant than traditional methods of research (including traditional trials) to health system decision making, and half considered CER more relevant (including 2 who considered it at least as relevant) to patient decision making than traditional methods of research. No participants considered CER less relevant. We observed similar results for the perceived relevance of PCTs. Participants were slightly more likely to change their practice or advocate for change in health system policies on the basis of consistent results from multiple pragmatic trials than for traditional randomized trials and less likely on the basis of results from a single trial. They were less likely to change on the basis of results from observational studies, case studies, or opinions of their colleagues or other clinicians.
When asked what evidence they wanted most when treating patients with advanced cancer, participants each provided between 3 and 10 ideas. About half of the proposed ideas involved comparisons of 2 (or more) different treatments (49%)—for example, comparing the effects of pemetrexed versus multiple older drugs on overall survival in patients with non-small-cell lung cancer (Table 1). Another 20% of proposed ideas involved comparing different aspects of a single drug or treatment, such as different doses, durations, or sequences—for example, comparing the adverse effects of 2 mg versus 4 mg of zoledronic acid (Zometa; Novartis Oncology, East Hanover, New Jersey) per month. Approximately 10% involved comparing the effects of maintenance therapy versus nothing, conducting palliative care interventions, or testing hypotheses around genetic testing. Of trial ideas that specified at least 1 health outcome, a third of ideas specified survival time as the outcome of interest, and twothirds specified toxicity and/or quality of life as the outcome.
Although participants were not directly questioned about what they perceived to be the weaknesses of existing evidence, this arose as a common theme across all interviews. Four specific weaknesses were raised by at least half of participants, including: “lack of generalizability of existing data,” “biases in funding,” “rapid development of new treatment options,” and “increasing individualization of treatment” (Table 2).
Lack of Generalizability of Existing Data
Many participants raised concerns about the generalizability of results from traditional randomized trials. According to participants, this issue greatly impacts day-to-day practice, with providers regularly deciding whether or not to apply the results of traditional trials to situations in which they may not be applicable (eg, to older patients with more comorbidities).
In one participant’s words, “A usual patient in a randomized adjuvant colon cancer study is 62; the average [age] with colon cancer in the US is 71. And of course, we always choose the healthiest quartile for our clinical trials—youngest and healthiest.”
PDF is available on the last page.