Objective: To assess barriers to physician participation in cancer clinical trials among oncologists, oncology leaders, and health plan leaders.
Study Design: Mail survey of 221 oncologists combined with semistructured telephone interviews with oncology and plan leaders at 10 integrated healthcare systems.
Methods: The survey instrument examined physicians' involvement in clinical trials; their perception of the value of trials to them, their patients, and their organization; and the presence of infrastructure support for trials and associated resource constraints. The interviews investigated similar issues from the leaders' perspective. We used linear regression to model trial enrollment and standard qualitative techniques to analyze the interviews.
Results: Oncologists estimated they enrolled 7% of patients in trials. They expressed extremely favorable attitudes toward trials as a source of high-quality patient care and a benefit to themselves professionally. While positive attitudes toward trials were common, and were significant bivariate predictors of enrollment, organizational factors were the predominant predictors in multivariate analysis. The best combination of factors independently predicting enrollment related to organizational support for trials, subspecialty of the oncologist, and limitations of trial eligibility requirements.
Conclusions: To increase trial participation, there is a critical need for infrastructure to support trials, especially additional support staff and research nurses. In addition, there is a need for better intra-organizational communication and consideration of the impact of trial design on internal health plan resources. This research supports the need to continue a national dialogue about the broadly defined benefits and costs of clinical trials to patients, physicians, and health plans.
(Am J Manag Care. 2005;11:413-421)
Participation in a clinical trial affords access to the latest investigational interventions and close monitoring of care, yet only a small proportion of eligible adult cancer patients are offered the opportunity to participate, and fewer actually enroll. As few as 2% to 3% of adult cancer patients are enrolled in clinical trials in the United States,1-12 and enrollment often is not representative of the general population.1,2,8,13-29
Historically, the majority of participants in cancer clinical trials have come from academic settings.12,30-32 Little research on barriers to accrual has been conducted in nonacademic environments such as integrated healthcare systems, despite their potential to recruit sizable and representative samples of patients for cancer clinical trials.33-35 Although Kaluzny and colleagues have examined ways that alliances between organizations affect enrollment in cancer clinical trials,36-40 factors internal to individual healthcare systems that may act as barriers to or facilitators of trial enrollment have rarely been examined. To address this gap, our study examined attitudes and beliefs of oncologists working in integrated care systems, as well as the attitudes and beliefs of their respective oncology and health plan leaders, about the role and importance of cancer clinical trials. We also investigated predictors of accrual in this environment.
This study was a project of the Cancer Research Network, a consortium of 11 health plans funded by the National Cancer Institute (NCI). Cancer Research Network plans include the Fallon/Meyers Primary Care Institute; the Group Health Cooperative; Harvard Pilgrim Health Care/Harvard Vanguard; the Health- Partners Research Foundation; the Henry Ford Health System; and Kaiser Permanente in Colorado, Hawaii, Northern California, Georgia, the Northwest, and Southern California. These plans, which provide comprehensive healthcare for approximately 9 million enrollees across the United States, are nonprofit, integrated care systems that have independent research departments committed to public-domain research.35 An oncologist and a researcher from 10 plans collaborated on this study. The institutional review boards of all participating plans approved the research.
Data were collected using a mailed survey to oncologists and semistructured, qualitative interviews with oncology leaders and health plan leaders conducted by local site investigators and 2 of the Kaiser Permanente Northern California researchers. The interviews were designed to describe the different organizational arrangements pertaining to clinical trials within integrated care systems and explore how these may facilitate or constrain trial activity.
The survey sample included the universe of adult oncology subspecialists (n = 221) at the 10 plans. Eligibility criteria included employment in the health plan for at least 1 year and at least half-time clinical practice. Surveys were returned anonymously to allay physicians' concerns about confidentiality. To make the survey anonymous and still allow follow-up with nonrespondents, oncologists separately mailed back a postcard when they returned the survey, indicating that they had completed the survey. If they did not complete the survey, Kaiser Permanente Northern California staff sent them a reminder letter and another survey and postcard.41
The survey instrument assessed physicians' involvement in clinical trials; their perception of the value of trials to them, their patients, and their organization; and the presence of infrastructure support for trials and associated resource constraints. (Contact the correspoinding author for a description of the survey questions.) Questions were based in part on an unpublished survey of oncologists conducted by the American Society of Clinical Oncologists.42 All perception questions used 5-point response scales ranging from highly negative to highly positive (eg, "strongly disagree" to "strongly agree," "a major barrier" to "not a barrier at all," "not at all helpful" to "very helpful"). The primary outcome measure was the self-reported percentage of each oncologist's patients enrolled in clinical trials in the prior 6 months.
The qualitative interviews with health plan and oncology leaders considered organizational affiliations with clinical-trial programs, physician involvement in clinical trials, resources, and the value of clinical-trial enrollment. For reasons of confidentiality, we did not tape-record interviews. Rather, interviewers took notes during the interviews, and then shared what they had recorded with each respondent for verification. We analyzed interview data using standard qualitative techniques,43,44 wherein we coded and categorized themes for comparison and analysis. Oncologists from the study team reviewed interpretations for accuracy and validity.
We hypothesized that higher self-reported trial enrollment would be associated with more favorable attitudes toward trials, fewer perceived barriers, and greater infrastructure support. All hypothesis tests were 2 sided, and an alpha level of 5% was used unless otherwise stated. We used linear regression to model adult enrollment. When building models, we accounted for possible clustering or correlation within health plans using generalized estimating equation models. First, we built models including only 1 possible variable at a time. Then, using the variables that were significant at P ≤ .20, we constructed a stepwise multivariate model. At each step, the variable that best improved the fit of the current model was included.
Description of Sample
Out of a total eligible survey sample of 221, 90% responded. Twenty-six percent of the respondents were female, and 62% were medical oncologists. Eighty-three percent worked in a group-model or staff-model plan. On average, oncologists estimated that they enrolled 7% of their patients in clinical trials (Table 1). We interviewed 9 pairs of oncology and health plan leaders from participating health plans. We were not able to interview leaders from the tenth plan because of administrative changes occurring at this plan during the research. Leader interviews showed that the health plans in our sample were diverse in terms of size and organizational complexity, as well as organizational commitment to and support for clinical trials. Although the health plans all were multisite organizations, some had their oncology practice centralized at 1 clinic, while others had multiple clinics. Three were members of local Community Clinical Oncology Programs; 2 either were affiliated with or were themselves cancer centers; and 5 were cooperative group members.
Attitudes Toward Clinical Trials
Overall, oncologists expressed highly favorable attitudes toward clinical trials (Table 2). Sixty-three percent of adult oncologists agreed that clinical trials were their first treatment choice for therapy if available. Almost 90% agreed that clinical trials provided high-quality care. About three quarters believed that clinical trials were an appropriate use of resources in a community setting, and most oncologists derived professional value from participating in trials. However, almost 25% of oncologists reported that their patients were not interested in clinical trials and that most patients had comorbid conditions that precluded trial participation.
The opinions of the health plan leaders about whether clinical trials provide high-quality care were more diverse than those of the practicing oncologists and the oncology leaders. Although 8 oncology leaders saw trial participation as key to high-quality care, 1 leader was fairly emphatic that he had not seen data supporting the notion that care received in a trial is of higher quality than treatment received under standard regimens. In contrast, health plan leaders were almost evenly divided on this issue. For example, although 1 health plan leader viewed trial participation as a benchmark of quality cancer care, another said he would not consistently agree with this proposition. He pointed out that care under a clinical trial could be futile or worse (as in the case of bone marrow transplants for breast cancer) since, by definition, the efficacy of the treatments tested cannot be fully known in advance of trial results.
Leaders' opinions about the value of clinical trials to their organization varied greatly. Three health plan leaders viewed trial participation as an organizational goal or value, while only 2 oncology leaders perceived that to be so in their plan. Responses of health plan and oncology leaders were congruent on this issue in only 3 pairs, indicating that at the majority of plans, the oncology leader and the health plan leader were not aware of each other's views on the value of trials to their organization (Table 3). As expected, the congruent positive responses came from the health plan with a very strong, well-funded clinical-trials program, while the congruent negative responses came from the plan with the least supportive environment for trial participation. In the case of matched equivocal answers, the oncology leader believed that providing clinical trials was an organizational goal of the physician group, but that it was highly problematic for the insurance side of the organization. The health plan leader's view was that trials were highly appropriate for some patients, but he was concerned with the proliferation of poorly designed trials.
Predictors of Self-reported Clinical Trial Enrollment
Table 4 shows the bivariate correlates of clinical-trial enrollment. The majority of favorable measures of attitudes toward trials, as well as perceived barriers and experienced infrastructure, were associated in the bivariate analysis with adult trial enrollment. For example, every 1-unit increase in physicians' belief that clinical trials provide high-quality care (eg, from "agree" to "strongly agree" or from "disagree strongly" to "disagree") was associated with an increase of 3 percentage points in the percentage of patients enrolled in clinical trials.
Neither age, sex, years in practice, years working at their plan, or practice setting was significantly related to enrollment (data not shown). Subspecialty had a significant effect on the level of enrollment. Compared with medical oncologists, gynecologic and radiation oncologists were less likely to enroll patients in trials; while surgical oncologists were somewhat more likely, and other subspecialists were significantly more likely, to enroll patients in trials.
To examine which domains were most strongly associated with trial enrollment, we conducted a multivariate stepwise analysis (Table 5). Only 5 of the items included in Table 4 were independently associated with trial accrual and 3 of these pertained to infrastructure. Positively related to enrollment were the perception that nurses were helpful in supporting trial participation, briefings by the department chair, and subspecialty. Lack of support staff to assist in patient enrollment and limitations of eligibility requirements negatively impacted enrollment. None of the measures of attitudes and values associated with enrollment in the bivariate analysis remained significant in the multivariate analysis.
Since 62% of our sample came from Kaiser Permanente Northern and Southern California, we compared the survey responses of those 2 plans with the others. There were no differences except on those variables related to resources: plans that reported greater resources also had higher accrual (data not shown).
Infrastructure and Support for Clinical Trials
Varying levels of infrastructure support for clinical trials were perceived by the oncologists surveyed and confirmed by the oncology and health plan leaders. About two thirds of practicing oncologists surveyed reported the lack of needed infrastructure to conduct trials. The number of open trials at different sites during 2001, in part related to the level of supporting infrastructure, ranged from 2 to 188 in the 7 plans for which data were available. At 6 plans, oncologists, or a subcommittee of oncologists, met regularly to evaluate which trials their group should offer to patients. The plan with the least trial enrollment, as determined by averaging oncologists' survey responses by health plan, had no nonphysician staff to support clinical trial participation. At this plan, oncologists were responsible for everything from introducing trials to patients to preparing institutional review board applications. At the other end of the staffing continuum, 1 oncology leader reported that his clinical-trial program did have sufficient staff at that time to cover their extensive research activity. This plan had case managers, institutional review board staff, and data managers. The other 7 sites fell somewhere in the middle.
At all plans, oncologists were notified about new protocols at least monthly, by hard copy and electronic communication. Most sites relied on one-on-one patient contact for recruitment. Two oncology leaders discussed recruiting patients via patient databases, and a third described his health plan's external Web site, on which all available trials were listed and by which patients could self-refer to trials.
Because financial decisions are directly related to the availability of infrastructure to support clinical-trial enrollment, we discussed financial/resource issues with health plan leaders. Eight health plan leaders addressed this issue, and 6 emphasized the importance of trials being cost neutral, or at least not being a significant drain that affects other patients' care. While supporting clinical trials in theory, several health plan leaders felt that oncologists' participation in trials was a resource issue because there were not sufficient full-time-equivalent oncologists to maintain normal clinical operations and full trial participation. As an oncology leader from a plan with very low trial participation said, individuals and payers were purchasing health insurance to support healthcare, not research. In contrast to this negative view of the financial impact of trials, both the oncology leader and the health plan leader at another site described their organization's recent commitment of major funding to support cancer clinical trials.
Health plan leaders also discussed the need for NCI to provide a fuller proportion of the actual costs associated with clinical trials. One leader estimated that cooperative group support for trials covers approximately 60% of trials' costs, meaning that local institutions absorb the remaining costs. Another leader pointed out that when participating plans absorb these costs, they are essentially a private institution contributing to the public good. As such, he felt it would be beneficial to give more credit to participating institutions, allowing for both internal and external acknowledgment of health plans' and other private organizations' contributions to federally sponsored research.
Two health plan leaders discussed the benefits of better communication within plans and with NCI. One leader at a multisite plan mentioned that his organization could have better communication among physicians about the real cost of clinical research. He reported that while some physicians in leadership roles may see trials as a drain of resources, he did not believe they typically were, especially considering the small percentage of the plan's overall clinical activity that clinical trials represent. In addition, another plan leader welcomed the opportunity to participate upstream with scientific leadership in the design of trials. He noted, for example, that imaging studies are complex, are hard on patients, and require significant resources. If researchers are removed from daily operations and are not aware of these issues, it is easy to think that 4 computerized-tomography scans a year are no more difficult than 3. In reality, it is much easier to have 3 scans a year than 4, both for the patient and the institution.
This is the first study of which we are aware to empirically assess physician and intra-organizational barriers to conducting clinical trials in integrated care systems. These barriers are important to address because such plans provide a fertile ground for conducting clinical research.35 We found that oncologists generally reported extremely favorable attitudes toward trials, both as a source of high-quality patient care and as a benefit to themselves professionally. However, while positive attitudes toward trials were common and were significant bivariate predictors of enrollment, organizational factors were the predominant independent predictors of increased trial enrollment when multiple variables were taken into account. Thus, of the many bivariate correlates of enrollment, the combination of factors that independently best predicted adult enrollment related to organizational support for trials (from nurses, support staff, and department chair briefings); the subspecialty of the oncologist (perhaps an indicator of lower patient load); and a characteristic of the trials themselves (limitations of eligibility requirements).
Interviews with oncology and health plan leaders demonstrated the importance of understanding the perspectives of both the health plan and the practicing oncologists when planning clinical-trial protocols. Health plan leaders generally take a broader view of the impact of clinical trials on the healthcare operations than do oncologists. Oncologists see the resource burden of trials largely in terms of the need for additional support staff to accomplish the multiple tasks associated with enrolling and following study patients. Health plan leaders view trials in terms of the financial cost of additional tests and protocols, as well as being concerned that additional research requirements might decrease access to medical services for patients not in trials. Previous studies that have assessed the costs of clinical trials have compared treatment costs of patients in trials with those not in trials and have concluded that participation in clinical trials does not result in substantial increases in the direct costs of medical care.45,46 It is important that future studies also measure the research infrastructure costs of conducting trials47,48 as well as the "cost" of decreased access to care for patients not in trials.
Our study identified several ways that better communication between individuals in different parts of healthcare organizations could improve the process of building clinical-trial programs in community-based health plans. For example, in 2 of 9 cases the health plan leader considered clinical trials as an organizational goal or value, while the oncology leader was not aware of this priority. In another case the health plan leader did not view clinical trials as an organizational goal, while the oncology leader perceived that it was. Our interviews also identified the need for better communication planwide about the actual financial and human costs of conducting clinical trials, as suggested by the health plan leader who noted that some physician leaders saw trials as a much greater drain on resources than they actually were, given the small percentage of overall clinical activity they represented.
A number of factors make integrated care systems ideal environments in which to conduct clinical trials, such as their large populations, sophisticated computerized data systems, and commitment to public-domain research and evidence-based decision making. These factors are counterbalanced to some extent by the belief expressed by several leaders in our study that the primary function of healthcare organizations is the delivery of medical care rather than pursuit of research. Still, the majority of practicing oncologists did not share the view that clinical trials are an inappropriate use of resources in their setting. The somewhat discordant views of oncologists and health plan leaders reflect the tension, present in clinical research generally, about whether the main reason for enrolling individual patients in clinical trials is to improve treatment for future patients or to ensure that current patients receive state-of-the-art treatment.49-51 Thus, this research points to the need to continue a national dialogue about the broadly defined benefits and costs of clinical trials to patients, physicians, and health plans.
This study has several strengths in addition to its 90% survey response rate. The combination of quantitative and qualitative data provides a more comprehensive picture of the opportunities and constraints regarding clinical-trial enrollment than has been presented before. Data collected from individuals at multiple levels of responsibility in the organization deepen our appreciation of the need to consider various perspectives in order to increase trial participation. The leader interviews demonstrate the range of opinions across plans and highlight the variability in structural factors that may affect trial accrual. Though our study was limited to integrated care systems, the participating plans are among the highest-accruing sites to many clinical trials via a variety of institutional affiliations with Community Clinical Oncology Programs, cooperative groups, and cancer centers. This research also adds an intra-organizational element to the organizational literature on clinical trials.36,37
A limitation of the study concerns our use of self-report to measure our outcome variable: enrollment in clinical trials. Our desire for a high response rate necessitated that the surveys be returned anonymously, precluding the possibility of directly validating respondents' self-report. Furthermore, all sites do not include trial enrollment statistics in their administrative data. However, we used 2 methods to indirectly validate self-report. Using administrative data at 3 plans for oncology consultations and trial accrual, we found the ratio of self-reported enrollment to actual enrollment varied from 1 (ie, no overestimation of accrual) to 2. Alternatively, using the number of new cancer patients diagnosed in 2001 at 7 of the 10 plans as a proxy for the number of oncology patients treated, we found that oncologists overestimated their enrollment by a factor of 2.5.
Although it is well known that in surveys physicians tend to overestimate their actual practice,52-55 the extent of overestimation in our study is comparable to that in a study of physicians in the Eastern Oncology Group, who overestimated their accrual by a factor of 3.4.52 While oncologists may have overestimated their accrual, there is no reason to believe that subspecialists overestimated differentially.52 In addition, the purpose of our study was to better understand factors associated with enrollment rather than to accurately measure the level of enrollment.
Inasmuch as our analyses are based on cross-sectional data, one should interpret the causal direction cautiously. It is likely that clinical-trial enrollment in integrated care systems is the product of the interaction between practicing oncologists' beliefs and values, and support from medical leadership. Leadership proximate to the practicing oncologist can shape the degree of clinical-trial participation through supportive managerial functions, including time and space allocations and individual recognition, while the more distant health plan leaders' influence is predominantly exerted through control over research infrastructure resources. For a clinical-trial program to flourish in an integrated care setting, our study illustrates the importance not only of strong interest among oncologists, but also of substantial support by health plan leadership to subsidize infrastructure.
The authors thank Martin Brown, PhD, and Rachel Ballard-Barbash, MD, MPH, for encouragement and support throughout the project; Andy Avins, MD, Joe Selby, MD, MPH; Bruce Fireman, MA, and Connie Weisner, DrPH, for their critical review of an earlier version of the manuscript; and Robin Altaras and Aikya Param, MA, for their administrative support.
From the Division of Research, Kaiser Permanente Northern California, Oakland, Calif (CPS, AA, LA); the Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, Calif (AMG, JP); Group Health Cooperative, Center for Health Studies, Seattle, Wash (SMG, EW); Kaiser Permanente Colorado, Denver, Colo (JM); Kaiser Permanente Hawaii, Honolulu, Hawaii (JH); Kaiser Permanente Medical Center, Vallejo, Calif (LF); HealthPartners Research Foundation, Minneapolis, Minn (AN); Kaiser Permanente Northwest, Portland, Ore (AG); Harvard Pilgrim Health Care/Harvard Vanguard, Boston, Mass (ST); Henry Ford Health Systems, Detroit, Mich (CS); and Meyers Primary Care Institute, Worcester, Mass (TF).
Supported through a supplement to grant NCI 1U19-CA-79689 from the National Institutes of Health (Edward Wagner, MD, MPH, Principal Investigator).
Address correspondence to: Carol P. Somkin, PhD, Kaiser Permanente Division of Research, 2000 Broadway, Oakland, CA 94612. E-mail: email@example.com.
1. Sateren WB, Trimble EL, Abrams J, et al. How sociodemographics, presence of oncology specialists, and hospital cancer programs affect accrual to cancer treatment trials. J Clin Oncol. 2002;20:2109-2117.
2. Tejeda HA, Green SB, Trimble EL, et al. Representation of African-Americans, Hispanics, and whites in National Cancer Institute cancer treatment trials. J Natl Cancer Inst. 1996;88:812-816.
3. McKenna RJ Jr. The impact of clinical trial protocols on patient care in a community hospital. Cancer. 1993;72:2828-2833.
4. Farrar WB. Clinical trials: access and reimbursement. Cancer. 1991;67:1779-1782.
5. Ho RCS. The future direction of clinical trials. Cancer. 1994;74:2739-2744.
6. Wagle N, Goldman DP, Kilgore M. Re: Surveys identify barriers to participation in clinical trials. J Natl Cancer Inst. 2001;93:238-239.
7. Friedman MA, Cain DF. National Cancer Institute-sponsored cooperative clinical trials. Cancer. 1990;65:2376-2382.
8. Christian MC, Trimble EL. Increasing participation of physicians and patients from underrepresented racial and ethnic groups in National Cancer Institute-sponsored clinical trials. Cancer Epidemiol Biomarkers Prev. 2003;12:277s-283s.
9. Brawley OW. The study of accrual to clinical trials: can we learn from studying who enters our studies? J Clin Oncol. 2004;22:2039-2040.
10. Chang SM, Barker FG, Schmidt MH, et al. Clinical trial participation among patients enrolled in the Glioma Outcomes Project. Cancer. 2002;94:2681-2687.
11. Lara PN Jr, Higdon R, Lim N, et al. Prospective evaluation of cancer clinical trial accrual patterns: identifying potential barriers to enrollment. J Clin Oncol. 2001;19:1728-1733.
12. Klabunde CN, Springer BC, Butler B, White MS, Atkins J. Factors influencing enrollment in clinical trials for cancer treatment. South Med J. 1999;92:1189-1193.
13. Shavers VL, Lynch CF, Burmeister LF. Factors that influence African Americans' willingness to participate in medical research studies. Cancer. 2001;91:233-236.
14. Shavers VL, Lynch CF, Burmeister LF. Racial differences in factors that influence the willingness to participate in medical research studies. Ann Epidemiol. 2002;12:248-256.
15. Swanson GM, Ward AJ. Recruiting minorities into clinical trials: toward a participant- friendly system. J Natl Cancer Inst. 1995;87:1747-1759.
16. Lovato LC, Hill K, Hertert S, Hunninghake DB, Probstfield JL. Recruitment for controlled clinical trials: literature summary and annotated bibliography. Controlled Clin Trials. 1997;18:328-357.
17. Trimble EL, Carter CL, Cain D, Freidlin B, Lingerleider RS, Friedman MA. Representation of older patients in cancer treatment trials. Cancer. 1994;74:2208-2214.
18. Thomas CR, Pinto HG, Roach M, Vaughn CB. Participation in clinical trials: is it state-of-the-art treatment for African Americans and other people of color? J Natl Med Assoc. 1994;86:177-182.
19. Roberson NL. Clinical trial participation: viewpoints from racial/ethnic groups. Cancer. 1994;74:2687-2691.
20. Shochat SJ, Fremgen AM, Murphy SB, et al. Childhood cancer: patterns of protocol participation in a national survey. CA Cancer J Clin. 2001;51:119-130.
21. Goodwin JS, Hunt WC, Humble CG, Key CR, Samet JM. Cancer treatment protocols. Who gets chosen? Arch Intern Med. 1988;148:2258-2260.
22. Kennedy BJ. Age-related clinical trials of CALGB. Cancer Control. 1995;2:14-16.
23. Kennedy BJ. Needed: clinical trials for older patients. J Clin Oncol. 1991;9:718-720.
24. Hutchins LF, Unger JM, Crowley JJ, Coltman CA Jr, Albain KS. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med. 1999;341:2061-2067.
25. Lewis JH, Kilgore ML, Goldman DP, et al. Participation of patients 65 years of age or older in cancer clinical trials. J Clin Oncol. 2003;21:1383-1389.
26. Kennedy BJ. Aging and cancer. Oncology (Huntingt). 2000;14:1731-1733.
27. Simon MS, Du W, Flaherty L, et al. Factors associated with breast cancer clinical trials participation and enrollment at a large academic medical center. J Clin Oncol. 2004;22:2046-2052.
28. Kemeny MM, Peterson BL, Kornblith AB, et al. Barriers to clinical trial participation by older women with breast cancer. J Clin Oncol. 2003;21:2268-2275.
29. Yee KW, Pater JL, Pho L, Zee B, Siu LL. Enrollment of older patients in cancer treatment trials in Canada: why is age a barrier? J Clin Oncol. 2003;21:1618-1623.
30. Moy E, Mazzaschi AJ, Levin RJ, Blake DA, Griner PF. Relationship between National Institutes of Health research awards to US medical schools and managed care market penetration. JAMA. 1997;278:217-231.
31. Thompson JN, Moskwitz J. Preventing the extinction of the clinical research ecosystem. JAMA. 1997;278:241-256.
32. Shine KI. Some imperatives for clinical research. JAMA. 1997;278:245-246.
33. Luft HS, Greenlick MR. The contribution of group- and staff-model HMOs to American medicine. Milbank Q. 1996;74:445-467.
34. Fishman PA, Wagner EH. Managed care data and public health: the experience of Group Health Cooperative of Puget Sound. Annu Rev Public Health. 1998;19:477-491.
35. Vogt TM, Elston-Lafata J, Tolsma D, Greene SM. The role of research in integrated healthcare systems: the HMO Research Network. Am J Manag Care. 2004;10:643-648.
36. Kaluzny AD, Lacey LM, Warnecke R, et al. Predicting the performance of a strategic alliance: an analysis of the Community Clinical Oncology Program. Health Serv Res. 1993;28:159-182.
37. McKinney MM, Morrissey JP, Kaluzny AD. Interorganizational exchanges as performance markers in a community cancer network. Health Serv Res. 1993;28:459-478.
38. Kaluzny A, Brawley O, Garson-Angert D, et al. Assuring access to state-ofthe- art care for US minority populations: the first 2 years of the Minority-Based Community Clinical Oncology Program. J Natl Cancer Inst. 1993;85:1945-1950.
39. Kaluzny AD, Ricketts T, III, Warnecke R, et al. Evaluating organizational design to assure technology transfer: the case of the Community Clinical Oncology Program. J Natl Cancer Inst. 1989;81:1717-1725.
40. Klabunde C, Kaluzny A, Ford L. Community Clinical Oncology Program participation in the Breast Cancer Prevention Trial: factors affecting accrual. Cancer Epidemiol Biomarkers Prev. 1995;4:783-789.
41. Puleo E, Zapka J, White MJ, Mouchawar J, Somkin C, Taplin S. Caffeine, cajoling, and other strategies to maximize clinician survey response rates. Eval Health Prof. 2002;25:169-184.
42. Emanuel EJ. 1999 ASCO Survey of Oncologists on Clinical Research. President's Panel, American Society of Clinical Oncologists Annual Meeting; May 1999; Atlanta, Ga.
43. Strauss A, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Newbury Park, Calif: Sage Publications; 1998.
44. Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Source Book. 2nd ed. Newbury Park, Calif: Sage Publications; 1994.
45. Fireman BH, Fehrenbacher L, Gruskin EP, Ray GT. Cost of care for patients in cancer clinical trials. J Natl Cancer Inst. 2000;92:136-142.
46. Wagner JL, Alberts SR, Sloan JA, et al. Incremental costs of enrolling cancer patients in clinical trials: a population-based study. J Natl Cancer Inst. 1999;91:847-853.
47. Emanuel EJ, Schnipper LE, Kamin DY, Levinson J, Lichter AS. The costs of conducting clinical research. J Clin Oncol. 2003;21:4145-4150.
48. Roche K, Paul N, Smuck B, et al. Factors affecting workload of cancer clinical trials: results of a multicenter study of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol. 2002;20:545-556.
49. Joffe S, Weeks JC. Views of American oncologists about the purposes of clinical trials. J Natl Cancer Inst. 2002;94:1847-1853.
50. Miller FG, Rosenstein DL. The therapeutic orientation to clinical trials. N Engl J Med. 2003;348:1383-1386.
51. Grunberg SM, Cefalu WT. The integral role of clinical research in clinical care. N Engl J Med. 2003;348:1386-1388.
52. Taylor KM, Feldstein ML, Skeel RT, Pandya KJ, Ng P, Carbone PP. Fundamental dilemmas of the randomized clinical trial process: results of a survey of the 1,737 Eastern Cooperative Oncology Group investigators. J Clin Oncol. 1994;12:1796-1805.
53. McDermott MM, Hahn EA, Greenland P, et al. Atherosclerotic risk factor reduction in peripheral arterial disease: results of a national physician survey. J Gen Intern Med. 2002;17:895-904.
54. Yarzebski J, Bujor CF, Goldberg RJ, Spencer F, Lessard D, Gore JM. A community- wide survey of physician practices and attitudes toward cholesterol management in patients with recent acute myocardial infarction. Arch Intern Med. 2002;162:797-804.
55. Drass J, Kell S, Osborn M, et al. Diabetes care for Medicare beneficiaries. Attitudes and behaviors of primary care physicians. Diabetes Care. 1998;21:1282-1287.