Jennifer A. Palmer, MS

Despite evidence that medication adherence can influence cost-effectiveness analysis (CEA) results, the extent to which published CEAs include adherence has not been fully characterized.

To characterize inclusion of patient adherence in CEAs of self-administered medications and to examine whether industry sponsorship affects adherence inclusion, because adherence exclusion might overstate the interventions’ costeffectiveness. 

Systematic review of the Englishlanguage medical literature published between 1998 and 2003 identified 177 original CEAs of self-administered medications. 

Two trained readers independently abstracted data. Adherence inclusion was estimated overall and by study characteristics. Predictors of inclusion were assessed with c2 tests and logistic regression.

Among 177 CEAs, 30.5% explicitly modeled adherence; of these, only half modeled adherence in both base-case and sensitivity analyses. Only 21% of studies performed sensitivity analysis on adherence; fewer than half of these provided sufficient information to determine the impact on results. Of the remaining 20 studies, 9 were sensitive to adherence. Adherence inclusion varied across clinical areas (

= .022). Only 30% of chronic anticoagulation studies, 52% of cardiovascular risk reduction studies, 38% of neuropsychiatric studies, and 32% of HIV antiretroviral studies considered suboptimal adherence. Among 128 CEAs that disclosed study sponsorship, adherence was included in 25.4% of industry-sponsored and 35.1% of non—industrysponsored studies (

Few CEAs modeled suboptimal medication adherence. As CEAs are meant to model “real world” costs and effects of interventions, investigators would do well to explicitly consider medication adherence in the future.

Suboptimal medication adherence remains pervasive in the US healthcare system, resulting in substantial adverse health and economic effects. Despite a growing body of evidence that adherence can influence cost-effectiveness analyses (CEAs), the extent to which published CEAs include adherence has not been fully characterized.

Fewer than one-third of published CEAs modeled suboptimal adherence. 

In the small subset of studies that varied adherence in sensitivity analyses, the impact on cost-effectiveness often was substantial. 

With payers and policymakers increasingly turning to these studies for guidance, failure to account for nonadherence may lead to suboptimal resource allocation strategies.

Medication adherence is perhaps the most important determinant of the therapeutic effectiveness of medications.

 Yet suboptimal adherence remains pervasive in the US healthcare system, resulting in substantial adverse health and economic effects.

 Overall, rates of nonadherence are high—fewer than 50% of patients with chronic diseases take their medications as prescribed.

 In turn, poor adherence is associated with disease progression, complications, avoidable hospitalizations, and death,

 as well as increased medical costs for some chronic diseases.

 Indeed, it has been estimated that suboptimal adherence is responsible for an estimated $177 billion in total direct and indirect health spending annually in the United States.

Given the substantial impact of adherence on health and spending, inclusion of adherence in economic evaluations may be critical for informing policymakers about the value of therapies in actual practice. Indeed, past studies have shown that adherence can influence the results and recommendations of cost-effectiveness analyses (CEAs).

 Despite the importance of adherence, however, the extent to which published CEAs consider it has not been fully characterized. Our objective was to examine the inclusion of patient adherence in CEAs of self-administered medications. Because the exclusion of adherence might overstate the cost-effectiveness of evaluated interventions, we also examined the relationship between pharmaceutical company sponsorship and inclusion of adherence.

Cost-effectiveness analyses compare alternative approaches to a health problem to determine the relative costs and benefits of each. Cost-utility analyses, the subset of CEAs with health benefits measured in quality-adjusted life-years (QALYs), represent the gold standard for conducting and reporting of CEAs in medicine.

 For the remainder of this article, we use CEA to refer to cost/QALY studies.

We previously reported on the development of a comprehensive registry of CEAs published in the public health and medical literature from 1976 through 1997.

 The Tufts cost-effectiveness analysis registry is continuously updated and maintained on the Internet as a free, public use resource (http://www.cearegistry.org). As part of the updates to this registry, we conducted a systematic review of the English-language medical literature published between 1998 and 2003. We searched MEDLINE for the following medical subject headings and text keywords: “qualityadjusted,” “quality-adjusted life year(s),” “QALY,” “adjusted life year(s),” “cost-utility,” and several variations of these terms. To identify potentially overlooked studies, our search findings were validated to those of the Health Economic Evaluation Database maintained by the British Office of Health Economics.

 Our search identified 2528 candidate articles published between 1998 and 2003. We then screened titles and abstracts to remove noneligible articles, such as methodologic and review articles, and studies that did not use cost/QALY as their main outcome measure. A final list of 568 published cost-utility analyses were included in the registry for the years 1998 through 2003.

Of the 568 cost-utility analyses, 226 had a medication component in the intervention or comparator. To restrict analyses to studies in which patient adherence was applicable, we excluded studies in which medications were administered by injection or infusion (n = 27), in an acute-care setting (n = 14), or by someone else (often the physician) for other reasons (n = 8). The final sample included 177 CEAs of self-administered medications (available at https://research.tufts-nemc. org/cear/related/ajmcappendix.pdf). Each article was reviewed by 2 independent readers, who abstracted select data elements from each article; consensus meetings were held to review and resolve discrepancies. We collected data on a wide variety of elements related to the methods, results, and quality of reporting in the article, along with data on study inclusion of adherence to therapies. In addition, we abstracted data on the source of funding for each study (when reported).

. Readers recorded whether medication adherence was (1) included in the base-case analysis only, (2) included in sensitivity analyses only, (3) included in both the base-case and sensitivity analyses, (4) mentioned in the manuscript but not included in analyses, or (5) not mentioned. If the results of sensitivity analyses on adherence were reported in the text, tables, figures, or an appendix, they were considered “sufficient” to judge the impact of adherence. Of studies with sufficient data, we report whether the authors describe the results as sensitive to adherence. In practice, adherence has many different definitions, measurement approaches, and even names (eg, compliance, persistence).

 For the purposes of this analysis, we included any form of adherence regardless of whether or how it was defined.

. Readers abstracted data from each study on the expected duration of intervention therapy, study sponsorship, and the clinical condition of interest. Intervention medications were classified as short term if therapy duration was expected to be less than 1 month. Study sponsorship was categorized as unknown if no funding source was listed, industry-sponsored if at least 1 funding source or 1 author’s affiliation was a pharmaceutical or medical device company, and non—industry-sponsored if all funding came from governments, foundations, or other nonindustry sources. Study settings included the United States, the United Kingdom, Canada, and an “other country” category that included any country that served as the CEA setting in 10 or fewer studies. Studies were grouped into the following clinical categories: cardiovascular risk factors and disease (CVD), cancers, HIV and AIDS, other infectious diseases, hepatitis, other gastrointestinal conditions,  musculoskeletal conditions (including osteoporosis), neurologic conditions, psychiatric conditions, pulmonary diseases, and an “other disease” category that included any clinical conditions examined in fewer than 5 CEAs. Additionally, a separate anticoagulation category was included because of the large number of CEAs published in this area. To assess whether the inclusion of adherence in CEAs has improved over time, we also created a variable indicating whether the study was published earlier (1998-2000) or later (2001-2003) in the 6-year study window.

We examined the frequency with which adherence was included in CEAs, as well as the extent to which it was included. The kappa interrater reliability score for adherence inclusion was 0.84, representing strong agreement between reviewers.

 Associations between adherence inclusion and duration of therapy, study sponsorship, publication window, study setting (country), and disease categories were assessed with 

 tests. Logistic regression was performed with inclusion of adherence in either the base-case or sensitivity analyses as the dependent variable, and therapy duration, study sponsorship, publication window, and disease category as independent variables. Analyses were conducted using SAS version 9.1 (SAS Institute, Inc, Cary, NC).

Among the 177 CEAs of self-administered medications, only 30.5% (n = 54) explicitly modeled patient adherence to therapy, whereas an additional 16.4% mentioned but did not model medication adherence (

). Of the 54 studies that did model adherence, only half (14.7% of all studies, or 26 studies) modeled adherence in both the base-case and sensitivity analyses. In those studies that included adherence in the base case, the most common base-case assumption was that medication nonadherence and/or discontinuation rates were the same as those rates reported in clinical trials. Adherence was equally likely to be considered in CEAs of long-term versus short-term drug therapy (29.6% vs 30.3%; not significant).

Study inclusion of adherence varied significantly across clinical areas (

= .022). Only 30% of chronic anticoagulation studies, 52% of cardiovascular risk reduction studies, 38% of neuropsychiatric studies, and 32% of HIV antiretroviral studies included suboptimal adherence in either the base-case or sensitivity analyses. Figure 2 shows the extent to which adherence inclusion varied across clinical areas.

Eighty-four CEAs were set in the United States, 23 in the United Kingdom, 19 in Canada, and 10 or fewer in all other countries. In 35% of studies set in the United States, 30% of those set in the United Kingdom, 32% in Canada, and 25% of studies set in other countries, adherence was included in either the base-case or sensitivity analyses (differences were nonsignificant).

Study sponsorship came from industry in 40.1% (n = 71) of CEAs, from sources other than industry in 32.2% (n = 57), and could not be determined in the remaining 27.8% (n = 49) of studies. Among the 128 CEAs for which study sponsorship was disclosed, 25.4% of industry-sponsored studies and 35.1% of non—industry-sponsored studies incorporated adherence into their models (

= .17). There was no change over time in the inclusion of adherence (31% for both time periods) in CEAs, and no significant difference over time in whether industry-sponsored versus non—industry-sponsored studies included adherence.

Among the 38 CEAs that performed sensitivity analysis on adherence, nearly half (47%, n = 18 studies) did not provide sufficient information in the study methods or results for reviewers to determine whether varying adherence had a significant impact on the cost-effectiveness ratios. Among the 20 remaining CEAs, 45% (n = 9) were reported as sensitive to changes in adherence, whereas 55% (n = 11) were not.

Despite the potential importance of medication adherence, few CEAs incorporate it into analyses, although the level of incorporation varies substantially across clinical areas. Of the small subset of CEAs that varied adherence in sensitivity analyses, half provided insufficient information for the reader to clearly discern whether or how adherence impacted the cost-effectiveness ratios, and close to half of the remainder reported that results were sensitive to adherence. Although industry-sponsored studies included adherence less frequently than did non—industry-sponsored studies, this finding was not statistically significant, perhaps because of small sample sizes.

Although prior studies have indicated the important impact suboptimal adherence may have on cost-effectiveness,

 this is the first study to our knowledge to document the extent to which adherence is actually included in published CEAs. The low rate of adherence inclusion was surprising given the substantial health and economic impact of medication nonadherence in practice. Further, even when studies performed sensitivity analysis on adherence, insufficient reporting of the methods and/or results precluded determination of the true sensitivity of the results to medication adherence.

Among the CEAs that did model adherence, their assumptions about the level of adherence were frequently not consistent with the published literature. Many studies, for example, used adherence rates from clinical trials, which notoriously overestimate adherence found in the general population. Still others modeled adherence levels that were higher than those reported in the literature. For example, a study by Marchetti and colleagues reported that screening for factor V Leiden followed by 2 years of anticoagulation therapy was cost-effective compared with standard 6-month therapy for individuals diagnosed with first deep venous thrombosis.

 However, the base-case results ($12,833/QALY) supporting this conclusion relied on an assumption of 100% warfarin adherence, and sensitivity analyses (which varied adherence from 85% to 100%) indicated that this strategy only remained cost-effective (defined as a ratio below $50,000 per QALY) as long as warfarin adherence remained above 94%.

 However, the literature suggests that warfarin adherence in practice falls far below that threshold; in a recent warfarin adherence study, patients were nonadherent (ie, took fewer than 80% of prescribed doses) 36% of the time.

Our findings were consistent with those of Hughes and colleagues,

 which suggested that the impact of suboptimal adherence on the cost-effectiveness of medications can be substantial. Indeed, several of the CEAs reviewed for this study confirmed the critical impact of adherence on medications’ cost-effectiveness or lack thereof. A study by Mar and Rodriguez-Artalejo demonstrated large variations in the cost-effectiveness of hypertension treatment by patient age, sex, hypertension stage, and drug class used; however, treatment adherence had by far the biggest impact on health benefits and, in turn, cost-effectiveness.

 Phillips and colleagues demonstrated that increasing beta-blocker use after myocardial infarction would lead to impressive health gains and potential cost savings.

 Clark and colleagues demonstrated that Canadian provincial coverage of angiotensin-converting enzyme inhibitors for patients with type 1 diabetes with nephropathy could be highly cost-effective but this cost-effectiveness hinged critically on adherence; indeed, the costeffectiveness ranged from a savings of $899 to expenditures of more than $1 million per QALY with adherence rates of 67% and 51%, respectively.

Recognizing the importance of patient adherence for the value of medical spending, a number of private payers have begun to overhaul their prescription drug benefit designs. Rather than setting enrollees’ cost-sharing levels based on the cost of the drugs, these payers have linked copayments to the value (ie, cost-effectiveness) of each therapy. These so-called value-based insurance designs

 are premised on the fact that cost-sharing should neither inhibit access nor discourage adherence to high-value medications. Although these experiments are generally too recent to be fully evaluated, a number of studies that model copayment changes (and the subsequent increase in adherence) have generated a number of important findings. A recent study by Rosen  and colleagues, for example, found that Medicare first-dollar coverage of a giotensin-converting enzyme inhibitors for beneficiaries with diabetes appeared to both extend life and reduce Medicare program costs. The long-term health and economic benefits to Medicare far surpassed the initial increase in medication spending because of improved patient adherence.

 A subsequent study by Choudhry and colleagues suggested that a typical insurer could prevent heart attacks, saving lives and money, if the insurer paid the full costs of secondary prevention medications after myocardial infarction.

 A clear understanding of the way in which adherence moderates the cost-effectiveness of therapies (and the way in which costsharing affects adherence) is critical to efficient targeting of these incentive programs. Including adherence information in CEAs will considerably help with this first aim.

Because adherence can play such an important role in CEAs, why is it not addressed more often in these studies? One possibility is the historical backdrop for CEAs. They first started appearing in the medical literature in the early 1970s, which was a time when the magnitude and import of suboptimal adherence to therapy were not as well recognized or characterized as they are today. In turn, the most widely cited standards for performing and reporting CEAs, those developed by the Panel on Cost-Effectiveness in Health and Medicine in 1996,

 do not explicitly recommend modeling suboptimal adherence in CEAs, potentially allowing journal editors, reviewers, and authors to implicitly overlook adherence. This problem is compounded by the absence of standard definitions of and measurement standards for adherence. Recently, Hughes and colleagues at the International Society for Pharmacoeconomics and Outcomes Research published a set of proposed standards for incorporating adherence into economic evaluations,

 but it is too early to assess the impact of these recommendations on actual practice. Another possible reason for the poor inclusion of adherence in CEAs is the source of data. Although CEAs often are used to inform real-world practice, their estimates of effect most often come from clinical trials, which demonstrate efficacy rather than effectiveness. The recent COURAGE trial provides a perfect example of just how different these constructs can be. Of participants enrolled in COURAGE (individuals with stable coronary artery disease), more than 90% (in both arms) remained on beta-blockers, statins, and aspirin at 5 years.

 By contrast, in the real world, 50% of individuals with coronary artery disease stop these therapies by 2 years.

Our study had several limitations. First, we only examined CEAs that reported outcomes in cost/QALY; therefore, our findings may not fully represent the handling of adherence by the broader universe of cost-effectiveness studies. Second, we may have overestimated the number of studies that included adherence by very liberally defining inclusion of adherence. For example, if a study explicitly stated that it addressed adherence by using intention-to-treat estimates from a clinical trial, it was considered to include adherence. On the other hand, if researchers used clinical trial estimates of effect but did not mention or discuss adherence anywhere in the paper, the study was considered not to include adherence. Third, because this analysis was restricted to self-administered medications, the inclusion of adherence in nonpharmaceutical CEAs remains to be investigated.

Finally, it is important to recognize that traditional sensitivity analyses of adherence may not be the only or even the most appropriate way to consider the relationship between adherence and cost-effectiveness. Sensitivity analyses can capture how uncertainty about adherence impacts uncertainty in the incremental cost-effectiveness ratio. However, even if adherence were known for certain, it is a modifiable parameter. Interventions (eg, reducing copayments, providing medication counseling) can increase adherence through the investment of additional resources. As such, sensitivity analyses may be far more informative

if CEAs include both adherence and the relationship between additional spending and changes in adherence.

This study demonstrates how infrequently medication adherence is addressed in published CEAs. As national efforts to contain rising healthcare costs via increased consumer cost-sharing move forward, a profound impact on medication adherence is likely.

 Because adherence can so markedly impact cost-effectiveness, it has become more important than ever to adequately address suboptimal adherence in CEAs. With payers and policymakers increasingly turning to these studies for guidance, failure to account for nonadherence may lead to suboptimal resource allocation strategies, worsening the value of healthcare spending in a system that is already plagued with inefficiencies.

The authors would like to thank Patricia Stone, PhD, for thoughtful comments on project conceptualization, and Susan Goold, MD, and Justin Timbie, PhD, for thoughtful comments on an earlier version of this manuscript.

From the Departments of Internal Medicine and Health Management and Policy (ABR), University of Michigan, Ann Arbor, MI; the Division of Epidemiology and Community Health (ABS), University of Minnesota, Minneapolis, MN; the Department of Health Systems Management (DG), Ben Gurion University of the Negev, Beer-Sheva, Israel; and the Center for the Evaluation of Value and Risk in Health (JAP, PJN), Tufts Medical Center, Boston, MA.

This study was funded by National Library of Medicine grant 1G08LM008413-02. Dr Rosen was supported by National Institutes of Health grant K12RR17607.

The authors (ABR, ABS, DG, JAP, PJN) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. Dr Rosen had full access to all study data and takes responsibility for the integrity of the data and the accuracy of the data analyses.

Concept and design (ABR, ABS, DG, PJN); acquisition of data (ABR, ABS, DG, JAP, PJN); analysis and interpretation of data (ABR, ABS, DG, PJN); drafting of the manuscript (ABR, ABS, DG); critical revision of the manuscript for important intellectual content (ABR, ABS, DG, JAP, PJN); statistical analysis (ABR); provision of study materials or patients (JAP); obtaining funding (ABR); and administrative, technical, or logistic support (ABS).

Allison B. Rosen, MD, MPH, ScD, Departments of Internal Medicine and Health Management and Policy, University of Michigan, 300 N Ingalls, Ste 7E10, Ann Arbor, MI 48109. E-mail: abrosen@umich.edu.

1. National Council on Patient Information and Education.

 Enhancing Prescription Medicine Adherence: A National Action Plan. August 2007. http://www.talkaboutrx.org/documents/enhancing_prescription_medicine_adherence.pdf. Accessed January 19, 2007.

Articles

A systematic literature review from 1998 to 2003 showed that few cost-effectiveness analyses of self-administered medications model suboptimal medication adherence.

Patient Adherence: A Blind Spot in Cost-Effectiveness Analyses?

The Panel on Integrating Cost-Effectiveness Considerations into Health Policy Decisions, composed of medical and pharmacy directors at public and private health plans, was convened to (1) explore the views of health plan purchasers about cost-effectiveness analysis (CEA) and (2) to develop a strategic plan for policymakers to address obstacles and to integrate CEA into health policy decisions, drawing on stakeholders as part of the solution. Panelists expressed strong support for a greater role for CEA in US health policy decisions, although they also highlighted barriers in the current system and challenges involved in moving forward. The strategic plan involves a series of activities to advance the use of CEA in the United States, including research and demonstration projects to illustrate potential gains from using the technique and ongoing consensus- building steps (eg, workshops, conferences, town meetings) involving a broad coalition of stakeholders. Funding and leadership from policymakers and nonprofit foundations will be needed, as well as the active engagement of legislators and business and consumer groups. Panelists emphasized the importance of the Medicare program taking a lead role, and the need for new â€œinfrastructure,â€ in the form of either a new institute for conducting research or increased funding for existing institutions.

A panel composed of medical and pharmacy directors at public and private health plans developed a strategic plan for incorporating cost-effectiveness analysis (CEA) into US health policy decisions.

The plan has 3 long-term goals: increasing use of CEA by the Centers for Medicare & Medicaid Services; creating infrastructure to support research, either at the Agency for Healthcare Research and Quality or at a new independent institute; and integrating CEA into other public and private initiatives.

The plan includes a series of activities to achieve these goals, including consensus-building steps with a broad coalition of stakeholders, developing research on CEA application, and implementing a legislative strategy.

Understanding resistance to cost-effectiveness analysis (CEA) in the United States has been the topic of numerous papers, books, and conferences.

 The Panel on Integrating Cost- Effectiveness Considerations into Health Policy Decisions was convened to build upon these efforts in 2 ways: (1) to explore the views of health plan purchasers on the value and use of CEA in their organizations and (2) to develop a strategic plan for policymakers to address obstacles and to integrate CEA into health policy decisions, drawing on stakeholders as part of the solution.

The panel, consisting of medical and pharmacy directors at public and private health plans, addressed this charge at a conference in Gaithersburg, Maryland, in May 2006. Members were selected based on their experiences in coverage and benefit-design decisions, and with an eye toward assembling a diverse group with respect to geography, type of health plan, and professional background. This commentary integrates discussion at the conference and the deliberations that followed.

The panel agreed that formal CEA can, in theory, provide policymakers with a useful technique for informing coverage and reimbursement decisions. Panel discussions also underscored the uneven acceptability of CEA in the United States. Although some initiatives have incorporated CEA,

 formal use of the approach is limited. Panelists noted that although their own organizations are actively engaged in efforts to deliver care more efficiently, they generally do not use CEA explicitly. Instead, policies focus on evidence-based medicine, cost-sharing, disease management, consumer-driven care, and pay-for-performance initiatives.

The strategic plan is offered as a means to address the obstacles and to move forward on CEA integration. Cultural acceptance for CEA likely will involve a long-term and gradual process, but can advance through active consensus building and experience. Panelists underscored that Medicare, as the largest health payer in which all citizens have a stake, is a logical agency to assume a leading role. The incomplete evidence base and lack of infrastructure can be helped by expanded resources devoted to CEA research, either at a new agency or at the Agency for Healthcare Research and Quality. Such an entity also can provide a standing forum for addressing legal and ethical issues. The conflicting incentives that private plans have for using CEA is a perpetual challenge, but other elements of the plan (eg, greater experience with CEA at the national level, an improved evidence base) could smooth its use in the private sector.

Several premises underlie this strategic plan. First, CEA is one input into decisions and can and should be used flexibly. Second, CEA is a tool for achieving better value for our healthcare system, rather than simply a cost-containment tool or a means of denying care. Third, CEA is an approach that can exist alongside other initiatives for cost control and quality improvement, and even enhance their design. Finally, conventional wisdom about the lack of acceptability of CEA in the United States among the American public has never truly been tested.

Panelists envisioned 3 phases of implementing the strategic plan, each of which incorporates specific steps: building consensus and support; transitioning to CEA; and integrating CEA (

Phase I: Building Consensus and Support (1-2 Years).

 Phase I, covering roughly 1-2 years, focuses on laying the groundwork. Key to this phase will be identifying leadership among legislators, policymakers, and stakeholders representing business, consumer, employers, and health plans. Nonprofit foundations can serve in a critical catalyst role by providing seed money to fund research projects and consensus- building steps.

The research projects should include white papers on precedents for using CEA abroad and in selected cases in the United States, as well as empirical estimates showing the potential gains in terms of increased health and decreased costs from using the technique. In early stages, the Centers for Medicare & Medicaid Services (CMS) should undertake demonstration and research projects to explore the potential value of CEA and to experiment with approaches for its implementation. Consensus-building steps should focus on educating stakeholders about the potential role and flexibility of CEA. Ideally, steps would include the convening of public forums, conferences, and workshops to help create and communicate an explicit framework for considering resource allocation decisions.

Phase II: Transitioning to CEA (2-5 Years).

 In Phase II, covering roughly 2-5 years, the strategic plan involves learning from the research and demonstration projects developed in Phase I and solidifying support for CEA integration. This work would include researchers disseminating and publicizing findings of the Phase I projects and ongoing consensus building in the form of workshops and conferences on methodologic, legal, ethical, and private-sector application issues related to CEA.

Phase II also would include the development and implementation of a dedicated legislative strategy to lay the groundwork for longer-term integration. The strategy would include plans for targeting key legislators and staffers; building support among a broad coalition of groups including employers, health plans, patients, and health professionals; enlisting academic champions; and holding congressional hearings on the issue. Importantly, CEA could be integrated into ongoing efforts to provide resources and possibly to create a new independent entity to conduct  comparative-effectiveness research.

 In Phase III, integration of CEA could occur at 3 levels: legislating Medicare&#8217;s use of CEA for coverage decisions, expanding infrastructure for CEA research, and integrating CEA into other public and private initiatives.

Congress could eventually authorize in statute the criteria that Medicare uses in covering new technologies, including CEA considerations. When legislating new benefits to the Medicare program, Congress could direct CMS to determine the most cost-effective strategy for implementing the new service. CMS also could incorporate CEA into other areas, including its use to inform payment rates and incentive-based programs such as its pay-for-performance initiative.

Further, Congress could expand infrastructure and funding for CEA through existing institutions such as the Agency for Healthcare Research and Quality or a new independent public/private institute (possibly piggybacking on ongoing efforts to develop comparative-effectiveness research, which in proposals to date focus on clinical effectiveness and exclude costeffectiveness considerations).

 New research findings could provide guidance to public and private healthcare payers on the cost-effectiveness of drugs, medical devices, diagnostic techniques, and medical and surgical procedures, and the clinical management of specific conditions. Expanded resources also could be used to provide direction on methodologic standards and key challenges for the field, thus helping to improve the quality of evidence and acceptability of the approach.

Finally, panel members envisioned that, in the longer run, this coordinated strategy would have important spillover effects at numerous levels. These effects include increased funding for CEA at government agencies such as the National Institutes of Health and the Centers for Disease Control and Prevention, and further use of CEA at the state level and in the private sector within formulary committees and technology assessment agencies.

Successful execution of the plan will begin with broad recognition that greater use of CEA can help the nation face its challenges in paying for effective healthcare. Implementation will require time, resources, and above all, political will. Support and action will require the involvement of multiple stakeholders.

Achieving the plan&#8217;s objectives is likely to be a long-term process. The plan involves early concrete steps and successes that can lead to further goals and accomplishments. The time frame presented is offered as a guideline. It will take time to build consensus and to foster an accommodating political environment. Many details on tactics and resources will need to be filled in. CEA will not solve all of the healthcare system&#8217;s problems, but it can be a key ingredient. This plan is offered as a start down that road.

Using Cost-Effectiveness Analysis to Improve Health Care: Opportunities and Barriers

. New York: Oxford University Press; 2005.

. Establishing a health technology assessment program. 2007. 

http://apps.leg.wa.gov/billinfo/summary.aspx?year=2006&bill=2575

6. Greenberg D, Rosen AB, Olchanski NV, Stone PW, Nadai J, Neumann PJ

. Delays in publication of cost utility analyses conducted alongside clinical trials: registry analysis. 

. Cost-effectiveness analysis in the courts: recent trends and future prospects. 

. Developing a center for comparative effectiveness information. 

A panel composed of medical and pharmacy directors expressed strong support for a greater role for cost-effectiveness analysis in US health policy decisions.

A Strategic Plan for Integrating Cost-effectiveness Analysis Into the US Healthcare System

 To investigate the quality and complete-ness of clinical and economic data in dossiers submitted by drug companies to a health plan using Academy of Managed Care Pharmacy guidelines (the Format) for formulary submissions. 

 We reviewed the quality of economic analyses in dossiers submitted to Premera Blue Cross Health Plan (Mountlake Terrace, Washington; enrollment 1.6 million) between January 2002 and September 2005. For dossiers submitted in 2003, we examined the clinical studies included.

 Dossiers were audited with a data collection form to judge the types of clinical studies used to support labeled and off-label indications, and the quality and transparency of economic analyses. We compared economic analyses for high-cost (30-day treatment cost > $1000) versus low-cost products, and for "innovative" versus "me-too" drugs.

 Evidence to support off-label indications often was included in 2003 dossiers, but the information was less extensive and of poorer quality than data for labeled indications. Of 115 dossiers submitted between 2002 and 2005, 53 (46%) included economic analyses. The economic analyses had low levels of compliance with standards: only 43% performed sensitivity analysis; 38% stated the study perspective; 37% discussed relevant treatment alternatives; 20% stated assumptions clearly; and 18% mentioned caveats to conclusions. Economic analyses of high-cost products and innovative products had higher compliance with recommended practices.

 Drug companies are submitting dossiers of evidence to formulary committees. Dossiers often included clinical data to support off-label indications, but concerns persist about their quality. About half of dossiers included economic analyses, but these analyses had relatively low levels of compliance with recommended practices.

Dossiers being submitted under AMCP Format provide an opportunity for health plans to evaluate a drugâ€™s clinical and economic evidence. However, concerns persist about the quality of the clinical data and the economic analyses in dossiers; therefore, plans should:

Be vigilant and cautious when reviewing clinical and economic evidence in dossiers, especially with lower-cost and â€œme-tooâ€ drugs.

Train formulary committee members regarding the potential value of the Academy of Managed Care Pharmacy guidelines (the Format).

Formalize a process for providing feedback to pharmaceutical companies on dossier quality and transparency.

In recent years, health plan officials have worked to standardize and improve the processes by which they decide whether and how to list new drugs for formularies, with the goal of grounding decisions in stronger clinical and economic evidence. The trend reflects 2 broader movements in healthcare, one toward evidence-based medicine and the other toward explicit consideration of cost-effectiveness or "value for money" arguments.

The movement toward formulary guidelines has evolved over the past decade. Although formulary committees have used clinical and pharma-coeconomic data to aid their decision-making processes for some time, the information was used informally, and great variation existed across plans.

With rapid increases in drug spending in the 1990s, public and private health plans as well as pharmacy benefit management (PBM) companies began to use formularies more aggressively in an attempt to contain costs.

This growing reliance on formularies, combined with the increase in the availability and acceptance of cost-effectiveness analyses, led to the issuance of new guidelines that called for a standardized process and format for the submission of clinical and economic evidence to support consideration of formulary listing in private health plans.

In 2000, the Academy of Managed Care Pharmacy (AMCP), a national professional society of pharmacists in managed care environments, adapted, revised, and then endorsed guidelines called the AMCP Format for Formulary Submission (hereafter, the Format), and began actively encouraging health plans nationwide to implement them.

The Format urges health plans to request formally that drug companies present a standardized dossier, which contains detailed information not only on the drug's effectiveness and safety for indications approved by the US Food and Drug Administration (FDA), but also on off-label indications and on the drug's economic value relative to alternative therapies. The Format also recommends that formulary committees establish a process for the transparent and efficient communication between drug companies and health plans (eg, sending an unsolicited request letter to drug firms 6 months before the product is placed on the pharmacy and therapeutics [P&T] committee agenda for a decision).

Many health plans, PBMs, and some hospitals have adopted the Format or a Format-like process.

Reports have indicated that most of these organizations use the clinical sections of the dossier and that some are beginning to consider meaningful cost-effectiveness analyses when these analyses are provided.

Little is known, however, about the information being submitted. Our objective was to evaluate the quality and completeness of clinical and economic information contained in dossiers adhering to the AMCP Format that were submitted to a health plan.

We audited all dossiers submitted to the Premera Blue Cross Health Plan in Mountlake Terrace, Washington (enrollment 1.6 million) between January 2002 and Septem-ber 2005. Each dossier audit was conducted by a physician researcher with graduate training in biostatistics, decision analysis, and cost-effectiveness analysis. Complete audits required approximately 4 hours per dossier. Dossiers were analyzed in either paper or password-protected CD-ROM form as submitted by pharmaceutical manufacturers to Premera.

  We developed a data collection form with more than 80 variables. The form was pilot-tested several times for clarity. The form included numerous items related to the characteristics of clinical studies used as evidence to support labeled indications (consistent with the FDA's regulatory approval) and off-label claims in dossiers, and variables related to the quality and transparency of economic analyses. 

Clinical studies included published or unpublished abstracts, posters, book chapters or sections, edited summaries, and full-text reprints of articles related to therapeutic uses of pharmaceutical products. We limited the detailed clinical analyses to the subsample of dossiers submitted in 2003 because of the large number of clinical studies included in dossiers (almost 400 studies in 2003 alone). We collected data on variables such as the research design used in the supporting studies, the sample size in those studies, and whether the study was published in a peer-reviewed journal.

  We defined economic analyses to include research studies or mathematical models that combined clinical and economic data to estimate the economic value of a drug for an indication. We excluded budget impact models, which examine the impact of a drug on a health plan's budget rather than the economic value of a drug. We investigated the extent to which economic analyses adhered to key recommendations for good practices in the field.

We hypothesized that clinical studies included or referenced in dossiers to support FDA-approved indications would be more extensive and have more rigorous study designs than studies supporting off-label claims. We also hypothesized that economic analyses in dossiers to support higher-cost and more innovative products would be of higher quality, because they would attract more resources and attention from drug companies seeking favorable formulary listing. We defined "high cost" as a cost per 30-day treatment higher than $1000 (based on the drug's price for that dossier's year).

The value $1000 was chosen arbitrarily as a round number that would reflect particularly expensive products (eg, biologics). 

An "innovative" drug was defined by our researchers as a drug in a new therapeutic class or a drug with a clear-cut, relevant therapeutic advantage compared with competitors in its class (as opposed to a "me-too" drug). We considered therapeutic edge based on our judgments about clinical, quality-of-life, safety, or dosing advantages. "Me-too" products were judged to have no substantial advantages on these dimensions. We hypothesized that the quality of economic analyses would improve over time, as drug industry and health plan officials gained expertise. We examined whether adherence to various quality indicators improved over the years covered by the study period time using logistic regression analyses with adherence as the dependent variable, adjusting for the presence of high-cost and innovative products.

All researchers involved in data collection signed confidentiality agreements before being granted access to dossiers. In addition, signature of confidentiality agreements compliant with Health Insurance Portability and Accountability Act of 1996 guidelines was required before the data collector was allowed to enter Premera's premises. No data are reported on specific manufacturers or brands.

shows the characteristics of 396 clinical studies contained in 35 dossiers submitted to Premera in 2003. Considerably more studies were submitted to support FDA-approved indications than off-label indications (314 vs 82). Proportionately more studies supporting on-label claims than off-label claims had random treatment allocation (50% vs 33%; 

= .01), were double-blinded (43% vs 27%; 

= .01), or were meta-analyses (8% vs 1%; 

= .02). They also contained many more full-text reprints (19% vs 6%; 

= .00) and larger sample sizes (492 vs 280; 

= .00); these differences were significant. In contrast, more of the studies supporting off-label claims were open-label clinical trials (27% vs 16%; 

= .02) or uncontrolled clinical trials (24% vs 6%; 

Between January 2002 and September 2005, 115 dossiers were submitted to Premera, of which 53 dossiers (46%) contained economic data. The 53 economic-content analyses represented 28 drug manufacturers. These dossiers included 106 separate economic analyses (some dossiers contained more than 1 economic analysis, each pertaining to different clinical indications or including different types of studies). Within the 106 economic analyses, there was notable variation in the methods used (

). For example, 82% stated the research question and 77% stated the primary outcome measure in the evaluation. Relatively few analyses reported sensitivity analyses (43%), discussed all relevant comparators (37%), clearly stated all assumptions (20%), accompanied conclusions with appropriate and specific caveats (18%), reported productivity changes (13%), or reported that comparators might be superior given changes in assumptions (8%).

Methods for conducting sensitivity analyses also varied considerably (

). Although 63% included a statement about the range over which parameters were varied, less than half varied the cost of the drug (44%) or the direct medical costs (39%), and few varied the time horizon (7%).

Adherence to methodologic standards was generally higher in economic analyses of high-cost products (

). Analyses of high-cost products were more likely to discuss all relevant comparators (75% vs 32%; 

= .01), state the form of economic evaluation used (92% vs 54%; 

= .01), provide details about the analysis/model design (100% vs 65%; 

= .02), and clearly state the model assumptions (42% vs 17%; 

= .06). Economic analyses of innovative products had better compliance to standards than economic analyses of me-too products, for example, in discussing all relevant comparators (81% vs 29%; 

= .00), stating the form of economic evaluation used (88% vs 53%; 

= .01), clearly stating all assumptions (44% vs 16%; 

= .02), and reporting the use of sensitivity analyses (75% vs 38%; 

We found no evidence of improvement in the quality of economic analyses in dossiers over time. Of the 23 items listed in Tables 2 and 3, only 2 (statement of ranges over which parameters were varied in sensitivity analyses and cost of drugs varied in sensitivity analyses) showed significant improvement over the study period after adjusting for the presence of high-cost and innovative drugs. For 7 of the items, the percentage of analyses adhering to good methodologic practices showed significant declines over time, including statement of the viewpoint of the analysis, statement of the rationale behind the choice of comparators, and report of sensitivity analyses being performed (data not shown). 

The advent of formulary submission guidelines, which call on pharmaceutical companies to generate and disseminate evidence of product value to health plans, represents a potentially powerful shift for consumers and producers of evidence. For formulary committees, guidelines mean more formal internal processes for judging whether a new drug is worthwhile. For pharmaceutical companies, the guidelines mean assembling and communicating standardized packages of evidence, including data supporting off-label claims and evidence of their product's value relative to alternative therapies.

One might expect that formulary guidelines would lead to increased efficiencies for the system and better health for patients. However, these expectations are based on an assumption that dossiers contain useful information. The findings here shed some new light on the issue and raise some concerns.

The clinical data reveal the type and breadth of the many clinical studies that are being submitted to support products. The data demonstrate that pharmaceutical and biotechnolo-gy companies use dossiers as a channel to support off-label claims with pharmacoeconomic evidence. Because such claims are unapproved, they cannot be supported by traditional marketing funds. The studies included to support these indications are generally less extensive and of lower quality than studies included to support FDA-approved indications. Although this finding is not surprising (by definition, on-label claims are required by the FDA to have substantiating evidence in the form of well-controlled trials), the data do raise questions about the extent to which plans may be basing decisions for off-label drugs on a relatively weak evidentiary base.

Only half of the dossiers we reviewed contained economic analyses. The methods used in the economic analyses that were included varied substantially, and frequently did not adhere to recommend protocols for the field. Many analyses lacked basic elements, such as stating the form of economic evaluation, stating the perspective of the study, or including sensitivity analyses, despite procedurally explicit guidance from the Format.

  Why the low level of compliance? In part it may simply reflect the nature of the field: researchers continue to detect variations in methods in published economic evaluations as well as failure to meet standards.

The poor quality of economic analyses in dossiers may also reflect the fact that drug companies are still climbing the learning curve in conducting analyses and communicating such information. That would be somewhat surprising, however, because pharmacoeconomic guidelines are not new–Australia, for example, has had them in place since 1992. Moreover, almost all large and many small pharmaceutical companies maintain internal health outcomes or economic departments, and all can contract out for the work to many research organizations with expertise.

Still, it is likely that some drug companies, particularly smaller ones, continue to develop expertise in assembling dossiers for formulary submission. Hill et al,

who analyzed economic analyses in submissions to the Australian reimbursement authorities, suggested that there is a long learning curve even where such submissions are mandated. It is also possible that dossiers are developed by marketing departments (rather than health economics and outcomes research departments), which lack expertise in conducting economic analysis or have different aims for its use.

  The data also indicate that economic analyses frequently contain inappropriate comparators and overly optimistic assumptions, and rarely mention caveats to their conclusions. Perhaps that, too, is not unexpected: pharmaceutical companies clearly endeavor to put products in the best light possible. However, it suggests that drug companies may have low regard for health plans' ability to distinguish good studies from poor ones. Moreover, poor-quality data likely contribute to ongoing suspicion about the usefulness of economic models. 

We also did not detect improvement in dossier quality over time, which could reflect a lack of learning among companies or possibly an increase in the number of smaller, inexperienced companies submitting dossiers. It also suggests the need for health plans to provide more feedback to drug companies about the quality of dossiers.

Our study did contain some positive findings: namely, that dossiers devoted to higher-cost and innovative products contained higher-quality economic analyses, suggesting that pharmaceutical companies put more resources into dossiers when the stakes are higher.

Because dossiers are proactively requested by health plans, they are considered unsolicited requests by the FDA and thus occupy a safe harbor against usual regulatory oversight by the agency against potentially false and misleading claims.

That is, for unsolicited requests, the FDA permits drug companies to communicate information such as economic analyses and data supporting off-label claims, even though the information extends beyond the FDA-approved indications for drugs in question. The AMCP Format has always been premised on a belief that formulary committees are capable of judging for themselves the totality of evidence supporting drugs, whether or not the FDA has sanctioned the data. Our results suggest that formulary committees must be vigilant and cautious about the information they are receiving in dossiers, given this lack of FDA oversight.

It is important to note a few limitations in our analysis. First, our study focused on a single health plan, and it is not clear whether our findings are generalizable to other plans, although in all likelihood drug companies use similar dossiers across plans. Indeed, the Premera Health Plan has been a leader in dossier review, and, if anything, one might expect dossier quality to be lower in other plans. Still, it will be important for researchers to audit dossiers submitted to other plans to draw more definitive conclusions. Second, some of the items we analyzed involve subjective judgments, and other reviewers may have come to different conclusions. Third, despite AMCP Format guidance about disclosures, it was not clear from many of the dossiers whether economic analyses were conducted by drug company employees or were contracted out to private research organizations.

In the future, analysts should continue to monitor the data in dossiers. It would be useful to examine the effectiveness and efficiency of the AMCP Format process. For example, of the dossiers requested from manufacturers, how many are actually submitted? How much time elapses between the request for a dossier and the P&T committee meeting? It also will be useful to explore the relationship between dossier submissions and actual formulary placement: a recent study found that receipt of a dossier itself did not appear to influence the likelihood of a product attaining favorable formulary status.

The impetus for future improvement in dossiers will ideally come from health plans themselves. Plan managers' insistence on high-quality submissions would create pressure on drug companies to provide better analyses. Further training and education of formulary committee personnel also would help. One problem nationwide may be that health plans do not possess the expertise necessary to judge the information in dossiers, particularly evidence contained in the economic models featured prominently in the guidelines. In the future, managed care plans themselves could educate formulary committee members about the potential value and how to make independent assessments of the contents of the dossiers that conform to the Format. The Foundation of Managed Care Pharmacy, which oversees research, educational, and other activities for the AMCP, has undertaken a series of initiatives to educate pharmacists, pharmaceutical company executives, and other interested professionals about the guidelines. It will be important to continue to monitor the progress of the field in the years ahead.

 We acknowledge the Academy of Managed Care Pharmacy for support during early phases of this project. We are grateful to Alison Timm, BS, for research assistance.

Harvard School of Public Health, Boston, Mass (FC); Pharmaceutical Outcomes Research and Policy Program, University of Washington, Seattle, Wash (SDS); Center for the Evaluation of Value and Risk in Health, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Boston, Mass (JAP, KB, PJN); Division of Paediatric Orthopaedic Surgery, BC Childrenâ€™s Hospital, The University of British Columbia, Vancouver, Canada (CAB); and Premera Blue Cross, Mountlake Terrace, Wash (JW). 

Funding was provided by the Agency for Healthcare Research and Quality under grant HHSP233200400483A. Carmen A. Brauer also was supported by a grant from the Alberta Heritage Foundation for Medical Research. Peter J. Neumann, ScD, and Sean D. Sullivan, PhD, had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the analysis.

Peter J. Neumann, ScD, Tufts-New England Medical Center, 750 Washington St, #063, Boston, MA 02111. E-mail: 

The authors (FC, SDS, JAP, CAB, KB, JW, PJN) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter discussed in this manuscript.

Concept and design (FC, SDS, JW, PJN); acquisition of data (FC, SDS, CAB, JW); analysis and interpretation of data (FC, SDS, JAP, CAB, KB, PJN); drafting of the manuscript (SDS, JAP, KB, PJN); critical revision of the manuscript for important intellectual content (SDS, JAP, CAB, KB, JW, PJN); statistical analysis (FC, SDS, KB); provision of study materials or patients (JW); obtaining funding (FC, SDS, PJN); administrative, technical, or logistic support (JAP, JW); supervision (PJN).

 AMCP guidance for submission of clinical and economic evaluation data to support formulary listing in U.S. health plans and pharmacy benefits management organizations. 

. Evidence-based and value-based formulary guidelines. 

. Principles for evidence-based drug formulary policy. 

. Pharmacy benefit management companies. In: Navarro RP, ed. 

Gaithers-burg, Md: Aspen Publications; 1999:221-242.

Mather DB, Sullivan SD, Augenstein D, et al.

 Incorporating clinical outcomes into drug formulary decisions: a practical approach. 

Jennifer A. Palmer, MS

Articles

Patient Adherence: A Blind Spot in Cost-Effectiveness Analyses?

A Strategic Plan for Integrating Cost-effectiveness Analysis Into the US Healthcare System

Quality of Clinical and Economic Evidence in Dossier Formulary Submissions