Regardless of the cause of bias, more awareness and scrutiny are needed when utilizing cost-effectiveness studies for healthcare decision making.
Published Online: December 18, 2012
Sammy Saab, MD; Young Choi, BS; Harman Rahal; Kun Li, MS; and Myron Tong, MD
Background: Cost-effectiveness analyses compare the value of different treatment strategies, given that one strategy is more effective but also more costly than the other. The results are often used as a pharmacoeconomic basis to support a particular treatment strategy.
Objectives: To determine whether studies that reported favorable cost-effectiveness results in the antiviral treatment of hepatitis B and hepatitis
C were more likely to be published.
Methods: A systematic literature search was performed to identify all original cost-effectiveness studies published from 1987 through 2011 regarding antiviral treatment of hepatitis B or hepatitis C. Data were abstracted independently by 2 reviewers. Corresponding authors were also contacted.
Results: Of the 77 original cost-effectiveness analyses, 55 (71.4%) utilized a cost-effectiveness ratio below $50,000 per quality-adjusted life-year (QALY) in reaching a main conclusion for the study. A total of 46 analyses (59.74%) utilized ratios below $20,000/QALY. Most analyses (71 [92.2%]) reported favorable cost-effectiveness results. Countries outside of the United States and Europe tended to report lower cost-effectiveness ratios. Academic-funded studies were less likely to report cost-effectiveness ratios below $20,000/QALY. Most of the corresponding authors stated that their cost-effectiveness studies were published.
Conclusions: Most published cost-effectiveness analyses regarding antiviral treatment of hepatitis B or hepatitis C reported favorable results. These results may suggest that researchers only attempt cost-effectiveness studies when there is a high likelihood of positive results. Regardless of the explicit cause of the bias, more awareness and scrutiny are needed when utilizing cost-effectiveness studies in decision making.
(Am J Manag Care. 2012;18(12):790-798)
Cost-effective studies are often used to support a particular change in treatment strategy, however:
Most cost-effective results are favorable in the literature.
Most cost-effective studies are supported by pharmaceutical industry and there is lack of financial disclosure.
Potential bias may exist at the author, funding source, reviewer, and/or editor level.
Over the past 10 to 15 years, there have been an increasing number of treatment strategies for hepatitis B and C. Overall, every generational succession in viral hepatitis treatment has been associated with incremental increase in efficacy and occasionally costs.1-3 Along with the introduction of new treatment strategies, cost-effectiveness studies have been used to explore whether these strategies compare favorably with other therapies on an economic basis. Cost-effectiveness studies are often used to justify a more expensive treatment or diagnostic strategy on the basis that its incremental increase in costs is within an arbitrary threshold ratio. In the United States, a benchmark of $50,000 per quality- adjusted life-year (QALY) is widely used as the upper limit of cost-effectiveness.4
A concern with cost-effectiveness studies is that there is a tendency to publish favorable results.5-7 This concern has been discussed with respect to a variety of disease categories, including cardiovascular, endocrine, infectious, musculoskeletal, orthopedic, neoplastic, and neurologic/psychiatric diseases.8-11 Factors shown to potentially bias publication include the country where the study was performed12,13 and whether the study was sponsored by a pharmaceutical company.8,9,11,14,15
Viral hepatitis is a medical and public health concern, with continually evolving treatment and diagnostic strategies. Given the presence of bias toward publishing studies with favorable results, we sought to determine whether there was a trend toward publishing studies that show the cost-effectiveness of new treatments and/or diagnostic strategies. Our hypothesis was that there would be a trend to publish studies that demonstrated a favorable cost-effectiveness result. In addition, we examined the predictors of publishing favorable results.
Two researchers independently conducted a systematic literature search of PubMed and the Tufts Medical Center Cost-Effectiveness Analysis Registry16 to find all original cost-effectiveness studies regarding the antiviral treatment of hepatitis B or hepatitis C (YC and HR). The search terms were cost-effectiveness, antiviral drug, hepatitis B treatment, and hepatitis C treatment. All articles were published in English between 1987 and June 2011 (Figure 1).
Cost-effectiveness ratios were recorded from the main conclusion of each study. If there was more than 1 result in the study’s conclusion, the mean value was calculated. Studies that did not report their cost-effectiveness analyses in dollars per QALY were identified as “not reported in dollars per QALY.” The conclusion of the author(s) about whether or not the study showed cost-effectiveness was documented as yes or no. Studies with conclusions stating both cost-effectiveness and non–cost-effectiveness were identified as yes/no but were included in the “no” category to bias against our hypothesis. All non-US currencies were converted to US dollars at the exchange rate of the year of publication.17
The source of funding for each study was documented as not reported (no mention of funding), private industry (partial or complete funding by a private industry), academic (government or public agency funding to an academic institution), or other (article explicitly mentioned that funding was neither nonindustry nor nonacademic).
Each article was judged on its quality using criteria for grading cost-effectiveness studies.18 Whether or not the article disclosed any conflict of interest was also documented as yes or not reported. The number of sensitivity analyses presented in each study was recorded, as well as the country of origin, the year of publication, and the journal impact factor at the time of publication.19
The corresponding authors of all the studies were contacted by e-mail and were e-mailed up to 3 times if no response was received. Alternative e-mails for the corresponding authors were also used in the event of no response. The authors were asked whether they (1) had conducted cost-effectiveness studies that they submitted but were never published (whether favorable or unfavorable) and (2) had conducted cost-effectiveness studies that they never submitted (whether favorable or unfavorable).
We systematically analyzed reported cost-effectiveness ratios and the conclusions of the author(s) from each study to examine whether there was any bias between positive predictors and negative predictors. We distinguished positive predictors from negative predictors using 2 methods. With the first method we directly cited the conclusion of the author(s) about whether the study showed favorable costeffectiveness. An answer of yes indicated positive predictors, while an answer of no implied negative predictors. Of the 77 studies, 71 implied favorable cost-effectiveness ratios. With the second method, we defined $50,000/QALY as a threshold by using reported cost-effectiveness ratios from each study. The reported cost-effectiveness ratios below the threshold were treated as positive predictors, while the rest were considered negative predictors. We also explored whether any study characteristic predicted the reported cost-effectiveness ratio. The reported cost-effectiveness ratios were directly used as our outcome instead of authors’ conclusions. For those studies with more than 1 number reported, the mean of multiple numbers was taken and used for our data analysis. The reports without any number were excluded.
We used a binomial test to examine the existence of bias among all the published papers, with the null hypothesis being that there is no bias between positive and negative predictors. The nonparametric Kruskal-Wallis test was used to determine whether each study characteristic had an effect on the reported cost-effectiveness ratio. We used SAS statistical software version 9.2 (SAS Institute Inc, Cary, North Carolina) for all analyses. P values were all 2-sided with significance set at .05.
From the more than 850 published papers related to hepatitis B and C infection, 77 met our entry criteria (Figure 1). Out of the 77 original cost-effectiveness analyses, 31 were hepatitis B related and 46 were hepatitis C related. Of the 46 hepatitis C treatment cost-effectiveness studies, 5 studies (10.9%) assessed different durations of interferon therapy,20-24 8 studies (17.4%) compared interferon and ribavirin with interferon, 25-32 5 studies (10.9%) compared pegylated interferon and ribavirin with no treatment,33-37 and 15 studies (32.6%) compared pegylated interferon and ribavirin with interferon and ribavirin.38-52 Of 46 hepatitis C–related studies, 5 (10.9%) compared the use of pegylated interferon alfa 2a plus ribavirin with pegylated interferon alfa 2b plus ribavirin.53-57 In 5 of 46 studies (10.9%), pegylated interferon and ribavirin were compared with pegylated interferon or interferon monotherapy. 51,52,58-60 In 18 of 46 studies (39.1%), there was an additional no-treatment arm.23,24,27,29,30,33-37,50,51,53,57-63 One study (2.2%) compared 2 different durations of treatment with pegylated interferon alfa 2a plus ribavirin,64 while one other study (2.2%) compared peginterferon with interferon and interferon plus ribavirin treatments.65
Of the 31 hepatitis B cost-effectiveness studies, 4 studies (12.9%) compared interferon with no treatment,66-69 5 studies (16.1%) compared interferon with nucleos(t)ide analogue,70-74 and 7 studies (15.2%) compared pegylated interferon with nucleos(t)ide analogue.75-81 In 3 of 31 studies (9.7%) there was an additional comparison with interferon.76,79,80 Of the 31 studies, 13 (41.9%) compared nucleotide with nucleos(t) ide analogues.82-95 One study (3.2%) specifically compared the use of a nucleoside analogue with no treatment.96
The characteristics of the studies are shown in Table 1. Most studies (46 of 77 [59.74%]) utilized a cost-effectiveness ratio below $20,000/QALY in reaching a main conclusion for the study (P = .0002) (Figure 2). A total of 55 of 77 (71.4%) studies utilized ratios below the $50,000/QALY benchmark (P <.0001). Only 5 of 77 (6.49%) studies utilized ratios above $50,000/QALY. Of the 77 studies, 71 (92.2%) reported a favorable cost-effectiveness ratio in their conclusions (P <.0001).
Most of the studies (53 of 77 [68.8%]) fell within the intermediate quality score range of 12 to 14 (Figure 3). With regard to funding source, 39 of 77 (50.6%) papers reported funding by private industry and 11 of 77 (14.3%) papers reported academic funding. Of the 77 studies, 51 (66.2%) did not have a statement of disclosure. The locations in which most of the studies were carried out were the United States and Europe: 29 of 77 (37.7%) and 28 of 77 (36.3%), respectively. Most of the studies (60 of 77 [77.9%]) were published in journals with an impact factor of less than 4, and 46 of 77 (59.7%) papers were published between the years 2005 and 2011 (Figure 4).
PDF is available on the last page.