Identifying Favorable-Value Cardiovascular Health Services
Published Online: June 17, 2011
R. Scott Braithwaite, MD, MS; and Sherry M. Mentor, MPH
A consensus is building to increase “value” as a guiding principle for US health reform1; indeed, value is used repeatedly throughout the health reform law as a unifying principle and as a descriptor for various new incentives that will be applied to providers and clinicians. At the same time, there is a complementary emphasis on the emerging role of consumers as active participants in their care, who engage in shared decision making with their clinicians and health organizations.1 As a result, it may be argued that health reform can be advanced by incentivizing and increasing consumer knowledge about high-value health services or health systems that deliver favorable value. In addition, emphasizing value rather than cost control may reduce the likelihood of rationing decisions that harm health by restricting high-value services.2
Although there is no consensus on how to define and measure value, the health reform law consistently juxtaposes the use of the word value with statements about the importance of improving quality or lowering cost.1 One published definition of value that is notably close to that embedded in the health reform law is the ratio of incremental benefits to incremental costs.3 In lay terms, this definition corresponds to the notion of “bang for the buck,” and in technical terms, this definition corresponds to the inverse of the incremental cost-effectiveness ratio.
The Cost-Effectiveness Analysis Registry4 summarizes and reviews published original English-language analyses that estimate incremental cost-effectiveness ratios using various methods (eg, mathematical modeling and primary data analysis). In principle, this registry should be an essential tool for informing the measurement of value and facilitating its use in US health reform. However, there are several important barriers to the use of this registry for policy decisions. First, the quality of analyses in the registry is not measured using a reproducible and validated approach, and the strength of evidence underlying particular analyses is sometimes questionable. This is a particularly important consideration because of the lack of transparency underlying assumptions in mathematical models of cost-effectiveness and because there sometimes is little high-quality evidence to inform model results.5 Second, analyses do not have expiration dates; therefore, an included analysis might concern a treatment that is obsolete or might involve a comparison that is no longer relevant. Third, analyses may often reach differing conclusions, rendering it difficult to know how to use conflicting analyses to inform policy. Fourth, some payers might argue that industry-funded analyses may present important conflicts of interest, which make their results hard to interpret because of the importance of subjective judgments in constructing the models that underlie their results. Fifth, analyses in the registry often include a wide range of healthcare settings and patient characteristics, and decision makers may want to base their decisions only on those analyses with similar settings and patients.
Herein, we describe an approach using the Cost-Effectiveness Analysis Registry that helps address these challenges. We used this registry to identify a subset of cardiovascular health services with a high level of evidence to suggest that they deliver favorable value. Identifying high-value services has many benefits for consumers in that they can be encouraged to use them when clinically appropriate, can engage in more informed health discussions with their clinicians, and can seek health plans that offer these services without barriers, such as copayments, deductibles, or burdensome administrative procedures.
This work was performed at the request of Consumers Union, publisher of Consumer Reports. The study objective was to provide comparisons and ratings of heart and vascular disease services that Consumers Union is pursuing.
We first describe how we identified cardiovascular health services with known value; second, how we applied quality-ofevidence standards together with nonobsolescence standards; third, how we applied consistency of evidence standards for high value; and fourth, how we applied additional inclusion criteria to ensure relevance to consumers. Through these stepwise filters, we identified a list of cardiovascular health services with particularly robust evidence to suggest high value and high relevance to consumers.
We adopted stringent standards for evidence. In other words, we sought to identify a limited number of health services that we are confident represent favorable value rather than seeking to identify a broader number of health services with less certain value estimations. We defined services broadly, including prevention, diagnosis, treatment, and management.
Identifying a Pool of Cardiovascular Health Services With Known Value
To identify a pool of cardiovascular health services potentially meeting high-value criteria, we queried the Cost- Effectiveness Analysis Registry4 to identify all published cost-effectiveness analyses of cardiovascular health services in the United States. The registry summarizes and reviews original English-language cost-utility analysis articles and can be searched by type of health services, such as cardiovascular, and by country of analysis, such as the United States. The articles undergo a screening and review process before being included in the registry. A MEDLINE search is performed using the keywords QALYs, quality, and cost-utility analysis, and then the Cost-Effectiveness Analysis Registry team screens the article abstracts to determine if the articles contain an original cost-utility estimate. Studies are excluded if they are reviews, editorials, or methodological articles, as well as cost-effectiveness analyses that do not measure health effects in quality-adjusted life-years (QALYs). These methods are described in more detail at the Cost-Effectiveness Analysis Registry Web site (http://www.cearegistry.org).
When synthesizing evidence, it is often necessary to supplement algorithm-based database searches with manual searches of journals that are likely to publish relevant articles and of bibliographies from select review articles. Accordingly, we supplemented our algorithm-based search of the Cost-Effectiveness Analysis Registry with manual searches of select national medical and scientific guidelines (eg, US Preventive Services Task Force, American Heart Association, and American College of Physicians), focusing our attention on those that were published in peer-reviewed scientific journals and that used explicit and standardized evidence syntheses. In addition, we searched select review articles for cost-effectiveness studies.3,6 We reviewed the titles or abstracts of all studies, and we obtained the source publications to evaluate quality-ofevidence standards and inclusion criteria. Finally, we searched other disease fields of the registry that might overlap with cardiovascular health services (endocrine for diabetes and lipids and smoking and tobacco for smoking).
Although value has many plausible alternative definitions and perspectives, we defined value for the purposes herein as the ratio of additional benefits to additional costs or, equivalently, as the inverse of the incremental cost-effectiveness ratio. We chose this definition because it is consistent with the scientific literature3 and because it corresponds to lay concepts (bang for the buck and best buy).
Published work suggests that the acceptable threshold for healthcare value in the United States is unlikely to be lower (eg, more restrictive) than the value of modern healthcare in aggregate (approximately $100,000 per quality-adjusted life-year or per life-year, in 2003 US dollars)7 and may be substantially higher (up to $265,000 per quality-adjusted life-year).8 Consequentially, we conservatively use $100,000 per quality-adjusted life-year or per life-year as our criterion threshold for high value in this proposal. However, because some prior published cost-effectiveness analyses have used an even more restrictive threshold of $50,000 per qualityadjusted life-year to demarcate high value, we performed sensitivity analyses incorporating this alternative threshold.9
A quality-adjusted life-year is a unit that simultaneously measures quality and quantity of life and reflects the idea that individuals often are willing to trade off some quantity of life if they can substantially improve their quality of life. Therefore, a year of life in high-quality health should “count for” more than a year of life in poor-quality health. Quality-adjusted life-years enable value to be compared across different healthcare interventions and represent an attempt to integrate all the benefits, harms, and burdens of interventions other than cost into a single number.
Quality of Evidence and Nonobsolescence Standards
We reviewed articles for quality of evidence by applying the Quality of Health Economic Studies, a validated instrument for measuring the quality of cost-effectiveness analyses.10 Scores vary from 0 to 100, and 75 is a commonly used cutoff for high quality.11 Each study was reviewed by at least 1 author, and studies were considered only if their Quality of Health Economic Studies score met this cutoff (in a comparison scoring of 20 randomly selected articles, our k value was .68).
We reviewed studies for robustness of clinical effectiveness and for nonobsolescence by verifying that services with favorable value were also favored or by using the most up-to-date clinical guidelines of a medical or scientific society. Because clinical guidelines are proliferating rapidly and are of varying quality, evidentiary basis, and health effect, we considered only those clinical guidelines that were published in peer-reviewed journals and used explicit evidence rating scales for level of endorsement and underlying evidence. (While it may be argued that peer review does not itself guarantee quality, we regarded peer review as a reasonable first step given the absence of standard quality metrics for clinical guidelines.) To meet criteria for robustness of clinical effectiveness, services had to receive the highest grade of supporting evidence (eg, level A in the case of American Heart Association guidelines) and the strongest recommendation in favor (eg, level 1 in the case of American Heart Association guidelines). To identify clinical guidelines meeting these criteria, we used the search tools of the National Guideline Clearinghouse,12 a repository of clinical guidelines from a wide variety of sources (eg, health plans and government, professional, and specialty organizations) that is supported by the Agency for Healthcare Research and Quality.
Consistency of Evidence Standards
We sought to include only services that were supported by consistent evidence. The following 2 criteria were used: (1) there should not be conflicting results if more than 1 high-quality published study addresses the same question and (2) the results of each study should be robust with regard to alternative but plausible assumptions.
Consistency was assessed by asking whether there was a different implication for decision making rather than asking whether a particular number was different outside the realm of chance. For example, 2 studies analyzing the same cardiovascular health service with results of $40,000 per life-year and $70,000 per life-year would be regarded as consistent because they were on the same side of the relevant decision threshold (<$100,000 per life-year denotes high value) and would yield the same inference for decision making (the service is high value and should be encouraged). In contrast, 2 studies with results of $40,000 per life-year and $140,000 per life-year would not be regarded as consistent because they were on opposite sides of the decision threshold and yielded inconsistent inferences for decision making (one suggesting high value and encouragement, with the other suggesting low value and an alternative decision).
PDF is available on the last page.