Wade M. Aubry, MD

 To provide recommendations that will improve approaches to measuring the value of new medical technologies to patients.

 Informed discussion by experts after literature review.

 A working group was formed, and participants discussed how value frameworks should incorporate key features important to patients in evaluating new medical technologies, particularly for chronic diseases.

 The working group suggests that new value frameworks should integrate real-world evidence to complement randomized controlled trials, incorporate the ways in which real-world behavior mediates outcomes, and explicitly discuss how therapies affect real-world equity and disparities in care.

 Collective stakeholders that include key decision makers within our healthcare system need to recognize the importance of implementing real-world evidence and devote resources to further research into the chronic disease areas in which the impact of human behavior is amplified by the duration of disease and treatment.

Steps have been taken to develop critical evaluations of drugs with the creation of value frameworks from organizations such as the Institute for Clinical and Economic Review, American Society of Clinical Oncology, and Memorial Sloan Kettering, but more work can and should be done to further refine these approaches. The Value FORWARD (Frameworks, Outcomes, and Real-World Adherence in Chronic Disease) Committee puts forth the following recommendations:

Value frameworks should incorporate real-world effectiveness to complement randomized controlled trial efficacy.

Value assessments must explicitly incorporate the ways in which real-world behavior mediates outcomes.

Value assessments should explicitly discuss how therapies affect real-world equity and disparities in care.

Policy makers, healthcare payers, patients, employers, and other stakeholders continue to fret about the rising cost of new medical technologies. Many scientific organizations have responded by developing new tools to assess the value of these technologies, commonly called “value frameworks.” For example, the American Society of Clinical Oncology (ASCO),

 and the Institute for Clinical and Economic Review (ICER) have offered alternative approaches to assessing the value of technologies. In principle, better information about value can help moderate growth in the cost of technologies that do not provide sufficient benefit to patients.

Yet there remains a gap between the goal of measuring value conferred to patients and the available evidence for new medical technologies. Randomized controlled trial (RCT) evidence generated for regulatory approval often incorrectly measures or fails to measure aspects of value that are key to patients. Some of these gaps and limitations are widely understood,

 but others have received too little attention, such as the role of patient behavior in mediating clinical outcomes.

 For example, evidence generated by RCTs often reflects outcomes achieved with motivated patients treated in environments in which individual patient behaviors that are critical to driving real-world outcomes, including adherence to medications, are controlled. As a result, interventions that improve adherence may not demonstrate large efficacy benefits in RCTs but may demonstrate larger benefits in real-world settings in which adherence is typically lower.

A working group was formed in partnership with Precision Health Economics and Intarcia Therapeutics in 2017 to make recommendations on how value frameworks should close the gap between evidence and patient behavior when evaluating new medical technologies, particularly for chronic diseases. Our working group recommended that these value frameworks enhance attention to (1) real-world evidence (RWE), (2) the role of adherence, and (3) nonclinical components of the patient experience.

Unlike healthcare systems in countries such as England, Germany, and Australia, where health technology assessment processes are well established,

 the US healthcare system does not have a formalized approach for defining or measuring value. Private organizations have stepped in to fill this gap but offer varying perspectives and approaches. For example, ASCO has released a value framework meant to aid physician communication of cancer treatment options with patients.

 ICER evaluates the cost-effectiveness of drug therapies from a payer and health system perspective.

 Memorial Sloan Kettering’s drug evaluation framework, the Drug Abacus, which evaluates cancer drugs, enables users to input their own preferences on different dimensions of value.

 These value frameworks differ in the perspectives that they use and in the attributes of value that they include in their evaluations. As a result, different value frameworks evaluating the same drug may have different conclusions about what may be considered reasonable value.

Recommendation 1: Value Frameworks Should Incorporate Real-World Effectiveness to Complement RCT Efficacy

Value frameworks have tended to emphasize evidence generated in RCTs, excluding or downplaying the use of RWE. Clinical trial evidence is emphasized as the gold standard for health benefit, although recently, there has been a growing awareness and movement to improve databases and infrastructure for RWE.

Clinical trial populations often fail to mirror real-world populations. This external validity problem is already well documented.

 Clinical trials are conducted in controlled environments where the structure of the environment lends itself to optimal rates of treatment adherence.

 More recently, some prominent researchers have argued that RCTs with smaller samples, often seen in breakthrough treatments with accelerated market approval, suffer from internal validity problems such as misalignment between the characteristics of subjects in the treatment and comparator groups.

RWE can help address these limitations. To be sure, it also suffers from limitations related to internal validity (eg, persons using novel therapies in the real world are often systematically different from their peers using older drugs at the same time). Methodological solutions exist to control for some of these confounders, and more generally, care must be taken to ensure the highest degree of internal validity in RWE. However, RWE is more than just a second-best alternative to RCT evidence; it provides inferences that are necessarily applicable to real-world populations. One major area of behavior around treatment that can be captured more realistically with RWE is adherence to treatment regimens.

 Our committee recommends explicit evaluation of RWE as a potential input into value assessments.

Recommendation 2: Value Assessments Must Explicitly Incorporate the Way Real-World Behavior Mediates Outcomes

As noted earlier, a critical moderator of health outcomes outside of clinical trials is adherence. At present, adherence is not explicitly included in any value framework or in the wider body of literature of cost-effectiveness models and budget impact models. Full adherence is typically “included” as an underlying assumption in these models and frameworks, as it is implicitly a part of clinical efficacy.

The need to do this is intuitive, particularly when comparing treatments in the same therapeutic area that possess different delivery mechanisms (eg, daily oral tablets vs weekly injections) or different adverse event profiles. An evaluation of a treatment that has a positive effect on behavior and encourages a higher rate of adherence than its comparator ought to consider the clinical consequences of better adherence (eg, an improvement in patient outcomes or achievement of treatment goals).

Adherence can also be considered an element for differentiating high-value and low-value care. Treatments that encourage improved adherence will result in less waste, because prescribed and dispensed treatments will actually be used. The importance of reducing this waste through improved adherence cannot be underestimated. Past estimates have found that more than $100 billion annually of avoidable costs are a result of medication nonadherence.

 Explicitly incorporating adherence would enable value frameworks to better identify high-value compared with low-value care.

The magnitude of this underuse has not been well understood, in part because our current adherence measures are not sufficiently nuanced to capture the full spectrum of behavior. Current measures of adherence do not infer medication consumption and are typically used as a binary measure of “good adherence” at or above 80% or “bad adherence” below 80%.

 This threshold implies that 80% is sufficient to achieve optimal outcomes, which has not been demonstrated across all diseases. Additionally, these measures primarily capture prescription behavior (ie, whether the prescription has been submitted/picked up), not whether the person has taken the treatment.

 Further, these measures do not capture primary nonadherence for persons who do not fill their initial prescription.

Starting with chronic diseases, we should seek to map adherence patterns and link those adherence patterns to outcomes. Work that has already started in this area—for example, in individuals with schizophrenia

—has demonstrated that adherence is not binary, with nuanced adherence patterns having real impacts on patient outcomes. We should seek to build on this work and develop studies that capture the value of adherence for health and quality-of-life outcomes so that adherence can be explicitly included as a separate value component in economic valuations of treatment.

Recommendation 3: Value Assessments Should Explicitly Discuss How Therapies Affect Real-World Equity and Disparities in Care

A critical aspect of treatment in the real world largely neglected by existing value frameworks is the impact on disparities in care. Previous research has demonstrated that nonclinical characteristics, such as education level, can affect adherence behaviors, which ultimately affect outcomes.

 Socially disadvantaged individuals are more likely to experience disease, at greater severity and with a higher likelihood of adverse effects.

 Combined with poorer adherence behaviors, disparities in healthcare can be compounded in chronic disease, where long-term consistent adherence to treatment is necessary for achieving positive outcomes. Explicitly evaluating how a new treatment can affect adherence and otherwise improve existing disparities in care would be an important element in determining the social impacts of therapeutic use.

A relatively strong body of literature already exists that articulates inequity in healthcare and some of the causes of those disparities. However, more research is required to develop methodologies to quantify and incorporate equity impacts into value assessments. It will be necessary to extend these equity concerns to additional nonclinical aspects of care, such as productivity and caregiver burden. The increased burden on productivity and caregivers also disproportionately affects disadvantaged individuals. Patients and providers have emphasized the importance of these aspects of care in making treatment decisions, but insufficient research exists to enable these aspects to play a large decision-making role. Further research and studies that take innovative approaches to, first, identifying the relevant nonclinical aspects of the patient experience and, second, measuring and quantifying these aspects are highly important to furthering our evidence base for treatment innovations and overall improving healthcare decision making.

A wide range of stakeholders are demanding better evidence of value. Value assessments and frameworks must adapt by making better use of real-world data and considering how individual behavior causes real-world outcomes to diverge from those seen in clinical trials. Our recommendations can help align value frameworks with value to individuals in the real world. Although we have already taken good steps forward to develop critical evaluations of drugs, more work can and should be done to further refine these approaches. Minimally, further efforts should be made to incorporate RWE to more accurately reflect the performance of new innovations by including adherence and measuring patient outcomes in the usual settings of patient care. To do so, the collective stakeholders that comprise key decision makers of our healthcare system need to devote resources to further research in chronic diseases where the impact of human behavior is amplified by the duration of disease and treatment.

The authors would like to thank Tiffany Shih for administrative and editorial support.

 Harvard Medical School (ABJ), Boston, MA; Precision Health Economics (JWC, LY), Los Angeles, CA; University of California, San Francisco (WMA), San Francisco, CA; Health Intelligence Partners (JB), Chicago, IL; University of Illinois, Chicago (WB), Chicago, IL; University of Michigan (AMF), Ann Arbor, MI; Pennsylvania State University (DMF), University Park, PA; Intarcia Therapeutics (DF, JY, BS, KY-I), Boston, MA; Dentons US LLP (RK), Washington, DC; University of Southern California (DNL), Los Angeles, CA; Tufts Medical Center (PJN), Boston, MA; The Brookings Institution (KP), Washington, DC.

 Funding for the Value FORWARD Committee was provided by Intarcia Therapeutics. Support was provided by the Office of the Director, National Institutes of Health (1DP5OD017897-01, Dr Jena). The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

 Dr Jena reports receiving consulting fees unrelated to this work from Amgen, Analysis Group, AstraZeneca, Biogen, Bristol Myers Squibb, Celgene, Eli Lilly, Hill Rom Services, Novartis, Pfizer, Precision Health Economics, Sanofi Aventis, Tesaro, and Vertex Pharmaceuticals. Ms Chou is an employee of Precision Health Economics, a healthcare consultancy with clients in the life sciences industry, and holds equity in Precision Medicine Group, parent company of Precision Health Economics. Ms Yoon is a previous employee of Precision Health Economics. Dr Aubry attended paid advisory boards and meetings for Intarcia on April 8, 2017, and May 16, 2017, and serves as a consultant to Precision Health Economics. Dr Burton has consulted for and received lecture fees from Amgen. Dr Fendrick has consulted for AbbVie, Amgen, Bayer, Centivo, Community Oncology Association, EmblemHealth, Exact Sciences, Freedman Health, Health at Scale Technologies, Lilly, Mallinckrodt, MedZed, Merck, Risalto, Sempre Health, State of Minnesota, Takeda, Wellth, and Zansors; has performed research for the Agency for Healthcare Research and Quality, Boehringer-Ingelheim, Gary and Mary West Health Policy Center, Laura & John Arnold Foundation, National Pharmaceutical Council, Patient-Centered Outcomes Research Institute, PhRMA, Robert Wood Johnson Foundation, and State of Michigan/CMS; and holds outside positions as co-editor-in-chief of 

, member of the Medicare Evidence Development & Coverage Advisory Committee, and partner in V-BID Health, LLC. Mr Franklin is employed by Intarcia Therapeutics, Inc. Dr Lakdawalla is a consultant to Precision Health Economics and holds equity in its parent company, Precision Medicine Group. Dr Neumann reports participating on advisory boards or consulting for AbbVie, Avexis, Bayer, Bluebird, Celgene, DePuy Synthes, GSK, Merck, Novartis, Novo Nordisk, Pacira, Paratek, Precision Health Economics, Vertex, Boston Health Economics, and Congressional Budget Office. Drs Yee, Sakurada, and Yu-Isenberg are former employees of Intarcia Therapeutics. Drs Sakurada and Yu-Isenberg have attended Academy of Managed Care Pharmacy, American Diabetes Association, Asembia, International Society for Pharmacoeconomics and Outcomes Research, and National Association of Managed Care Physicians conferences when under employment of Intarcia Therapeutics. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (ABJ, JWC, WMA, JB, WB, DMF, RK, DNL, KP, JY, BS, KY-I); acquisition of data (LY, DF, KY-I); analysis and interpretation of data (ABJ, WMA, JB, AMF, DMF, RK, DNL, PJN, KP, BS, KY-I); drafting of the manuscript (ABJ, JWC, LY, WB, AMF, PJN, KP, KY-I); critical revision of the manuscript for important intellectual content (JWC, WMA, JB, WB, AMF, DMF, DF, RK, DNL, PJN, KP, JY, BS, KY-I); provision of patients or study materials (KY-I); obtaining funding (DF, KY-I); administrative, technical, or logistic support (JWC, LY); and supervision (DNL, KY-I).

 Anupam B. Jena, MD, PhD, 180 Longwood Ave, Boston, MA 20115. Email: jena@hcp.med.harvard.edu.

1. Schnipper LE, Davidson NE, Wollins DS, et al. Updating the American Society of Clinical Oncology value framework: revisions and reflections in response to comments received. 

. 2016;34(24):2925-2934. doi: 10.1200/JCO.2016.68.2518.

2. Drug Abacus. Drug Pricing Lab website. drugpricinglab.org/tools/drug-abacus. Accessed October 1, 2017.

Best Care at Lower Cost: The Path to Continuously Learning Health Care in America

. Washington, DC: National Academies Press; 2013.

4. Nordon C, Karcher H, Groenwold RH, et al. The “efficacy-effectiveness gap”: historical background and current conceptualization. 

. 2016;19(1):75-81. doi: 10.1016/j.jval.2015.09.2938.

5. Maclean JR, Pfister M, Zhou Z, Roy A, Tuomari VA, Heifets M. Quantifying the impact of nonadherence patterns on exposure to oral immunosuppressants. 

. 2011;7:149-156. doi: 10.2147/TCRM.S16870.

6. Sun M, Larcher A, Karakiewicz PI. Optimal first-line and second-line treatments for metastatic renal cell carcinoma: current evidence. 

. 2014;7:401-407. doi: 10.2147/IJNRD.S48496.

7. Oortwijn W, Broos P, Vondeling H, Banta D, Todorova L. Mapping of health technology assessment in selected countries. 

. 2013;29(4):424-434. doi: 10.1017/S0266462313000469.

8. ICER Value Assessment Framework. Institute for Clinical and Economic Review website. icer-review.org/methodology/icers-methods/icer-value-assessment-framework. Accessed October 1, 2017.

9. Bothwell LE, Greene JA, Podolsky SH, Jones DS. Assessing the gold standard—lessons from the history of RCTs. 

. 2016;374(22):2175-2181. doi: 10.1056/NEJMms1604593.

10. Frieden TR. Evidence for health decision making—beyond randomized, controlled trials. 

. 2017;377(5):465-475. doi: 10.1056/NEJMra1614394.

11. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence—what is it and what can it tell us? 

. 2016;375(23):2293-2297. doi: 10.1056/NEJMsb1609216.

12. van Onzenoort HA, Menger FE, Neef C, et al. Participation in a clinical trial enhances adherence and persistence to treatment: a retrospective cohort study. 

. 2011;58(4):573-578. doi: 10.1161/HYPERTENSIONAHA.111.171074.

13. Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials [NBER working paper no. 22595]

 National Bureau of Economic Research website. nber.org/papers/w22595. Published September 2016. Updated October 2017. Accessed October 1, 2017.

14. Cohen AT, Goto S, Schreiber K, Torp-Pedersen C. Why do we need observational studies of everyday patients in the real-life setting? 

. 2015;17(suppl D):D2-D8. doi: 10.1093/eurheartj/suv035.

15. Burton WN, Chen CY, Schultz AB, Edington DW. The association between achieving low-density lipoprotein cholesterol (LDL-C) goal and statin treatment in an employee population. 

. 2010;13(1):1-8. doi: 10.1089/pop.2009.0020.

Articles

New value frameworks should incorporate real-world evidence that reflects patient treatment behavior, adherence to medication, and equity concerns arising from disparities in care.

Understanding and Improving Value Frameworks With Real-World Patient Outcomes

 To determine the lifetime social value of using the guideline-recommended vaccines for children born in the United States in 2009. 

This study utilized an economic model with parameter values sourced from clinical and observational data, as well as the literature. 

The model quantified the health effects of routine vaccination for 14 diseases in terms of quality-adjusted life-years (QALYs) saved. The health effects were then valued by applying an economic value of a QALY. Producers’ profits were estimated using data on vaccine prices, profit margins, and the number of vaccines administrated in the 2009 US birth cohort. The costs of producing the vaccines were subtracted from the value of the health effects to yield the total social value of vaccination. The producers’ and consumers’ shares of this social value were calculated. Sensitivity analyses were conducted to determine how results depend on underlying parameter assumptions. 

Estimates indicated that vaccination of this cohort will save 1.2 million QALYs, relative to no vaccination. Of those health gains, 88% stemmed from reduced mortality and 12% from reduced morbidity. We estimated a social value of $184.1 billion from these gains, of which $3.4 billion accrues to manufacturers as profits, while $180.7 billion accrues to the rest of society. In sensitivity analysis, the total social value ranged from $40 billion to $675 billion, and the manufacturers’ share ranged from 0.3% to 11.5%. 

 Policy makers should account for this social value when considering policies affecting incentives to vaccinate and develop new vaccines.

		By preventing illness and premature deaths, vaccination of children born in the United States in 2009 will generate $184 billion in lifetime social value above the costs of the vaccines. 

		Because saving a child’s life yields many healthy life-years, the large majority (88%) of the health benefits of vaccines is due to avoided premature deaths rather than reduced morbidity (12%). 

		The high social value of vaccines has improved population health and provided economic benefit to multiple stakeholders, including patients, health plans, and vaccine manufacturers, whose profits in this cohort amount to approximately 2% ($3.2 billion) of the total social value.

The innovation of childhood vaccines has resulted in a decline in infectious disease, as well as gains in length and quality of life. Smallpox has been eradicated, poliomyelitis is nearly eliminated, and many other vaccine-preventable diseases have seen declines in incidence.

 Although adverse events (AEs) can occur with vaccines,

 and recent research has focused on their rising costs,

 the postvaccine era has seen life expectancy increase 15 to 25 years compared with the pre-vaccine era, and further gains are expected.

 Evidence suggests a large share of these survival gains is due to the control of infectious disease through vaccination.

When encouraged by public health policies, vaccination also provides a benefit to government and private payers by reducing overall costs and increasing population health. The CDC has cited evidence that common childhood vaccinations save over $5 in direct medical costs and effects on productivity for every $1 spent.

 Maciosek and colleagues found that preventative childhood immunization produced annual net medical savings of $267 per person.

 Vaccination also generates community (herd) immunity by reducing disease incidence and transmission, thus resulting in a healthier population.

Because vaccines have been successful at preventing disease, the public is no longer regularly confronted with many vaccine-preventable diseases, and the health and economic benefits of vaccination may be underappreciated.

 As childhood vaccines have reduced disease prevalence, real and perceived AEs of vaccination have become more salient to parents than the vaccine-targeted diseases.

 Consequently, vaccination rates in many US states have declined in recent years.

 As vaccination rates slip, the risk of new outbreaks increases.

Moreover, although consumers are insulated from the cost of many vaccines, vaccine cost is an important consideration for payers and providers and has been criticized.

 This focus on AEs and costs has obscured vaccines’ overall value to individuals and society. Previous research has yet to show how the total social value of vaccines is divided between innovators who develop these technologies and patients and the broader society who benefit from them. Therefore, in this study, we sought to measure the social value of childhood vaccines in the United States and the distribution of that value to manufacturers versus the rest of society.

The concept of social value of therapies and its distribution between manufacturers and patients has been described in other disease areas. For instance, Grabrowski et al found that statin usage resulted in a social value of $1.25 trillion, of which patients received 76%.

 Yin et al performed a similar analysis on tyrosine kinase inhibitors for the treatment of chronic myeloid leukemia and found a social value of $143 billion—90% of which was retained by patients.

 Recent gains in cancer survival have provided $1.9 trillion of additional social value, with 81% to 95% of that being retained by patients.

 Lastly, HIV/AIDS therapies have generated $1.38 trillion in social value, with 95% accruing to patients.

 Such analyses are net monetary benefit analyses, which is a common economic way of thinking about value which is distinct from cost-effectiveness analysis. The aim is to measure the total value a given health intervention generates for society, and how that value is distributed across patients and manufacturers.

This study applied similar methods to determine the social value of childhood vaccines for a birth cohort in the United States. Consistent with previous research, social value was defined from an economic perspective as the quantity of resources, in monetary terms, that society would be willing to give up in order to retain the health gains attributable to vaccines. Put another way, the overall social value of vaccines equals the aggregate value retained by consumers (above the actual payments for vaccines) plus the value retained by manufacturers (in the form of vaccine profits). We decomposed the social value into the shares accruing to manufacturers versus the rest of society. For infectious diseases, the social value includes not only those vaccinated, but also those not vaccinated who benefit from the reduction in disease incidence.

The study entailed constructing an economic model based on observational and clinical data. The model calculates the social value of the routine pediatric vaccination schedule used in the United States in 2009. We do so by quantifying the health effects of routine vaccination of children born in the United States in 2009. In particular, vaccines to prevent the following 14 diseases were considered: congenital rubella syndrome, diphtheria, haemophilus influenzae type b (Hib), hepatitis A, hepatitis B, measles, mumps, pertussis, pneumococcus-related diseases (including pneumococcal disease, otitis media, pneumonia, and meningitis), polio, rotavirus, rubella, tetanus, and varicella. The influenza vaccine was not included because its changing seasonal nature would have required different methods.

The social value was estimated by applying an economic value to the health effects of vaccines, measured in terms of quality-adjusted life-years (QALYs) saved through vaccination. QALYs take into account both duration and quality of life. A year in perfect health would be measured as 1 QALY, whereas death counts as 0. From the value of the QALYs gained, the costs to produce vaccines were subtracted. This yielded the social value—or in economic terms, the total surplus&mdash;of vaccines, and represents the economic value of the health gains from vaccines minus the resources society spent to produce them. The shares of the total surplus accruing to manufacturers (producer surplus) versus the rest of society (consumer surplus) were also calculated.

It should be noted that vaccine-preventable illnesses impose additional costs on society beyond the utility loss infected individuals experience, including caregiver utility loss and the use of special services for persistent disability. Therefore our estimate of the health value of vaccination should be considered a lower bound.

In addition, one should exercise caution in interpreting the results of this framework for rotavirus, since in industrialized countries like the United States, the costs of rotavirus are mainly hospitalization and caregiver utility loss, as rotavirus mortality and morbidity are lower in the industrialized setting. In contrast, many of the other studied vaccines target illnesses that imposed a high mortality and morbidity burden in the pre-vaccine era.

According to the CDC, 4,130,665 children were born in the United States in 2009.

 The health effects of vaccination in this cohort were estimated by combining data from the literature with life tables from the Human Mortality Database.

 From the literature, we obtained for each disease data on cases of illness prevented, premature deaths avoided, average age of onset, average age at death from the disease, average duration of disease, and utility loss. Specific parameter values and sources are available in the eAppendix (

). The survival benefits of vaccination were net of adverse reactions to vaccination. To obtain the economic value of a QALY, we considered values generated by revealed and stated preference studies.

 A mid-range value of $150,000 was used and varied in sensitivity analysis.

Estimating manufacturers’ profits required 3 types of data: 1) data on vaccine prices, 2) data on manufacturers’ profit margins, and 3) data on the number of vaccines administered in the 2009 US birth cohort. We obtained data on prices (available in the eAppendix) from the CDC Vaccines for Children Program website, which contains archived data on public and private vaccine prices from 2008 to 2015.

We obtained data on vaccine manufacturers’ profit margins from annual reports and financial statements. When measuring manufacturers’ profits, we used the gross profit margin, which represents the sales volume minus production costs. Obtaining a companywide average across the top 5 vaccine manufacturers produced an average gross profit of 75%.

 This is a conservative approach, as gross profits do not subtract out manufacturer research and development (R&D) and marketing expenses. By using the gross profit margin, we can view vaccines’ social value as society’s benefit from vaccination, and society’s investment in R&D as the cost of inventing and developing the vaccines. (Subtracting R&D from profits would negate this framing.) This framing is useful because investments should be undertaken when the benefits (ie, the return on investment) exceeds the cost; social value is an important part of this equation. Moreover, R&D costs include the costs of many failures that the innovator encountered on the way to the given successful product; there is not an established method for measuring R&D costs for vaccines.

To estimate the number of vaccines administered, we required data on vaccine coverage rates, dosage schedules, wastage, and the cohort size. Following previous work, we assumed that 53% of vaccine doses were publicly (vs privately) administered and the wastage rate—the rate at which additional vaccines must be purchased beyond those needed for each vaccinated child because some vaccines will be unused&mdash;was 5%.

 from the CDC. Doses administered between ages 0 and 18 were included, but the costs of any adult booster doses were excluded. Given that any adult booster doses occur many years into the child’s life, whereas the lives saved and illnesses avoided from vaccination are realized mainly in early childhood, the effect of the focus on childhood doses should be minimal.

The 3 analytic steps are described broadly below. Additional detail is provided in the eAppendix. Throughout the analysis, monetary values were inflation-adjusted to 2014 US dollars using the Consumer Price Index,

 and an annual discount rate of 3% was applied.

Step 1: Value Health Effects of Vaccination

The health effects of vaccines were calculated by summing the changes in morbidity and mortality among the 2009 US birth cohort due to vaccination. The mortality effects were calculated as the number of deaths averted from vaccination multiplied by the QALYs the typical child would lose from dying of the given disease (calculated as average life expectancy minus average age at death from the given disease times aged-adjusted utility). The morbidity effects were calculated as the number of cases of illness prevented through vaccination times the typical duration of illness times the disutility from the given illness. The health effects of vaccines in QALYs were then converted to economic terms by valuing each QALY at $150,000.

 This yielded the economic value of the health effects of vaccination.

Step 2: Estimate Manufacturers’ Profits

Vaccine manufacturers’ profits from a given vaccine were estimated by multiplying the vaccine’s price by the number of vaccines sold by the profit margin. Although vaccines typically consist of multiple doses, and vaccination rates vary by dose, in the profit calculations we assumed that all children who received the first dose of a vaccine would also receive all subsequent doses. This assumption overestimates profits because some children will not receive all doses, and manufacturers’ profits will be lower than they would be had these children received all of their doses.

Using the results of the previous 2 steps, we calculated consumer, producer, and total surplus (ie, social value). Consumer surplus was calculated as the value of the health effects of vaccines (from Step 1) minus the cost of the vaccines (from Step 2). Producer surplus equaled the manufacturer profits (from Step 2). Consumer and producer surplus together yielded the total surplus.

We performed analyses to test the sensitivity of the model to the parameters. Specifically, we varied all parameters by 

10%, except for the disease-specific vaccination rates, which were varied by 

 5% to avoid specifying rates over 100%. In addition, we also varied the value of a QALY from $50,000 through $250,000 to reflect the wide range of values in the literature, and varied the discount rate from 0% to 6%. Lastly, we conducted an analysis, which included the parents’ time cost to take the children to receive the vaccines, taking into account that multiple vaccines may be given in the same visit.

The health effects of vaccination are reported in 

. Compared with no vaccination, an estimated 1.2 million QALYs will be saved due to vaccination among children born in the United States in 2009, for a value of $185.2 billion. Because vaccines typically prevent deaths that would have occurred in childhood, these avoided deaths save a large number of QALYs, whereas QALY gains from avoided illness are more modest. Consequently, 88% of the health value of vaccines is due to avoided death compared with 12% due to avoided illness. Among vaccines, diphtheria, tetanus, and pertussis (DTaP/Tdap) will have the largest health value by an order of magnitude, at nearly 800,000 QALYs saved for a value of $119 billion. Other vaccines with large health values are pneumococcus-related diseases (154,000 QALYs, $23 billion); measles, mumps, and rubella (MMR) (135,000 QALYs, $19 billion); and hepatitis B (79,000 QALYs, $12 billion). The smallest health value will be for rotavirus, at 5264 QALYs and $790 million.

 reports estimates of manufacturers’ revenues, costs, gross and net profits by vaccines given to the 2009 US birth cohort. Estimated profits are the lowest for the hepatitis A vaccine ($109 million gross, $36 million net), and the highest for pneumococcus-related diseases ($1.0 billion gross, $350 million net). Across all 14 diseases, vaccines generate an estimated $4.5 billion in revenues, $1.1 billion in costs, $3.4 billion in gross profits, and $1.1 billion in net profits.

The total social value by vaccine and its distribution across manufacturers and consumers are reported in 

. Consumers’ share of value, or surplus, ranges from $10 million for rotavirus to $119 billion for DTaP/Tdap. Vaccines producing the greatest consumer surplus do not necessarily provide the largest profit, or producer surplus, to manufacturers. Total social value ranges from $595 million from rotavirus to $119 billion from DTaP/Tdap, with the full vaccination schedule generating $184 billion in social value (

), or $45,000 per child. Of that total social value, 1.8% accrues to manufacturers, whereas 98.2% accrues to the rest of society. The manufacturers’ share ranges from 0.3% for DTaP/Tdap to 98.3% for rotavirus, and is less than 15% for 7 out of the 9 vaccines.

In sensitivity analyses, we found that the model is most sensitive to the economic value of a QALY, the premature deaths prevented from vaccination, and the discount rate, which together contributed 98.1% of the variance in results (

). In simulations, the total social value ranged from $153 billion to $227 billion, and the manufacturers’ share ranged from 1.3% to 2.6%. In other words, varying parameters by 

10% maintained a social value of at least $37,172 per child, with at least 97% of the value retained as consumer surplus. In addition, varying the value of a QALY from $50,000 to $250,000 led to social value ranging from $61 billion to $308 billion, and varying the discount rate from 0% to 6% led to social value ranging from $447 billion to $107 billion. Subtracting parents’ time cost led to a social value of $182 billion and manufacture share of 1.8%.

Our estimates suggest that routine childhood vaccination produces a large social value. In particular, we found that vaccination will save 1.2 million QALYs among children born in the United States in 2009, relative to a counterfactual of no vaccination. Vaccines treating common diseases, such as DTaP/Tdap, pneumococcus, and MMR, generate a particularly large share of the health benefits.

Childhood illness is often transitory, but many childhood diseases have nontrivial fatality rates, and a death in childhood costs many years of healthy life. Therefore, it is not surprising that the bulk of these health gains (88%) stems from reduced mortality, with the remainder (12%) from reduced morbidity. Valuing each life-year at $150,000 and subtracting the production costs of vaccines, we found that vaccination of the 2009 birth cohort generated $184 billion in social value, or $45,000 per child. Of this, $3.4 billion (2%) accrues to manufacturers in the form of profits, while $180.7 billion (98%) is retained by the rest of society. Sensitivity analysis showed that the large social value and large share retained by society are robust to a wide range of plausible parameters.

The 2% manufacturers’ share of social value we found is smaller than that found in similar analyses of other health interventions. The manufacturers’ share of social value was between 5% to 24% in HIV/AIDS, chronic myeloid leukemia, and heart disease.

 The distribution of gains between manufacturers and the rest of society is determined by the cost of the intervention and the health gains it produces. Health gains will generally be larger in the context of diseases that are prevalent or take life early. Therefore, it is not surprising that childhood vaccines come out favorably in these comparisons, as they are among the most cost-effective health interventions

 and they preserve the health of children, who, in the absence of a devastating childhood illness, typically go on to lead a long, healthy life. To put the health results in context, in other industries, the producers’ share of total social value has ranged from 4% for minivans

The fact that 98% of the social value from existing childhood vaccines goes to children and their families is good news for access to current vaccines; however, it has longer-term implications for innovation. A 2% share of value suggests relatively weak incentives for the development of new and improved vaccines, relative to the incentives to develop cancer drugs, cholesterol-lowering drugs, or broadband Internet. However, there is debate in the literature about the exact manufacturer share of surplus required to optimally incentivize innovation, and therefore, although this study informs the debate, it does not determine optimal vaccine pricing.

Because the model relies on inputs from the literature, it is limited by their availability and quality. When exact parameters were not available, conservative assumptions were made. For example, costs and profits were overestimated by assuming children who receive 1 dose of a vaccine receive all subsequent doses, thus underestimating social value and overestimating manufacturers’ share. In cases in which illness is usually short and mild but can be severe and long-lasting (eg, polio), we assumed a short and mild case in our utility calculations. In calculating profit margins, we assumed the companywide profit margin applied to a given vaccine, although vaccines may comprise a less profitable division of pharmaceutical firms.

 We also used gross margin, which does not take R&D and marketing costs into account, rather than net margin, so that we could consider the social value as the return on society’s investment in developing the vaccine.

Additionally, while the 2009 birth cohort was selected for this analysis due to the availability of data on the health effects of vaccination,

 it has the drawback of including in the analysis a year of shortage of the Hib vaccine.

 However, given the scale of the findings of this study, this consideration likely had minimal impact.

Vaccination of children born in the United States in 2009 will save 1.2 million quality-adjusted life-years, generating $184 billion in social value net of vaccination costs.

The Social Value of Childhood Vaccination in the United States

Wade M. Aubry, MD, associate clinical professor, University of California, San Francisco, continues discussions saying that comparative effectiveness research (CER) and the evolution of evidence-based medicine can facilitate innovations under the Affordable Care Act (ACA), especially with the inclusion of value-based insurance design (VBID).

Dr Aubrey notes that the ACA’s aim is to address problems in the US healthcare system that leads to high costs and low-quality healthcare delivery. Several initiatives in the ACA, like CER established by the Patient-Centered Clinical Research Network Initiative (PICORI), promote evidence-based medicine which furthers the evolution of systems like pay-for-performance initiatives, and the establishment of accountable care organizations.

Under CER, specifically in managing diabetes, there is a need for better evidence regarding alternative interventions including diagnostic, therapeutic, and other systems or strategies. VBID in diabetes has the potential to incentivize high- value services, and de-incentivize low-value services. As well, ACA recommendations made by the US Preventive Services Task Force pushes screening for diabetes as a high-value preventive services that must be included in VBID health plans.

“I think the future of CER is good and that it will increasingly be applied to these types of value-based insurance designs, formularies, or medical policies with differentials copays to promote high-value care and discourage low value care,” Dr Aubry said.

Wade M. Aubry, MD

Articles

Understanding and Improving Value Frameworks With Real-World Patient Outcomes

The Social Value of Childhood Vaccination in the United States

Session 2: Discussant