Effectiveness and Cost of Influenza Vaccine Reminders for Adults With Asthma or Chronic Obstructive Pulmonary Disease

Jo Ann Shoup, MS; Carlos Madrid, MA; Caroline Koehler, RN, MSN; Cynthia Lamb, BS, RN; Jennifer Ellis, MSPH; Debra P. Ritzwoller, PhD; and Matthew F. Daley, MD

Influenza infections cause substantial morbidity and mortality every year in the United States.1 Individuals with chronic medical conditions such as asthma or chronic obstructive pulmonary disease (COPD) are at increased risk of influenza-associated morbidity compared with the general population.2-5 Influenza vaccination is 1 of the primary means of reducing this disease burden, and while annual vaccination is recommended for everyone 6 months and older,6,7 influenza vaccination is particularly important for individuals with chronic medical conditions. Despite these recommendations, influenza vaccination rates in individuals with chronic medical conditions8 have been consistently lower than national goals.9
Patient reminder systems have proven effective at increasing influenza vaccination rates in adults; recipients of reminders typically have vaccination rates 15 percentage points higher than those who do not receive reminders, though effectiveness has varied widely across published studies.10,11 Most prior studies of influenza vaccination reminders have relied on letter, postcard, or auto-dialer reminders12-14—methods that are not interactive and not easily tailored to individual patients. However, newer technologies such as interactive voice response (IVR) systems create opportunities to provide more individualized reminders, which may prove more effective. IVR systems can gather information from call recipients and use this information within complex branching logic to provide a more tailored message.15 While IVR systems have been used for a variety of health promotion and disease management purposes,16-18 such systems have not been used extensively for influenza vaccination reminders. In addition, few studies have examined the cost of IVR systems compared with standard mail or auto-dialer–based reminders.
To address these gaps in knowledge, a study was conducted to examine the effectiveness and cost of different types of influenza vaccination reminders among adults with asthma or COPD. The specific objectives of this study were: 1) to assess the effectiveness of IVR reminders (either alone or in conjunction with postcard reminders) compared with postcard reminders only; 2) to determine the cost associated with each reminder method; and 3) to estimate the projected cost of these reminder methods if they were used in the future for all high-risk adults and children at the study site, a large managed care organization.
Study Setting
The study was conducted between July 2012 and March 2013 in Kaiser Permanente Colorado (KPCO), a managed care organization with approximately 480,000 members in the metropolitan Denver area. KPCO uses an electronic health record (EHR), which captures demographic data, health plan enrollment information, encounter data including diagnosis codes, and immunization administration information. The local human subjects review board approved the study and written consent was not required.
Study Population
All adults aged 19 to 64 years at KPCO with a diagnosis of asthma or COPD were identified. Children and the elderly were not included in the trial, because they were already receiving vaccination reminders based upon their age. Subjects with asthma were included if they had an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code of asthma (493.x) in the prior 3 years. From this population, subjects were excluded if they had no dispensing of an asthma-related medication in the prior 2 years. Subjects with COPD were included if they had an ICD-9-CM diagnosis code of COPD (491.x, 492.x, 493.2, and 496.x) at any time in the past. From the population with asthma or COPD, subjects were excluded if they lived in a household with other individuals in high-risk categories for influenza morbidity,2-4 because these households were already scheduled to receive postcard reminders for influenza vaccination as usual care.
Study Design and Randomization
A 3-arm, randomized control trial was conducted of different reminder strategies for annual influenza vaccination. Subjects were aware of what type of reminder they received; however, the study aims were not described in the reminders. Randomization was performed by simple random allocation with no restrictions.
Reminder Intervention
At KPCO, it is considered usual care for every individual with asthma or COPD to receive an annual postcard reminder for influenza vaccination. In the current study, subjects received 1 of the following interventions: 1) postcard reminder only (usual care); 2) IVR reminder only; or 3) postcard-plus-IVR reminder. The content of the postcard and IVR reminders was similar: subjects were encouraged to receive influenza vaccination; groups at increased risk from influenza were highlighted; subjects were informed that no appointment was needed for vaccination; and subjects were told that vaccination was provided at no cost. However, subjects receiving IVR reminders could access additional information during the IVR call, as described below.
Postcards were mailed to subjects via standard mail during the last 2 weeks of September 2012. As is typical for standard postcards, no “return request” was made on the study postcards; therefore, it was not possible to know how many postcards were undeliverable.
An existing IVR system was used to contact subjects by telephone. The caller identification displayed “Kaiser Permanente” on the subject’s phone. The IVR reminders were designed to be interactive; using the numbers on a touch-tone telephone, subjects could listen to general information about influenza vaccination and hear a message from an asthma/COPD specialist at KPCO, with the option to listen to additional information about influenza infections if desired. A maximum of 2 telephone calls were made per subject. If the IVR system reached an answering machine, a message was left encouraging influenza vaccination. Calls were made to the primary listed telephone number in the EHR, and the IVR system requested the responder to verify their identity. The IVR system tracks when the call ends, providing specific information on how much call content each subject received. If after 2 attempts the IVR system was not able to detect a person or answering machine, the call was classified as not delivered. IVR calls were made during the last 2 weeks of September 2012; subjects in the postcard-plus-IVR reminder group may have received their IVR call before or after their postcard.

Outcome Measures
The primary study outcome was receipt of influenza vaccine by October 31, 2012, as documented in the EHR. This date was chosen because we speculated that any impact of reminders on behavior was most likely to occur within the month following the intervention. As secondary outcomes, receipt of influenza vaccine by December 31, 2012, and by March 31, 2013, were examined. Additionally, the costs of each of the 3 interventions were examined; costs were calculated for the study population and were also extrapolated to the entire population at KPCO that typically receives influenza vaccination reminders (approximately 100,000 individuals).
Statistical Analyses
Vaccination rates were compared among the 3 study arms using pairwise comparison statistics. Wald asymptotic confidence limits were used to test for differences in rates. Analyses were based on intention-to-treat.
Multivariable analyses were used to examine the effect of reminder type on receipt of influenza vaccine, while controlling for relevant covariates. For these analyses, the dependent variable was vaccination (“yes” or “no”), and the primary predictor variable was the study arm. Other covariates included age, sex, race, Hispanic ethnicity, Chronic Disease Score (CDS),19 Charlson comorbidity index (CCI) score,20 insurance type, KPCO group coverage (employees of the healthcare system and their family members), outpatient visit rate, and prior receipt of influenza vaccine. CDS and CCI are comorbidity measures; CDS is based on current medication use, while CCI is based on diagnosis codes. A Poisson regression model with robust error variance21 was used to examine the relative risk of receipt of vaccine. Each demographic and clinical covariate of interest was tested individually for its association with risk of vaccination. A priori, study arm, age, gender, race, and Hispanic ethnicity were included in a multivariable model. Other covariates with P <.20 in bivariable models were included in preliminary models; those with an adjusted P ≥.05 were removed in a step-wise fashion to arrive at a final model. CCI was removed from the final model due to an adjusted P ≥.05. All analyses were performed using SAS version 9.2 (SAS Institute, Cary, North Carolina).
Intervention Cost Analyses
The methods used to estimate intervention costs associated with the 3-arm trial were consistent with those used in other intervention trials conducted at KPCO and elsewhere.22-24 All intervention-related costs associated with the postcard and IVR reminders were measured. All staff tracked their time spent on reminder activities. Measured personnel hours were converted into costs, based on salary and benefits per labor category, using the highest wage range of the 2011 United States Bureau of Labor Statistics and a fringe rate of 35%.
In addition to labor costs, other intervention-related expenses were accounted for. Costs for supplies, printed materials, mailing of postcards, and the costs associated with the IVR system (telephone lines, server, licensing) were recorded. The IVR system is used for other interventions within KPCO; therefore, only the portion of IVR system costs that were associated with developing and delivering the influenza vaccination reminders was used in cost calculations. The cost per subject of each study arm was calculated by dividing the number of participants in the study arm by the total costs for the intervention in that arm.
Extrapolation of Costs
In prior years, KPCO sent approximately 100,000 postcard reminders for influenza vaccination to members at increased risk of influenza-related morbidity and mortality. While this study was conducted among patients with asthma or COPD, KPCO planned to use these results to make decisions about reminder methods for the entire high-risk population. Therefore, additional analyses were conducted to extrapolate cost estimates to a population size of 100,000. For extrapolation, it was assumed that the costs associated with cohort identification, content development, and voice talent would be fixed (ie, would not vary with the number of patients contacted), but mailing cost would vary directly with the number of mailed reminders. IVR costs associated with licensing fees and phone lines would increase only when additional capacity was needed.
A total of 12,428 adults aged 19 to 64 years were identified with asthma or COPD. As shown in the Figure, 141 subjects were excluded due to having no available telephone number, and 2 were excluded due to having no mailing address. The remaining 12,285 subjects were randomly allocated to 1 of the 3 study arms, resulting in 4095 subjects per arm.
Subjects’ baseline demographic and clinical characteristics are summarized in Table 1. Asthma was the qualifying condition for 94% of the population, COPD for 5%, and asthma with COPD for 1%.
Information on completion of the reminder calls, for the IVR-only and the postcard-plus-IVR arms, was as follows: for 56.6% and 57.9%, respectively, messages were left on answering machines; 5.6% and 5.2%, respectively, listened to a vaccination reminder but hung up prior to receiving additional information; 20.4% and 19.3%, respectively, listened through to the message from the asthma/COPD specialist; 1.3% and 1.5%, respectively, listened to all content and requested more information from the influenza hotline; and 16% and 16% of calls, respectively, were incomplete with no answer and no message left.
By October 31, 2012, 29.5% of subjects in the postcard-only arm, 31.1% in the IVR-only arm, and 30.6% in the postcard-plus-IVR arm had received influenza vaccination. As shown in Table 2, no significant difference was found in vaccination rates among the 3 study arms for the primary outcome. Table 2 also shows influenza vaccination receipt by December 31, 2012, and March 31, 2013.
Multivariable analyses of factors associated with receipt of influenza vaccination are shown in Table 3. After adjusting for other covariates, vaccination was not significantly higher for the IVR-only and postcard-plus-IVR arms compared with the postcard-only arm. Of interest, several covariates were significantly associated with vaccination, which was significantly more likely in older age groups, for example. Compared with those with a traditional managed care insurance plan, those with a deductible plan were significantly less likely to receive influenza vaccine (adjusted odds ratio, 0.89; 95% CI, 0.84-0.94).
The cost associated with the different types of reminders is shown in Table 4. In the postcard-only arm, the cost per subject to receive the intervention was $0.78; for IVR-only, the cost was $1.23; and for postcard-plus-IVR, the cost was $1.93. When these costs were extrapolated to a theoretical population of 100,000 subjects, the anticipated costs per subject would be $0.55, $0.05, and $0.60 in the postcard-only, IVR-only, and postcard-plus-IVR groups, respectively. We performed an additional analysis to find the “break-even” point; 7916 postcards could be sent for the same cost as 100,000 IVR calls.
In this investigation of different reminder methods for influenza vaccination among adults with asthma or COPD, using IVR-based reminders was not significantly more or less effective for promoting influenza vaccination compared with postcard reminders or postcard-plus-IVR reminders. In addition, the IVR calls were not as interactive or tailored as was intended, as only 20% listened to a message from an asthma/COPD specialist, and only 2% requested information from the influenza hotline. The intervention achieved only modest influenza vaccination rates of 45% to 47% by the end of the influenza season, and the strongest predictor of vaccination was receipt of influenza vaccine in prior years. In cost analyses in the study population of 12,285 individuals, IVR reminders were more costly than postcard reminders due to the higher fixed costs of the IVR system. However, when costs were extrapolated to the entire high-risk population at KPCO, IVR-based reminders would have been the least expensive among the methods tested. These findings led KPCO to use IVR-based reminders as the primary method for influenza vaccination reminders for the entire high-risk population at KPCO for the 2013-2014 influenza season.
Although it was anticipated that IVR reminders would be more effective than postcard reminders for encouraging influenza vaccination, vaccination rates by October 31, 2012, were similar across the 3 study arms. IVR is considered a promising tool for health promotion and disease management because messages can be personalized and respondents can interact with the system through voice or touch-tone responses via telephone.25-27 However, in this study, use of the IVR system did not result in substantial interaction with respondents, because most either had a message left on an answering machine or did not choose to receive additional influenza information through the hotline. It is possible that higher immunization rates could have been achieved had more individuals listened to the additional information provided.
Nationally, during the 2012-2013 influenza season, an estimated 47% of adults aged 19 to 64 years with chronic medical conditions were immunized against influenza,8 and 34.6% of adults with asthma aged 18 to 49 years were immunized.28 These influenza immunization rates are considerably lower than the national goal of 90% coverage.9 The current study was conducted among insured adults in a managed care organization with walk-in immunization available—a setting presumably with fewer barriers to vaccination than faced by the general population. However, across the 3 study arms, only 45% to 47% were immunized by the end of the influenza season; several factors may have contributed to the lower-than-expected rates. Because KPCO patients are sent reminders every year and are potentially exposed to other types of reminders within KPCO (eg, in Kaiser newsletters), awareness may be high enough that patients do not need the additional cue to action that reminders provide. Additionally, it is possible some subjects were vaccinated outside of KPCO, and that this information was not entered into the KPCO EHR, resulting in falsely low rates.29 Finally, misperceptions about the need for influenza vaccination are prevalent, even among individuals with chronic medical conditions30,31; it is possible that the brief reminders used in the current study did little to change attitudes and behaviors.
While annual influenza vaccination reminders have proven effective in a variety of settings when compared with no reminders, much less is known about the cost of various reminder methods. Interestingly, IVR-based reminders were more costly than postcards for the study population of 12,285, but substantially less costly than postcards when applied to a much larger population, a finding that led KPCO to adopt IVR-based reminders for the entire high-risk population for the subsequent influenza season.
This study is subject to several important limitations. The study did not include a control group that received no reminders; however, because vaccination reminders are a recommended standard of care nationally, and a long-standing part of usual care at KPCO, having a “no reminder” study arm was not appropriate on ethical grounds. Reminders were left on answering machines, but it is not known whether these messages were ultimately heard by the intended recipients. Some subjects may have received influenza vaccination outside of KPCO, such as at a pharmacy or workplace, and this information would not routinely be captured within the EHR.29 While standard cost-capture methods were used, it is possible that not all reminder costs were measured. Finally, IVR systems can be expensive, costing $50,000 or more to initially implement or purchase. The IVR purchase or start-up costs were not included in our IVR reminder cost estimates because the KPCO IVR system has been in place for more than a decade, it is currently used for multiple purposes, and it is unlikely that a healthcare entity would purchase an IVR system solely for influenza vaccination reminders. This may limit the generalizability of our cost findings to organizations with existing IVR systems or those willing to purchase one, and we are unaware of published estimates of how prevalent IVR systems are among healthcare organizations.
In summary, in a randomized control trial, IVR-based reminders were not more or less effective for promoting annual influenza vaccination than postcard reminders. While IVR is a promising new technology for health promotion, when IVR-based reminders were used for influenza vaccination, many patients did not receive any tailored messaging as intended. Regardless of the type of reminder used in this study, only modest influenza vaccination rates were achieved, indicating that additional strategies may be needed to overcome misperceptions about the need for annual influenza vaccination.

We gratefully acknowledge Jeff Holzman for data management and Jonah Langer and Jenn Boggs for their technical assistance with the IVR delivery system.
Print | AJMC Printing...