Currently Viewing:
The American Journal of Managed Care July 2015
Low-Value Care for Acute Sinusitis Encounters: Who's Choosing Wisely?
Adam L. Sharp, MD, MS; Marc H. Klau, MD, MBA; David Keschner, MD, JD; Eric Macy, MD, MS; Tania Tang, PhD, MPH; Ernest Shen, PhD; Corrine Munoz-Plaza, MPH; Michael Kanter, MD; Matthew A. Silver, MD; and Michael K. Gould, MD, MS
Usefulness of Pharmacy Claims for Medication Reconciliation in Primary Care
Dominique Comer, PharmD, MS; Joseph Couto, PharmD, MBA; Ruth Aguiar, BA; Pan Wu, PhD; and Daniel J. Elliott, MD, MSCE
No Longer a Unicorn: Improving Health Through Accountable Care Organizations
Risa Lavizzo-Mourey, MD, MBA, president and CEO, The Robert Wood Johnson Foundation
ACA-Mandated Elimination of Cost Sharing for Preventive Screening Has Had Limited Early Impact
Shivan J. Mehta, MD, MBA; Daniel Polsky, PhD; Jingsan Zhu, MBA; James D. Lewis, MD, MSCE; Jonathan T. Kolstad, PhD; George Loewenstein, PhD; and Kevin G. Volpp, MD, PhD
Determinants of Medicare Plan Choices: Are Beneficiaries More Influenced by Premiums or Benefits?
Paul D. Jacobs, PhD; and Melinda B. Buntin, PhD
Acupuncture and Chiropractic Care: Utilization and Electronic Medical Record Capture
Charles Elder, MD, MPH; Lynn DeBar, PhD, MPH; Cheryl Ritenbaugh, PhD, MPH; William Vollmer, PhD; Richard A. Deyo, MD, MPH; John Dickerson, PhD; and Lindsay Kindler, PhD, RN
Worksite Medical Home: Health Services Use and Claim Costs
Christopher Conover, PhD; Rebecca Namenek Brouwer, MS; Gale Adcock, MSN, RN, FNP-BC, FAANP; David Olaleye, PhD, MSCe; John Shipway, BS; and Truls Østbye, MD, PhD
Currently Reading
Effectiveness and Cost of Influenza Vaccine Reminders for Adults With Asthma or Chronic Obstructive Pulmonary Disease
Jo Ann Shoup, MS; Carlos Madrid, MA; Caroline Koehler, RN, MSN; Cynthia Lamb, BS, RN; Jennifer Ellis, MSPH; Debra P. Ritzwoller, PhD; and Matthew F. Daley, MD
It's Time for CMS to Utilize Functional Assessment as a Key Quality Component
Wayne Matthews, MS, PA-C, DFAAPA
Colorectal Cancer Screening in the 21st Century: Where Do We Go From Here?
Jennifer K. Maratt, MD; and Sameer D. Saini, MD, MS
Pilot of Decision Support to Individualize Colorectal Cancer Screening Recommendations
Amy R. Schwartz, MD; Forrest L. Levin, MS; Joseph A. O'Neil, Jr, BS; and R. Scott Braithwaite, MD, MSc

Effectiveness and Cost of Influenza Vaccine Reminders for Adults With Asthma or Chronic Obstructive Pulmonary Disease

Jo Ann Shoup, MS; Carlos Madrid, MA; Caroline Koehler, RN, MSN; Cynthia Lamb, BS, RN; Jennifer Ellis, MSPH; Debra P. Ritzwoller, PhD; and Matthew F. Daley, MD
Interactive voice response reminders had neither a positive nor a negative effect on promoting influenza vaccination over reminders via postcards, but are a potentially less expensive option.

Objectives: To assess the effectiveness and cost of interactive voice response (IVR) reminders for influenza vaccination compared with postcards, among adults with asthma or chronic obstructive pulmonary disease (COPD).

Study Design: Pragmatic, 3-arm, randomized control trial.    

Methods: The trial was conducted in an integrated healthcare organization during 2012 and 2013, using an existing IVR system. All adults aged 19 through 64 years with asthma or COPD (n = 12,285) were randomized to receive 1 of the following vaccination reminders: 1) postcard reminder only, 2) IVR reminder only, or 3) postcard plus IVR reminder. The primary outcome was influenza vaccination by October 31, 2012; the secondary outcomes were influenza vaccination by December 31, 2012, and by March 31, 2013.

Results: For subjects receiving an IVR call, 57% received a message on their answering machine; 27% answered the call; and 16% were not reached. Influenza vaccination rates were 29.5%, 31.1%, and 30.6% in the postcard-only, IVR-only, and postcard-plus-IVR study arms, respectively. After controlling for relevant covariates, IVR reminders were not significantly more or less effective than postcard reminders. Program costs were $0.78, $1.23, and $1.93 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively.  Extrapolating costs to the entire population at the study site that typically receives influenza vaccination reminders (approximately 100,000 individuals), reminder costs would have been $0.55, $0.05, and $0.60 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively.

Conclusions: IVR reminders are not more effective at promoting influenza vaccination than postcard reminders, but IVR reminders may be less expensive for large patient populations.

Am J Manag Care. 2015;21(7):e405-e413
Take-Away Points
This study was conducted in an integrated healthcare delivery system. Adults with asthma or chronic obstructive pulmonary disease were randomized to receive interactive voice response (IVR) calls, postcards, or both as a reminder for influenza vaccination.
  • Influenza vaccination rates were not significantly higher or lower among those who received IVR calls versus postcards.
  • When costs were extrapolated to the entire population at the study site that typically receives influenza vaccination reminders (approximately 100,000 individuals), IVR was the least costly reminder method.
  • Based on study findings, IVR was adopted as the primary strategy for annual influenza vaccination reminders at the study site.
Influenza infections cause substantial morbidity and mortality every year in the United States.1 Individuals with chronic medical conditions such as asthma or chronic obstructive pulmonary disease (COPD) are at increased risk of influenza-associated morbidity compared with the general population.2-5 Influenza vaccination is 1 of the primary means of reducing this disease burden, and while annual vaccination is recommended for everyone 6 months and older,6,7 influenza vaccination is particularly important for individuals with chronic medical conditions. Despite these recommendations, influenza vaccination rates in individuals with chronic medical conditions8 have been consistently lower than national goals.9

Patient reminder systems have proven effective at increasing influenza vaccination rates in adults; recipients of reminders typically have vaccination rates 15 percentage points higher than those who do not receive reminders, though effectiveness has varied widely across published studies.10,11 Most prior studies of influenza vaccination reminders have relied on letter, postcard, or auto-dialer reminders12-14—methods that are not interactive and not easily tailored to individual patients. However, newer technologies such as interactive voice response (IVR) systems create opportunities to provide more individualized reminders, which may prove more effective. IVR systems can gather information from call recipients and use this information within complex branching logic to provide a more tailored message.15 While IVR systems have been used for a variety of health promotion and disease management purposes,16-18 such systems have not been used extensively for influenza vaccination reminders. In addition, few studies have examined the cost of IVR systems compared with standard mail or auto-dialer–based reminders.

To address these gaps in knowledge, a study was conducted to examine the effectiveness and cost of different types of influenza vaccination reminders among adults with asthma or COPD. The specific objectives of this study were: 1) to assess the effectiveness of IVR reminders (either alone or in conjunction with postcard reminders) compared with postcard reminders only; 2) to determine the cost associated with each reminder method; and 3) to estimate the projected cost of these reminder methods if they were used in the future for all high-risk adults and children at the study site, a large managed care organization.


Study Setting

The study was conducted between July 2012 and March 2013 in Kaiser Permanente Colorado (KPCO), a managed care organization with approximately 480,000 members in the metropolitan Denver area. KPCO uses an electronic health record (EHR), which captures demographic data, health plan enrollment information, encounter data including diagnosis codes, and immunization administration information. The local human subjects review board approved the study and written consent was not required.

Study Population

All adults aged 19 to 64 years at KPCO with a diagnosis of asthma or COPD were identified. Children and the elderly were not included in the trial, because they were already receiving vaccination reminders based upon their age. Subjects with asthma were included if they had an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code of asthma (493.x) in the prior 3 years. From this population, subjects were excluded if they had no dispensing of an asthma-related medication in the prior 2 years. Subjects with COPD were included if they had an ICD-9-CM diagnosis code of COPD (491.x, 492.x, 493.2, and 496.x) at any time in the past. From the population with asthma or COPD, subjects were excluded if they lived in a household with other individuals in high-risk categories for influenza morbidity,2-4 because these households were already scheduled to receive postcard reminders for influenza vaccination as usual care.

Study Design and Randomization

A 3-arm, randomized control trial was conducted of different reminder strategies for annual influenza vaccination. Subjects were aware of what type of reminder they received; however, the study aims were not described in the reminders. Randomization was performed by simple random allocation with no restrictions.

Reminder Intervention

At KPCO, it is considered usual care for every individual with asthma or COPD to receive an annual postcard reminder for influenza vaccination. In the current study, subjects received 1 of the following interventions: 1) postcard reminder only (usual care); 2) IVR reminder only; or 3) postcard-plus-IVR reminder. The content of the postcard and IVR reminders was similar: subjects were encouraged to receive influenza vaccination; groups at increased risk from influenza were highlighted; subjects were informed that no appointment was needed for vaccination; and subjects were told that vaccination was provided at no cost. However, subjects receiving IVR reminders could access additional information during the IVR call, as described below.

Postcards were mailed to subjects via standard mail during the last 2 weeks of September 2012. As is typical for standard postcards, no “return request” was made on the study postcards; therefore, it was not possible to know how many postcards were undeliverable.

An existing IVR system was used to contact subjects by telephone. The caller identification displayed “Kaiser Permanente” on the subject’s phone. The IVR reminders were designed to be interactive; using the numbers on a touch-tone telephone, subjects could listen to general information about influenza vaccination and hear a message from an asthma/COPD specialist at KPCO, with the option to listen to additional information about influenza infections if desired. A maximum of 2 telephone calls were made per subject. If the IVR system reached an answering machine, a message was left encouraging influenza vaccination. Calls were made to the primary listed telephone number in the EHR, and the IVR system requested the responder to verify their identity. The IVR system tracks when the call ends, providing specific information on how much call content each subject received. If after 2 attempts the IVR system was not able to detect a person or answering machine, the call was classified as not delivered. IVR calls were made during the last 2 weeks of September 2012; subjects in the postcard-plus-IVR reminder group may have received their IVR call before or after their postcard.

Outcome Measures

The primary study outcome was receipt of influenza vaccine by October 31, 2012, as documented in the EHR. This date was chosen because we speculated that any impact of reminders on behavior was most likely to occur within the month following the intervention. As secondary outcomes, receipt of influenza vaccine by December 31, 2012, and by March 31, 2013, were examined. Additionally, the costs of each of the 3 interventions were examined; costs were calculated for the study population and were also extrapolated to the entire population at KPCO that typically receives influenza vaccination reminders (approximately 100,000 individuals).

Statistical Analyses

Vaccination rates were compared among the 3 study arms using pairwise comparison statistics. Wald asymptotic confidence limits were used to test for differences in rates. Analyses were based on intention-to-treat.

Multivariable analyses were used to examine the effect of reminder type on receipt of influenza vaccine, while controlling for relevant covariates. For these analyses, the dependent variable was vaccination (“yes” or “no”), and the primary predictor variable was the study arm. Other covariates included age, sex, race, Hispanic ethnicity, Chronic Disease Score (CDS),19 Charlson comorbidity index (CCI) score,20 insurance type, KPCO group coverage (employees of the healthcare system and their family members), outpatient visit rate, and prior receipt of influenza vaccine. CDS and CCI are comorbidity measures; CDS is based on current medication use, while CCI is based on diagnosis codes. A Poisson regression model with robust error variance21 was used to examine the relative risk of receipt of vaccine. Each demographic and clinical covariate of interest was tested individually for its association with risk of vaccination. A priori, study arm, age, gender, race, and Hispanic ethnicity were included in a multivariable model. Other covariates with P <.20 in bivariable models were included in preliminary models; those with an adjusted P ≥.05 were removed in a step-wise fashion to arrive at a final model. CCI was removed from the final model due to an adjusted P ≥.05. All analyses were performed using SAS version 9.2 (SAS Institute, Cary, North Carolina).

Intervention Cost Analyses

The methods used to estimate intervention costs associated with the 3-arm trial were consistent with those used in other intervention trials conducted at KPCO and elsewhere.22-24 All intervention-related costs associated with the postcard and IVR reminders were measured. All staff tracked their time spent on reminder activities. Measured personnel hours were converted into costs, based on salary and benefits per labor category, using the highest wage range of the 2011 United States Bureau of Labor Statistics and a fringe rate of 35%.

In addition to labor costs, other intervention-related expenses were accounted for. Costs for supplies, printed materials, mailing of postcards, and the costs associated with the IVR system (telephone lines, server, licensing) were recorded. The IVR system is used for other interventions within KPCO; therefore, only the portion of IVR system costs that were associated with developing and delivering the influenza vaccination reminders was used in cost calculations. The cost per subject of each study arm was calculated by dividing the number of participants in the study arm by the total costs for the intervention in that arm.

Extrapolation of Costs

In prior years, KPCO sent approximately 100,000 postcard reminders for influenza vaccination to members at increased risk of influenza-related morbidity and mortality. While this study was conducted among patients with asthma or COPD, KPCO planned to use these results to make decisions about reminder methods for the entire high-risk population. Therefore, additional analyses were conducted to extrapolate cost estimates to a population size of 100,000. For extrapolation, it was assumed that the costs associated with cohort identification, content development, and voice talent would be fixed (ie, would not vary with the number of patients contacted), but mailing cost would vary directly with the number of mailed reminders. IVR costs associated with licensing fees and phone lines would increase only when additional capacity was needed.


Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up