Published Online: June 15, 2009
Natalie Oake, MSc; Alison Jennings, MA; Carl van Walraven, MD, FRCPC, MSc; and Alan J. Forster, MD, FRCPC, MSc
Objective: To comprehensively describe the populations, interventions, and outcomes of interactive voice response system (IVRS) clinical trials.
Methods: We identified studies using MEDLINE (1950-2008) and EMBASE (1980-2008). We also identified studies using hand searches of the Science Citation Index and the reference lists of included articles. Included were randomized and controlled clinical trials that examined the effect of an IVRS intervention on clinical end points, measures of disease control, process adherence, or quality-of-life measures. Continuous and dichotomous outcomes were meta-analyzed using mean difference and median effects methodology, respectively.
Results: Forty studies (n = 106,959 patients) met inclusion criteria. Of these studies, 25 used an IVRS intervention aimed at encouraging adherence with recommended tests, treatments, or behaviors; the remaining 15 used an IVRS for chronic disease management. Three studies reported clinical end points, which could not be statistically pooled. In 6 studies that reported objective clinical measures of disease control (glycosylated hemoglobin, total cholesterol, and serum glucose), the IVRS was associated with nonsignificant improvements. In 14 studies that measured objective process adherence outcomes, the median effect was 7.9% (25th-75th percentile: 2.8%, 19.5%). For the 16 studies that assessed patient-reported measures of disease control and the 11 studies that assessed patient-reported process adherence outcomes, approximately one-third of the outcomes significantly favored the IVRS group.
Conclusion: IVRS interventions, which enable patients to interact with computer databases via telephone, have shown a significant benefit in adherence to various processes of care. Future IVRS studies should include clinically relevant outcomes.
(Am J Manag Care. 2009;15(6):383-391)
A systematic review was performed to comprehensively describe the populations, interventions, and outcomes of interactive voice response system (IVRS) clinical trials.
- IVRS-based interventions are feasible in many settings and can result in modest improvements in adherence to many processes of care.
- We caution against the interpretation that IVRS improves outcomes, as there are currently insufficient data to support such a conclusion.
For high-quality ambulatory care, physicians and their healthcare team must have a high level of communication with patients between visits. These interactions frequently include the provision of treatment advice and monitoring of chronic disease.1-3 Physicians must support patients and advise them to achieve adherence with test, treatment, and behavioral recommendations. Patients who receive a timely reminder are significantly more likely to have screening tests.4-6 Similarly, the management of chronic diseases often requires ongoing assessments of various clinical parameters between visits (eg, glucose values in patients with diabetes or body weight in patients with heart failure).7-10 Patients may experience improved outcomes if their treatment is promptly modified after measurements outside of the desired treatment ranges. The capacity for physicians to fulfill such monitoring and support functions is greatly limited by various factors, including a reimbursement system that does not explicitly recognize the time required to perform them.2,3
Information and communication technologies may effectively and efficiently facilitate intervisit management.1,11,12 One such technology, the interactive voice response system (IVRS), could be used to contact patients with reminders or to track patient-assessed parameters measured at home. The IVRS is a technology that enables patients to interact with computer databases via telephones.13,14 It prompts patients to provide information following a scripted dialogue. It captures responses using keypad entry or speech recognition and stores the information in a database. Patient responses may trigger the IVRS to perform other actions such as sending electronic notifications. Other information and communication technologies, such as patient-accessible Web portals and e-mail, also could be used to support intervisit management. However, IVRS may be more easily adopted because most people own and can use a telephone.
A comprehensive analysis of the utility of IVRS-based interventions is needed. Although used in industries other than healthcare for years, IVRSs only recently were adopted for use in healthcare settings.13 As a result, their effectiveness in improving care and acceptability to patients is largely unknown. Although some data suggest they are effective, negative studies also exist. Furthermore, very few published studies of IVRS interventions have used a comprehensive health technology assessment framework that evaluates processes and outcomes of care. A systematic review of IVRS interventions will identify gaps in the current evidence, investigate the utility of this technology, and inform future technology development. Although prior reviews of IVRS technologies were conducted in 2002 and 2003,15,16 these studies did not quantitatively examine the effects of IVRS on outcomes. Also, more than 10 IVRS randomized controlled trials (RCTs) have been published since 2003. For these reasons, we conducted a systematic review to comprehensively describe the populations, interventions, and outcomes of IVRS clinical trials.
We identified potentially pertinent citations in the MEDLINE database (1950-2008) using the search strategy in eAppendix A (available at www.ajmc.com). We used a combination of key words related to IVRS (eg, automated, telephone). We modified the MEDLINE search to identify citations in the EMBASE database (1980-2008). We also included studies found using hand searches of the Science Citation Index and the reference lists of included articles.
We retrieved the full text of articles if the title or abstract suggested that the investigators evaluated an IVRS. We included studies published in English that examined the effect of an IVRS intervention on at least 1 of the following types of outcomes: clinical end points, measures of disease control, process adherence, and quality-of-life measures. Clinical end points included disease-related outcomes such as death or hospitalization; measures of disease control were objective and patient-reported markers of disease or health status such as blood pressure, glycosylated hemoglobin (A1C), or a score on a disease-specific validated scale; process adherence outcomes assessed whether patients followed a targeted process of care such as screening tests, immunization protocols, or home glucose monitoring; and quality-of-life measures included general health scores from validated questionnaires. Only RCTs and controlled clinical trials (CCTs) (ie, trials that included at least 2 groups and used a quasi-random allocation method) were included in the review. Studies that used an IVRS only to collect data or conducted a validation study were excluded.
From each study, we abstracted details about the population, the IVRS intervention, and outcomes according to an intention-to-treat approach. If a study presented data for more than 2 groups, we abstracted data for 1 intervention group and 1 control group to capture the most direct comparison (eAppendix B available at www.ajmc.com). Where possible, we included the intervention group that received the IVRS intervention only and the control group that received no intervention. If no such groups were reported, we included the intervention group that received the IVRS intervention plus the simplest other intervention (eg, educational booklet) and the control group that received the same simple intervention. Two reviewers (NO and AJ) independently abstracted the data.
Study Quality Assessment
We assigned studies an overall quality score using the checklist specified by the Cochrane Effective Practice and Organization of Care group.17 The checklist, used for both RCTs and CCTs, included 3 primary criteria (concealment of allocation, blind assessment of primary outcome, completeness of follow-up) and 3 secondary criteria (balanced baseline measures, reliable outcome measures, protection against contamination). Studies were classified as high quality when they satisfied all primary criteria and did not elicit significant concerns regarding the secondary criteria, as moderate quality if 1 or 2 of the primary criteria were scored as “not clear” or “not done,” and as low quality if the 3 primary criteria were scored as not clear or not done.18
Meta-analyses within the various outcome categories were limited because of heterogeneity across the outcomes. In addition, a significant number of studies captured patient-reported outcomes that were not externally validated (eg, patient-reported exercise frequency). To minimize bias, outcomes that were not externally validated were not included in our quantitative analyses. Among the studies that measured outcomes that could not be externally validated (eg, pain), heterogeneity across the outcomes prevented meta-analyses. Study data could be pooled only for measures of disease control that were clinical and dichotomous process adherence outcomes.
For the different types of measures of disease control, we calculated the overall mean difference. These calculations required the mean and standard deviation for intervention and control groups. If a study did not report standard deviations, we used reported statistics (eg, P value) to impute a standard deviation for both study groups.19 Analyses were performed using Review Manager version 4.2.10 (The Cochrane Collaboration, The Nordic Cochrane Centre, Copenhagen, Denmark).
For dichotomous process adherence outcomes, we calculated the overall effect estimate using median effects methodology. 20 We first calculated the median effect, defined as the absolute percentage change between the intervention and control groups, for each study. If a study presented more than 1 dichotomous process adherence outcome, we ranked the study’s effect sizes and selected the median. Individual study effects then were ranked, and the median and interquartile range were selected as the summary process adherence measure. These analyses were performed using SAS version 9.1 (SAS Institute Inc, Cary, NC).
Change scores were calculated for objective outcomes that could not be pooled and patient-reported outcomes. The change score was defined as the difference between the change from baseline to follow-up in the intervention and control groups. If a study did not report baseline measures, the change score was the difference between follow-up measures for the 2 groups. No analyses were performed on these data.
Study Identification and Description
Our electronic search yielded 3018 citations (Figure 1). The full text of 165 citations was retrieved and reviewed. We excluded studies where the IVRS was not the study intervention (n = 88), the study design was not an RCT or CCT (n = 36), and the study outcomes could not be grouped into this study’s outcome categories (n = 6). An additional 5 included studies were retrieved by hand-searching.
PDF is available on the last page.