The American Journal of Managed Care October 2006
Implementation of Evidence-based Alcohol Screening in the Veterans Health Administration
In 2004, the VA also used patient satisfaction surveys to assess alcohol misuse and 1 element of brief alcohol counseling. Each month, the VA mails a patient satisfaction survey, the Survey of Healthcare Experiences of Patients (SHEP), to a random sample of about 29 000 patients who had a VA clinic visit in the previous month (response rate, >70%). SHEP sampling varied over time in 2004 (footnotes to Table 3), and unlike EPRP sampling (which did not include new patients) SHEP included patients new to the VA. Four alcohol-related questions were included in SHEP in 2004. These included the AUDIT-C to identify patients who screened positive for alcohol misuse and the following question to assess brief alcohol counseling: "In the past 12 months has a VA doctor or other VA health care provider advised you about your drinking (to drink less or not to drink alcohol)?" Similar questions about alcohol-related advice have been used in previous research,41 because advice has been a consistent component of efficacious brief alcohol counseling,4,6 recollection of physician advice is associated with improved outcomes for patients in alcohol treatment,46 and 5 minutes of advice is as effective as longer brief alcohol counseling.20 However, such questions may not capture all alcohol counseling, for example, when counseling does not include explicit advice or if patient embarrassment or social desirability leads to underreporting. Therefore, the SHEP survey may also underestimate brief alcohol counseling.
Although EPRP medical record reviews and SHEP surveys use different sampling methods and measures (footnotes to Table 3), so that statistical comparison of results of the 2 systems is not appropriate, descriptive comparisons of findings indicate some of the strengths and limitations of each system. Based on SHEP for 2004, 22% of respondents screened positive on the AUDIT-C, with less variation across networks compared with the rates based on EPRP for January-March 2005 (Table 3). Although rankings of the 21 networks based on the prevalence of positive AUDIT-Cs from EPRP and SHEP were generally similar, some differed markedly (eg, network K in Table 3 ranked 18th on EPRP vs 3rd on SHEP). Moreover, the difference between the prevalence rates based on EPRP and SHEP was large for some networks (eg, network T [11% vs 21%]). These observed differences may be related to differences in eligible samples, sample sizes, data sources (medical record documentation vs mailed survey), or other factors. When VA networks were divided into terciles based on rankings of the prevalence of alcohol misuse according to the 2 systems (EPRP and SHEP), only 2 networks were in the top tercile in one system and in the bottom tercile in the other system for the prevalence of alcohol misuse (data not shown).
Direct comparison of findings from EPRP and SHEP for evidence that clinicians addressed alcohol misuse is more difficult because of large differences in samples and measures (Table 3). Moreover, because EPRP samples were not designed to assess subgroups of patients, small numbers may limit the reliability. Therefore, the network rates of alcohol-related advice reported by patients who screened positive for alcohol misuse on SHEP were less variable (20%-36%) than the network rates of documented assessment for alcohol use disorders on EPRP (5%-84%). As in previous research,39 the rates of advice reported by patients on surveys increased as the severity of alcohol misuse increased (for an AUDIT-C score of 3-4 points, 13% were advised; for 5-7 points, 32% were advised; and for 8-12 points, 56% were advised). The low rate of alcohol-related advice among patients with the lowest positive AUDIT-C scores (4 points for men and 3-4 points for women) may reflect the fact that some patients with these low scores report drinking within recommended limits and might have false-positive alcohol screening results.
SUMMARY AND FUTURE DIRECTIONS
To summarize, the VA successfully implemented a new evidence-based alcohol screening program in more than 800 outpatient clinical sites nationwide in 2004, as the essential first step toward implementing brief alcohol counseling.9 Lessons from these initial implementation efforts have several implications for other systems.
First, the decision to implement the performance measure requiring all VAs to screen for the whole spectrum of alcohol misuse, including risky drinking and alcohol use disorders, was made by the highest leadership in the VA, motivated by the prevalence of alcohol misuse and patients' reports that they were not receiving the help they needed for their drinking.47 Moreover, leaders in each network and facility were held personally accountable for implementing screening for alcohol misuse. However, the implementation did not include active collaboration with each site, unlike many recent implementation efforts that use the more intensive quality improvement collaborative methods.48 The national performance measure effectively created demand for education about brief alcohol counseling. Moreover, deimplementation of screening for alcohol use disorders with the CAGE questionnaire alone and implementation of AUDIT-C screening occurred rapidly, more by diffusion than active dissemination.49 Such leadership commitment, incentives, and system readiness are essential components of successful implementation of new technologies,49 which may have been lacking in previous efforts to implement brief alcohol screening and counseling.9,22,30
Second, timely availability of a self-scoring electronic clinical reminder for screening with the AUDIT-C, used more than 1.5 million times in the first year, appeared to facilitate diffusion. We suspect that this optional electronic clinical reminder was widely adopted because (1) it was developed and disseminated by a national VA clinical reminder opinion leader before the performance measure took effect and (2) it incorporated an efficient feature (self-scoring of the AUDIT-C) that could not be easily programmed locally. The fact that an office-based version of CPRS will be widely available suggests that such a system may soon be within reach of providers outside the VA.
Third, implementation of alcohol screening is only the first step, and screening alone has not dramatically increased brief alcohol counseling.41 Although this could be viewed as an unsuccessful effort to implement brief alcohol counseling, it can also be viewed as a highly successful effort at implementing this essential prerequisite to brief alcohol counseling, namely, screening for alcohol misuse with a validated questionnaire. Moreover, even with implementation of alcohol misuse screening alone, 28% of those who screened positive for alcohol misuse (7% of the screened sample) reported brief advice in the past year. This suggests that the low estimates of the proportion of primary care patients who can benefit from brief alcohol counseling that have been extrapolated from the recruitment rates for clinical trials (eg, 2.5% in the study by Beich et al15) may underestimate the number of patients who could benefit from brief alcohol counseling even with screening alone. In fact, the wisdom of the VA Office of Quality and Performance in initiating screening as a distinct step, before implementation of brief alcohol counseling, may ultimately allow it to succeed where other efforts have failed. This may be one of the greatest lessons of this implementation effort for other organizations wishing to implement brief alcohol counseling.
Further implementation of brief alcohol counseling will likely require a performance measure reflecting appropriate follow-up for alcohol misuse. Previous research findings suggest that primary care providers will require education about the efficacy of brief alcohol counseling and training in the development of effective skills for brief alcohol counseling.50 The most feasible approach to developing these provider skills might be a crosscutting educational program that addresses behavior change counseling in general (eg, for smoking cessation and obesity), including skills in motivational interviewing51 and other common components of behavior change counseling.6,52 However, counseling about alcohol misuse may also present primary care providers with unique challenges that require training specifically focused on brief alcohol counseling.53-55 As the VA moves ahead with further implementation of brief alcohol counseling, much could be learned from a formal prospective evaluation of that process.
Fourth, performance measures create powerful incentives, intended and unintended. For example, the alcohol misuse screening performance measure created an incentive for documented alcohol screening but no incentive for maximizing the validity of that screening. The differences in the estimated prevalence of alcohol misuse based on medical record documentation and patient survey, if confirmed by analyses among a single cohort, suggest that the next generation of the alcohol misuse screening performance measure should include incentives to maximize screening validity (eg, by ensuring privacy, asking screening questions verbatim, and using standardized assessments for nondrinkers and any other exclusion criteria).56
Fifth, the sample and method used for any future performance measure of brief alcohol counseling will need to be carefully considered and pilot tested. The current program of medical record reviews was not designed to include adequate numbers of patients with alcohol misuse to provide precise estimates of follow-up at the facility level. Therefore, if medical record review is used to measure performance for follow-up brief alcohol counseling, the EPRP sampling strategy would need to change. Mailed patient satisfaction surveys or nationally mandated electronic clinical reminders that send results to a central data repository may offer alternative approaches for national monitoring of follow-up for alcohol misuse. Ultimately, as noted in a recent Institute of Medicine report,57 performance measurement of alcohol screening and brief alcohol counseling will be facilitated by the development of International Classification of Diseases codes for risky drinking, to complement existing codes for alcohol abuse and dependence.57 In addition, the development of practical Current Procedural Terminology and Centers for Medicare & Medicaid Services Healthcare Common Procedure Coding System codes for alcohol screening and brief alcohol counseling could further improve nationwide performance measurement for alcohol screening and brief alcohol counseling,57 complementing the system of recently developed performance measures for alcohol use disorders diagnosis and treatment.58