Screening Electronic Veterans' Health Records for Medication Discontinuation

July 18, 2012
Thomas S. Rector, PharmD, PhD
Thomas S. Rector, PharmD, PhD

Sean Nugent, BA
Sean Nugent, BA

Michele Spoont, PhD
Michele Spoont, PhD

Siamak Noorbaloochi, PhD
Siamak Noorbaloochi, PhD

Hanna E. Bloomfield, MD, MPH
Hanna E. Bloomfield, MD, MPH

Volume 18, Issue 7

Many patients stop taking medications for chronic conditions, thereby reducing the effectiveness of healthcare. An attempt to screen electronic VA healthcare records to identify patients as they discontinued a medication was not an efficient approach to this problem.


Determine the viable yield of screening electronic Veterans Health Administration (VHA) records to identify patients who stop taking a long-term medication for reasons that might be addressed by healthcare providers.

Study Design:

Prospectively screened cohort with mailed follow-up of positive screens.


Electronic healthcare records were screened to identify patients receiving care in a Veterans Administration (VA) Health Care System who became past due for resupply of medication (statin) prescribed to reduce cholesterol and risks of adverse cardiovascular events. Subsequently, administrative data were used to classify and characterize patients as true or false positive screens. A follow-up survey mailed to the fi rst 1000 positive screens asked them if they were still taking a statin provided by the VHA, and if not, why?


From February to July 2010, 1000 (4.6%) of the statin-recipient cohort of 21,935 became past due for a resupply. Subsequently 824 (3.8%; 95% confi dence interval [CI] 3.5%-4.0%) were classified as true positives (positive predictive value 82%; 95% CI 80%-85%), and 176 (0.8%; 95% CI 0.7%-0.9%) as false positives. However, the 824 true positives included 95 deceased, 17 long-term care residents, 302 who reported good reasons for no longer getting the statin, and 208 who eventually got another supply. The overall yield of good candidates for efforts to reinstate long-term use of statins was only 20%.


The viable yield from electronically screening VA healthcare records to fi nd patients who stopped taking statins was low. More complete records and sophisticated screening programs are needed to improve the yield.

(Am J Manag Care. 2012;18(7):352-358)

  •  This attempt to screen electronic healthcare records to identify patients who appeared to stop taking a long-term medication identifi ed a large number of patients, but a low yield of viable candidates for efforts to restore use of the medication.

  •  More sophisticated screening algorithms and complete healthcare records are needed to improve the actionable yield.

  •  Each patient’s reasons for stopping a medication need to be ascertained to guide remedial actions.

Studies have consistently observed that many patients stop taking prescribed medications, thereby reducing the effectiveness of care for chronic conditions.1-4 Investigators have associated numerous factors with discontinuation of medications; however, there are no prediction models that can accurately forecast when an individual will stop taking a prescribed medication or why.5,6 Integrated healthcare systems such as the Veterans Health Administration (VHA) may be able to use their electronic healthcare records to screen enrolled populations and identify patients as they appear to discontinue a medication.7 However, mass screening programs may compel healthcare systems to follow up a large number of positive screens. Thus, false positives and identifi cation of individuals that most likely would not be amenable to restoration of medication use must be minimized.8

We developed and evaluated a screening program for VHA electronic health records. The primary objective was to identify patients as they stopped taking statin medications prescribed by Veterans Administration (VA) healthcare providers to reduce their cholesterol, hence the risk of adverse cardiovascular events including death. Specifi c aims were to estimate the proportions of true and false positive screens and the positive predictive value. A follow-up survey was mailed to positive screens to ascertain the reasons they stopped getting their statin prescriptions from the VHA and to help determine the percentage of positive screens that might be good candidates for remedial intervention.

METHODSScreening Program

The screening program was developed and tested in a VA healthcare system. Similar to other VA healthcare systems, the Veterans Health Information Systems and Technology Architecture (VistA) hierarchical files contain the electronic health records for a large hospital with outpatient clinics and several regional community-based outpatient clinics. We focused on continuing use of statins because they are commonly prescribed for long-term use, and lack of persistent use has been well documented in several healthcare systems.9-12 This evaluation and consent process was reviewed and approved by the local Committee for Human Subjects Research.

Files of VHA prescriptions (new, refill, partial fill) in VistA were screened biweekly beginning in February 2010 with 9 months of prior prescription records. Eligible individuals had received at least 2 separate 30- day supplies of a statin during the screening period to exclude dose adjustments and the impact of side effects that occur early in the course of treatment. A “past due date” was calculated as the date of release of the last statin supply plus the days of supply dispensed and any carryover from previous supplies in the fi le of statin prescriptions accumulated by the screening program. In addition, the calculated past due date was postponed by any inpatient days in the patient treatment file and a 120-day grace period. This grace period was based on past studies of medication persistence,13 and a separate (different data) preliminary examination of how much false positive screening rates increased as the length of the grace period was reduced from 180 to 30 days. Whenever the Minneapolis VistA records indicated a patient was past due for a statin resupply and there was no indication the recipient was deceased, VistA files maintained by other VA healthcare systems were searched to fi nd additional statin supplies, inpatient stays, and deaths. A positive screen was identifi ed whenever a patient’s calculated past-due date occurred before the screening date. Several descriptive variables were extracted from VistA files, including the type of prescriber (physician, other); whether the last supply was a new order or refill, delivered by mail, or on formulary; patient demographics; and address information.

Follow-up Survey

A 3-page survey was mailed to the first 1000 positive screens that had a valid non-institutional postal address in the patient file. An introductory letter was sent, then the survey package, including a cover letter, preaddressed, postage-paid survey return envelope, and a $5 payment. A second survey packet without money was sent if no response was received within about 2 weeks. A third packet was sent via Federal Express or United Parcel Service after another 2 weeks without a response.

The survey instructed the recipient to look at the labels on their prescription containers, and asked whether they still had any of their statin which the VHA had most recently dispensed, and whether they were still taking any of that particular statin. If they were not taking the statin, the survey asked for the reason(s) they stopped it, using a 21-item checklist and an open-ended query. The survey asked everyone if they got their statin supply from any place other than a VHA pharmacy during the past 4 months, whether others (Medicare, Medicaid or state assistance program, employer, union or retirement insurance, military) helped them pay for their medicines, if they had stayed overnight in a non-VHA hospital or traveled away from home for a week or more during the past 4 months, and whether a doctor, nurse, or pharmacist had reduced their statin dose, or if they had done so on their own.

Administrative Data

Nine months after the last survey was sent to positive screens, scrambled social security numbers of the 1000 positive screens were used to extract information from national VHA administrative records. The VA Decision Support System (DSS) Outpatient Pharmacy National Data Extract that is extracted from all VistA systems was used to determine whether each positive screen received any more supply of any statin from the VHA before their calculated past due date (a false positive screen) or did not (a true positive screen), and whether they received a VHA statin supply within 6 months after their past due date that would be consistent with a temporary gap in supply.

Baseline variables representing the year prior to the last statin supply dispensed before the patient screened positive were extracted from several sources. Prescription variables from the DSS National Data Extract included the total number of prescription fills, number of statin fills, and whether each subject received a prescription in each of 32 different classes of medications, including those used for atherosclerotic diseases (ischemic heart disease, etc) or risk factors (blood pressure, diabetes, etc) and depression. The VHA Medical SAS data files compiled by Austin Information Technology Center provided national information on VHA outpatient encounters and inpatient stays. All diagnoses recorded during the baseline year were used to identify chronic comorbidities based on an enhanced Elixhauser classification scheme.14 In addition, categories for ischemic heart disease (ICD-9-CM codes 410 through 414), heart failure (398.91, 402.01, 402.11, 402.91, 404.03, 404.11, 404.13, 404.91, 404.93, 425.4, 425.7-9, 428), and dementia (290.1-4, 290.9, 291.1-2, 294.1, 331.0-2, 331.82) were created. Monthly enrollment records were matched to the date of each subject’s last VHA statin supply to determine the duration of VHA enrollment; enrollment priority (1-8), which, in part, determines whether there was a VHA prescription copayment; and other active health insurance including Medicaid. Dates of death were obtained retroactively from monthly enrollment records.

Statistical Analyses

Descriptive statistics, mean ± standard deviation, median (25th-75th quartiles), and proportion (percentage) as appropriate for the level of measurement and distribution of the data are reported. Binomial 95% confi dence intervals are estimated for proportions. We did not collect follow-up data on patients who did not become positive screens, thus the sensitivity, specificity, and negative predictive value could not be estimated. Group characteristics were compared by Student t test for normally distributed continuous variables, Wilcoxon rank-sum test for skewed continuous measures, or Pearson’s χ2 test for proportions. Reported P values are not adjusted for multiple comparisons. Stata software (version 10.1) was used for all analyses.


Table 1

Table 2

Within 5 months of beginning to screen the statin-recipient cohort approximately every 2 weeks, 1000 patients (4.6%; 95% confi dence interval [CI] 4.3%-4.8%) out of 21,935 became past due for a resupply of statin medication, ie, became a positive screen. Their statin prescriptions are characterized in . Consistent with the 120-day grace period and the predominant dispensing of a 90- day supply, the positive screens became past due 214 ± 23 days after their last statin supply was sent out. The positive screens received 3.1 ± 1.7 statin supplies and 249 ± 92 days of supply during the baseline year. Thus, the majority appeared to have stopped after receiving more than the required minimum of 2 statin supplies. Subsequently 824 in the statin user cohort of 21,935 (3.8%; 95% CI 3.5%-4.0%) were judged to be true positives based on not getting a resupply of any statin medication from the VHA for at least 4 months after their past supplies should have been consumed. The estimated positive predictive value was 824 out of 1000, or 82% (95% CI 80%-85%). During follow-up we discovered that 11% of the 824 true positives had died (n = 95) before their past-due date, or became nursing home residents (n = 17), where most likely they would not be personally responsible for taking or discontinuing the statin. Later DSS prescription extracts found 176 (0.8%; 95% CI 0.7%-0.9%) had received another supply before their calculated past-due date and therefore were false positives. The statin supplies defi ning false positive screens were all dispensed in February 2010, the first month of screening the prescription records. Characteristics of the false and true positives are compared in . There were no major differences in these or any other comorbidities or medication classes that are not shown, or variables listed in Table 1.

Table 3


The follow-up survey was sent within 18 ± 10 days after the calculated past due date. Excluding the 95 deceased, 17 nursing home residents, and 21 cases with undeliverable addresses, the response rate was 786 out of 867 (91%), including 640 true and 146 false positives. Reasons reported by the true positives for no longer taking the statin provided by the VHA are listed in . Of the 640 true positives, 302 reported at least 1 reason that could be diffi cult for healthcare providers to address, including physician orders, side effects, switching to alternative treatments, or using non-VHA sources of supply. Other reasons that were given might be more amenable to efforts to reinstate long-term use of a statin. However, 208 of the remaining true positives that gave reasons that might be more amenable to reinstating use received another statin supply from the VHA within 6 months after becoming past due. Thus, efforts to reinstate their statin would not be required. The summari zes the overall yield of good candidates for efforts by healthcare providers to reinstate long-term statin use.


The majority of patients identifi ed by this particular approach to screening electronic healthcare records were questionable candidates for reinstatement of long-term statin use. The yield could be improved in several ways. Having more up-to-date records of deaths and transitions into long-term institutional care would eliminate some that definitely did not require follow-up. A more advanced screening program that could determine whether VHA providers had discontinued the statin would be very helpful. Most likely this would require a search of textual notes. Searching for side effects such as elevated liver enzymes or a diagnosis of myopathy could exclude those with adverse effects that could have prompted discontinuation of a statin. However, some adverse effects, such as patients not liking how they felt on medication, will not be recorded or may be recorded as textual notes. Likewise, information on alternative treatments might not be readily accessible to an electronic screening program. Searching the healthcare record for cholesterol levels indicating adequate cholesterol control after patients became past due could further reduce the large number of positive screens needing follow-up.

A substantial proportion of the positive screens obtained statin supplies outside of the VHA. Lower out-of-pocket costs provided by insurance coverage or generic supplies provided by discount pharmacies increasingly are reducing the VHA’s ability to rely solely on internal prescription records. Efforts to reconcile medications could alleviate this problem, if reconciliation becomes routine during all patient encounters and the information is readily retrievable.15 Perhaps the reasons patients are no longer taking previously prescribed medications could be ascertained and recorded as well.

Another way to reduce the burden of follow-up of positive screens would be to increase the grace period and thereby exclude patients who eventually obtain another supply of the medication. Thirty-seven percent of the positive screens reported they were still taking the statin they got from the VHA, and 76% of them did get another statin supply within 6 months of their calculated past-due date. Although the follow-up survey may have prompted some to request another supply of statin, the percentage that eventually obtained another statin supply is similar to our preliminary data wherein positive screens did not receive a follow-up survey. Others have noted the dynamic nature of patients’ prescription supplies.16 Other analyses not presented herein suggested that most of the patients that reported they were still taking the statin despite being past due for a refill relied solely on the VHA for this prescription medication and may have accumulated suffi cient supplies to bridge a 4-month gap. Thus, the screening program may have given the wrong impression in these cases. However, accumulation of excess supply could be due to unrecognized self-made or provider-directed dosage reductions. Another study of veterans with hypertension reported nearly 50% disagreement between a 90-day measure of insuffi cient medication supply and a self-reported measure of nonadherence.17 These results along with ours indicate that more information is needed before acting on observed gaps in medication supplies. Conversely, only 22% of those that confi rmed they were no longer taking the VHA statin received another statin supply within 6 months, confi rming the veracity of this type of self-report.18

Asking positive screens whether or not they stopped taking a medication provided by the VHA and if they stopped, why, is a reasonable approach to try to reinstate use of a medication.19 The survey respondents gave many reasons for stopping their statin that typically would not be noted in medical records, and therefore would need to be elicited by some sort of follow-up with the patient. Although not responding to a survey about medication beliefs has been associated with worse adherence, the response to our follow-up survey was 91% for both true and false positive screens.20 However, the fi rst survey package included a $5 payment and as many as 3 survey packages had to be sent to get a response. This approach may not be practical when there are a large number of positive screens. The effectiveness of a simple refi ll reminder sent to all positive screens may be limited if the reminder does not address the patients’ reasons for stopping the medication, and may enrage those who receive an inappropriate reminder.

This attempt to screen electronic records produced a substantial number of false positives. We were not able to determine why the screening program did not incorporate statin prescriptions that were later found in prescription records extracted from VistA files. None of the comparisons of true versus false positive screens provided much insight into this problem. It is somewhat encouraging that the false positives were limited to prescriptions dispensed during the fi rst month of the screening program when there might have been an unrecognized problem while initiating the screening process. However, if timely transfer of prescription-filling records from the mailorder pharmacy to the local VistA fi les was responsible, false positives could be a recurrent problem.

This attempt to use the VHA electronic health records to monitor continued use of a medication prescribed for a chronic condition is limited to a single class of medications in a single VA Health Care system. However, the electronic health records are used by all VA Health Care systems, and the likelihood of, and reasons for, discontinuing statins are similar to several other long-term medications.1,5 Thus, insights from this evaluation could apply more generally and help guide further development of electronic screening programs. By design, we did not begin to screen patients until they had received at least 2 statin supplies to try to avoid any effects of trial doses, dose titration, early intolerance, and patients that didn’t intend to take the medication that was prescribed. We were most interested in screening patients that at least initially appeared to be taking and tolerating the medication. Beginning to screen patients after they fi ll their fi rst prescription may alter the results reported herein. Not all of the positive screens returned the survey or indicated whether or not they were still taking the statin, making it diffi cult to estimate the overall percentages in the population that stopped statin use for each reason. The viable yield of the screening program could be overestimated for the same reason. Nevertheless, the list of reasons and number of times they were cited provides important insights into longterm statin use and what issues need to be ascertained and addressed by this type of mass screening program. For example, healthcare providers may have to convince some patients that the effort and cost of long-term statin use is worthwhile and make sure to show them how much their cholesterol and risk has been reduced. Other patients need help managing their medications and out-of-pocket costs. Healthcare providers might not be able to resolve some of the reasons patients gave for discontinuing a statin.

In conclusion, the actionable yield of this particular effort to screen the VHA’s electronic healthcare records of patients that were filling prescriptions for a long-term medication was quite low, due to information that was not placed in the electronic records in a timely manner or that would be diffi cult to extract without a much more sophisticated program including text searches. Furthermore, supplemental information is necessary to determine the patients’ undocumented reasons for discontinuing a medication. Mass screening of electronic health records followed by a large number of mailed follow-up surveys may not be an effi cient method for identifying viable

candidates for interventions to reinstate long-term medication use.21-23 Allocation of limited healthcare resources to this type of mass screening program would have to be justified by the magnitude of improvement in patient outcomes that would depend on improving the viable yield as well as whether healthcare providers could satisfactorily address the variety of reasons patients have for discontinuing a medication.Acknowledgments

The excellent work of Andrea Cutting, MA, who compiled the survey data and extracted and processed the data from VHA administrative records, and Hannah Fairman, BS, who managed the follow-up survey, is gratefully acknowledged.

Author Affiliations: From VA Health Care System (TSR, SN, MS, SN, HEB), Minneapolis, MN; University of Minnesota School of Medicine (TSR, MS, SN, HEB), Minneapolis, MN.

Author Disclosures: Drs Rector, Spoont, Noorbaloochi, and Bloomfield and Mr Nugent report employment with the VA Health Care System and report receiving grants from the Veterans Health Administration.

Authorship Information: Concept and design (TSR, SN, MS, SN, HEB); acquisition of data (TSR, SN); analysis and interpretation of data (TSR, HEB); drafting of the manuscript (TSR); critical revision of the manuscript for important intellectual content (TSR, SN, MS, SN, HEB); statistical analysis (TSR, SN); obtaining funding (TSR, MS); administrative, technical, or logistic support (TSR); and supervision (TSR).

Funding Source: This evaluation was funded by VA Health Services Research and Development grant IIR 08 -356. The authors are solely responsible for the design, analyses, interpretations, and writing of this report. Their views are not necessarily the views of the Veterans Health Administration.

Address correspondence to: Thomas S. Rector, PharmD, PhD, VA Medical Center, 152/2E, One Veterans Dr, Minneapolis, MN 55417. E-mail: DiMatteo MR. Variations in patients’ adherence to medical recommendations: a quantitative review of 50 years of research. Med Care. 2004;42(3):200-209.

2. Cramer JA, Benedict A, Muszbek N, Keskinaslan A, Khan ZM. The significance of compliance and persistence in the treatment of diabetes, hypertension, and dyslipidaemia: a review. Int J Clin Pract. 2008; 62(1):76-87.

3. Furman A, Meier JL, Malmstrom RA, Lopez JR, Schaefer S. Comparative efficacy of ezetimibe/simvastatin, rosuvastatin and atorvastatin in uncontrolled hyperlipedemia patients. Am J Managed Care. 2011;17(8):538-544.

4. Ho PM, Bryson CL, Rumsfeld JS. Medication adherence: its importance in cardiovascular outcomes. Circulation. 2009;119(23):3028-3035.

5. McHorney CA, Spain CV, Alexander CM, Simmons J. Validity of the adherence estimator in the prediction of 9-month persistence with medications prescribed for chronic disease: a prospective analysis of data from pharmacy claims. Clin Ther. 2009;31(11):2584-2607.

6. Chan DC, Shrank WH, Cutler D, et al. Patient, physician and payment predictors of statin adherence. Med Care. 2010;48(3):196-202.

7. Siska MH, Tribble DA. Opportunities and challenges related to technology in supporting optimal pharmacy practice models in hospitals and health systems. Am J Health-System Pharmacy. 2011;68(12):1116-1126.

8. Heisler M, Hogan MM, Hofer TP, Schmittdiel JA, Pladevall M, Kerr EA. When more is not better: treatment intensifi cation among hypertensive patients with poor medication adherence. Circulation. 2008;117(22): 2884-2892.

9. Doshi JA, Zhu J, Lee BY, Kimmel SE, Volpp KG. Impact of a prescription copayment increase on lipid-lowering medication adherence in veterans. Circulation. 2009;119(3):390-397.

10. Brookhart MA, Patrick AR, Schneeweiss S, et al. Physician followup and provider continuity are associated with long-term medication adherence: a study of dynamics of statin use. Arch Intern Med. 2007: 167(8):847-852.

11. Caspard H, Chan AK, Walker AM. Compliance with a statin treatment in a usual-care setting: restrospective database analysis over 3 years after treatment initiation in health maintenance organization enrollees with dyslipidemia. Clin Ther. 2005:27(10):1639-1646.

12. Avorn J, Monette J, Lacour A, et al. Persistence of use of lipid-lowering medications, a cross-national study. JAMA. 1989;279(18):1458-1462.

13. Andrade SE, Kahler KH, Frech F, Chan KA. Methods for evaluation of medication adherence and persistence using automated databases. Pharmacoepidemiol Drug Safety. 2006;15(8):565-574.

14. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defi ning comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139.

15. Spina J, Glassman PA, Simon B, et al. Potential safety gaps in order entry and automated drug alerts, a nationwide survey of VA physician self-reported practices with computerized order entry [published online ahead of print June 9, 2011]. Medical Care. 2011;49(10):904-910. PMID 21666510.

16. Brookhart AM, Patrick AR, Schneeweiss S, et al. Physician followup and provider continuity are associated with long-term medication adherence: a study of the dynamics of statin use. Arch Intern Med. 2007;167(8):847-852.

17. Thorpe CT, Bryson CL, Maciejweski ML, Bosworth HB. Medication acquisition and self-reported adherence in veterans with hypertension. Med Care. 2009;47(4):474-481.

18. Eraker SA, Kirscht JP, Becker MH. Understanding and improving patient compliance. Ann Intern Med. 1984;100(2):258-268.

19. Cushing A, Metcalfe R. Optimizing medicines management: from compliance to concordance. Ther Clin Risk Manag. 2007;3(6):1047-1058.

20. Gadkari AS, Pedan A, Gowda N, McHorney CA. Survey nonresponders to a medication-beliefs survey have worse adherence and persistence to chronic medications compared with survey responders [published online ahead of print June 9, 2011]. Med Care. 2011;49(10): 956-961. PMID 21666510.

21. Petterson AM, Takiya L, Finley R. Meta-analysis of trials of interventions to improve medication adherence. Am J Health Syst Pharm. 2003; 60(7):657-665.

22. Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev. 2008(2): CD000011. doi:10.1002/14651858.CD000011.pub3. Accessed August 29, 2011.

23. Schedbauer A, Davies P, Fahay T. Interventions for enhancing medication adherence to lipid lowering medication. Cochrane Database Syst Rev. 2010(3):CD004371. doi:10.1002/14651858.CD004371.pub3. Accessed August 29, 2011.