We are most grateful to all the people who have reviewed manuscripts for us in 2008.  Thanks to their efforts, the 

 is able to publish scientifically sound and relevant research for our readers.

Articles

WEB EXCLUSIVE

We are most grateful to all the people who have reviewed manuscripts for us in 2008. Thanks to their efforts, the Journal is able to publish scientifically sound and relevant research for our readers.

List of Peer Reviewers (2008)

: To determine how Medicare benefits affect veterans’ use of Veterans Health Administration (VHA) pharmacy services.

: Retrospective analysis of veterans dually enrolled in the Veterans Health Administration and Medicare healthcare systems.

: We used VHA and Medicare administrative data for calendar year 2002 to examine the effect of Medicare HMO pharmacy benefit levels on VHA pharmacy use.

: In 2002, 64% of the VHA and Medicare dually enrolled veterans in our study sample received medications from the VHA. Use of VHA pharmacy services varied monotonically by the level of pharmacy benefits among Medicare HMO enrollees, with veterans enrolled in plans with both low and high pharmacy benefit levels significantly less likely to use VHA pharmacy services than veterans in plans with no pharmacy benefits (odds ratios = .83 and .53, respectively, versus plans with no benefits). Among VHA pharmacy users, enrollment in plans with high levels of benefits was associated with significantly lower annual pharmacy costs than enrollment in plans with no benefits or enrollment in traditional Medicare.

: Our findings indicate that non-VHA pharmacy benefits affect both the likelihood and magnitude of VHA pharmacy use. This suggests that Medicare pharmacy coverage (Part D) may significantly reduce the demand for VHA pharmacy services, particularly in geographic regions previously underserved by Medicare managed care plans.

	To determine how Medicare benefits affect veterans’ use of Veterans Health Administration (VHA) pharmacy services, we did a retrospective analysis of veterans dually enrolled in the VHA and Medicare healthcare systems.

		In 2002, 64% of VHA and Medicare-enrolled veterans (5% of all Medicare enrollees) received some or all of their medications from the VHA.

		Medicare managed care plan pharmacy benefits appeared to reduce demand for VHA pharmacy services, with the effect on demand greatest in plans with the highest level of pharmacy benefits.

		Medicare pharmacy coverage through Part D may significantly reduce the demand for VHA pharmacy services, particularly in geographic regions previously underserved by Medicare managed care plans.

	The Veterans Health Administration (VHA) is the largest single purchaser of pharmaceuticals in the United States. Virtually all honorably discharged veterans are legally eligible to receive care under the VHA system. Veterans Health Administration eligibility rules determine the size of copayments and the specific services to be provided to eligible veterans.

	Copayments for prescriptions for veterans have historically been very low. Until 2002, the prescription copayment for veterans was $2 for each 30-day supply of medication; this copayment increased to $7 in February 2002 and to $8 in 2006.

 These low copayments have made the VHA an attractive source of pharmacy care for veterans.

 Research suggests that approximately 10% of VHA users come to the VHA primarily for pharmacy services,

 although the percentage of veterans who rely on the VHA as their primary pharmacy provider may be much higher.

	As an alternative source of care and pharmaceutical services, more than 50% of VHA users also are enrolled in Medicare, including more than 90% of those over the age of 65 years. In January 2006, Medicare implemented Medicare Part D as part of the Medicare Modernization Act (MMA). The MMA dramatically changed the health services environment in which the VHA has long operated. For the first time, Medicare is explicitly offering outpatient pharmacy coverage through Medicare prescription drug plans (PDPs), independent of the pharmacy benefits offered through Medicare managed care organizations, and has made PDPs (alone or in combination with managed care plans [MAPDPs]

	A recent survey of Medicare Part D enrollees indicated that, relative to Medicare beneficiaries who relied primarily on stand-alone PDPs or MA-PDPs,  beneficiaries who relied on the VHA for pharmacy care reported lower out-of-pocket spending and were less likely to have delayed filling prescriptions due to cost.

 However, there is still little information to guide VHA policy makers on how Medicare pharmacy benefits might affect pharmacy expenditures within the VHA healthcare system, or on how Medicare plan benefits affect Medicare-enrolled veterans’ use of VHA pharmacy services. In the current study, we examined how Medicare HMO pharmacy benefit levels among veterans dually enrolled in the VHA and Medicare healthcare systems affected the likelihood and magnitude of VHA pharmacy use. Specifically, we addressed 2 questions:

	1. Do greater Medicare health plan pharmacy benefits reduce the likelihood that veterans will use VHA pharmacy services?

	2. Among veterans who do use VHA pharmacy services, do Medicare health plan pharmacy benefits reduce the magnitude of their use?

	We used national VHA and Medicare administrative files from calendar year (CY) 2002 to examine how Medicare HMO enrollment status and pharmacy benefit coverage affected VHA pharmacy use. Our analyses focused on veterans who were enrolled in the VHA and in Medicare for at least 1 month during CY 2002. Each beneficiary’s Medicare enrollment and demographic information were obtained from the Medicare denominator file records and merged with the VHA Enrollment file to identify veterans who were dually enrolled in the VHA and Medicare healthcare systems.

	Health maintenance organization plans vary substantially in the level of pharmacy benefits offered. Thus, for each managed care—enrolled veteran, we classified benefit levels associated with his or her HMO contract number. Contract numbers were obtained from the Centers for Medicare&Medicaid Services’ Medicare Enrollment Database. Because multiple HMO plans or benefit packages are allowed within a contract’s service area, we used the strategy described in the Commonwealth Fund report 

Medicare Choice 1999-2001: An Analysis of Managed Care Plan Withdrawals and Trends in Benefits and Premiums

 to assign a “basic” plan for each managed care contract active in a particular county. The benefits associated with the basic plan were assigned to each beneficiary enrolled in that county.

	Also following the Commonwealth scoring algorithm,

 Medicare HMO—enrolled veterans were assigned to subgroups based on whether their HMO offered no additional pharmacy benefits, a “low” level of benefits, or a “high” level of benefits. Low-benefit plans were those that either offered only generic medications or had a cap of $750 or less on either generic or brand name medications. High-benefit plans were those with benefit caps of greater than $750 for both generic and brand name medications.

	Veterans Health Administration pharmacy use data were obtained from the VHA Decision Support System, a national automated management information system that integrates data from clinical and financial systems for inpatient and outpatient care.

 The system creates an outpatient encounter for each day a patient fills a prescription, including all pharmacy costs for that day. For our analyses, encounters (costs) were aggregated within each person across the entire calendar year.

Access to Veterans Health Administration Medical Centers

	The availability of Veterans Health Administration Medical Centers (VAMCs) was determined by recording the distance from the beneficiary’s zip code to the nearest VAMC or Community-Based Outpatient Clinic (CBOC). We also recorded VHA eligibility (priority code) from the VHA Enrollment file. We assigned a “home station” to all veterans to control for clustering and facility characteristics in our multivariate regressions. For VHA-Medicare dual enrollees who had any VHA pharmacy costs during the CY, the home station was where the majority of pharmacy services (according to cost) were obtained. Dual enrollees who did not use VHA pharmacy services were assigned to a home station based on proximity (ie, the closest VAMC). If their closest VHA facility was a CBOC, the VAMC associated with that CBOC was assigned as their home station.

	In our multivariable models, we adjusted for patients’ health status using the Centers for Medicare&Medicaid Services Hierarchical Condition Category (HCC) risk-adjustment model for community-dwelling populations.11 The model adjusts for diagnoses, age, sex, Medicaid eligibility, and original entitlement of Medicare (disability, end-stage renal disease, or Old Age and Survivor’s Insurance). The risk score was created for each patient by using 

International  classification of Diseases, Ninth Revision

 codes from the VHA administrative data files for CY 2002.

	We assigned income level according to zip code, age (<65 years, >65 years), and race/ethnicity using the 5-digit zip Code Tabulation Area income data available in 2000 US Census Summary File 3. Veterans were classified as residing in metropolitan or nonmetropolitan counties using urban influence codes

 to group all 3141 counties in the United States according to the most recently revised federal definitions of metropolitan statistical areas (MSAs).

	The Medicare administrative files did not include an indicator for concurrent participation in the Medicaid program. However, we did classify veterans by whether they received state buy-in through Medicaid to offset their Medicare Part B premiums (Medicare state buy-in). This group included veterans who had their Part B premiums only paid or their Part B premiums and cost-sharing paid (eg, Specified Low-Income Medicare Beneficiaries [SLMBs] and Qualified Medicare Beneficiaries [QMBs], respectively), as well as veterans who received full Medicaid benefits in addition to being an SLMB or QMB enrollee.

	We grouped VHA enrollees into 3 sets of priority levels that broadly differentiated copayment levels and out-of-pocket maximums.

 These groupings are priority level 1, priority levels 2-6, and priority levels 7 and greater. Priority level 1 veterans generally had no copayments, while veterans with

	priority levels of 7 or greater (7 ) paid the maximum VHA copayment of $7 ($2 prior to February 2002) for each of their VHA prescriptions.

	We conducted all analyses using SAS version 9.1.2 (SAS Institute Inc, Cary, NC). We used a generalized-estimating-equations approach in our multivariable models of pharmacy use and costs to account for clustering of veterans within VAMCs. For our analysis of VHA pharmacy use, we performed a logistic regression using VHA pharmacy use versus no use as a binary outcome. For our analysis of total annual pharmacy cost, we used total cost as a continuous outcome measure. Because the distribution of pharmacy costs was asymmetric and highly skewed, we estimated a regression model assuming a gamma distribution for our outcome measure. This type of distribution has been shown to provide robust estimations in the presence of skewed data.16 Examination of the regression residuals indicated that our use of a gamma distribution was appropriate.

	In CY 2002 there were 3.4 million dual VHA-Medicare enrollees. About half of all dual enrollees (52%) lived in areas with available Medicare managed care plans, with approximately 11% overall (22% of those living in HMO counties) actually enrolled in an HMO plan for all or part of CY 2002. Of the 388,633 veterans enrolled in HMO plans, 34% were in contracts whose basic plans did not offer pharmacy benefits, 51% had basic plans with low benefits, and 14% had basic plans with high benefits (

	Approximately 64% of the 3.4 million Medicare-enrolled veterans (which accounted for 5% of all Medicare enrollees) received some or all of their medications from the VHA (

). Nationally, the VHA pharmacy expenditures totaled $2.3 billion (or 68% of all VHA pharmacy costs) for providing pharmacy services to VHAMedicare dual users. Use of VHA pharmacy services varied monotonically by the level of pharmacy benefits among Medicare managed care enrollees (Table 2).

	In our multivariable models, veterans enrolled in Medicare HMO plans with both low and high pharmacy benefit levels were significantly less likely to use VHA pharmacy services than veterans in HMO plans with no pharmacy benefits (

	Age, Medicare state buy-in, VHA priority level, and HCC risk score were the strongest predictors of VHA pharmacy use. VHA-Medicare enrolled veterans under the age of 65 years were the most likely to use VHA pharmacy services. When veterans younger than age 65 years are eligible for Medicare, it is primarily because they are classified as disabled, and are likely to have substantial healthcare needs. Most of our other covariates also were significant (Table 3). Women were less likely than men to use VHA pharmacy services, and the likelihood of VHA pharmacy use was lower among veterans residing in MSA counties, veterans with incomes of more than $20,000 per year, and veterans whose home VAMCs were teaching hospitals. Hispanic veterans were more likely to use VHA pharmacy services than either African American or white veterans. Surprisingly, distance to the closest VHA facility was not predictive of VHA pharmacy use in our multivariable model, possibly because of the heavy use of VHA mail-order pharmacy services, and because MSA status and HMO versus non-HMO county residence may have served as proxies for distance.

Predictors of Total VHA Pharmacy Costs Among VHA Pharmacy Users

	Medicare managed care enrollees accounted for 8.9% of all VHA pharmacy costs attributable to Medicare-enrolled veterans. The average unadjusted cost to the VHA pharmacy service in CY 2002 for veterans enrolled in fee-for-service (FFS) plans and using VHA pharmacy services was substantially greater than the average cost for HMO enrollees (Table 2). Unadjusted costs for veterans living in areas not served by HMO plans were similar to those for FFS veterans.

), veterans enrolled in plans with high pharmacy benefit levels had total annual costs that were approximately 5% lower than those for veterans in HMO plans with no pharmacy benefits ($50 less), veterans enrolled in Medicare FFS ($63 less), or veterans living in counties not served by HMO plans ($67 less). Veterans with high HMO benefit levels also had lower costs than those in plans with low benefit levels ($35 less), but a post hoc comparison indicated that this difference was not statistically significant.

	Our work shows that Medicare managed care plan pharmacy benefits may affect demand for VHA pharmacy services, with the effect on demand concentrated among enrollees in plans with the highest level of pharmacy benefits. In CY 2002, VHA-Medicare dually enrolled veterans enrolled in HMO plans with high levels of pharmacy benefits were less likely to use any VHA pharmacy services and had lower total pharmacy costs when they did use VHA pharmacy services. These associations were robust to adjustment for enabling and predisposing factors previously shown to affect VHA service use. In this study, we had information about levels of non-VHA pharmacy coverage among Medicare HMO enrollees only. If similar information had been available for Medicare FFS—enrolled veterans, we believe that we would have seen a similar pattern among veterans with differing levels of coverage from other forms of supplemental insurance.

	Implementation of Medicare Part D has greatly increased the availability of non-VHA pharmacy services for Medicare beneficiaries. Geographic areas that have historically not had access to HMO plans, accounting for almost half of the VHA-Medicare dually enrolled population, now have access to stand-alone PDPs, as well as other Medicare managed care plans that include PDPs. At the end of the first official enrollment period in May 2006, enrollment in PDPs was  widespread, with voluntary enrollment rates exceeding 15% in more than 90% of all US counties. Average enrollment rate across all US counties was about 30%.

	When the Government Accountability Office estimated the cost of the new Medicare prescription drug benefit, they assumed that prior VHA pharmacy users would continue to use the VHA for their medications and that they would not switch to PDPs to any meaningful extent.17 However, veterans’ responses to Medicare Part D are unlikely to be that straightforward. Historically, VHA-using veterans have enrolled in available HMO plans at about the same rate as the general population. Their decisions to enroll (or not) in a Part D PDP are likely to be driven by their access to VHA care (eg, their priority category), the availability and benefit levels of the PDP plans, their income and demographic characteristics, and their history of VHA use.

	Our analyses have several notable limitations. We did not have access to non-VHA pharmacy records. Now that Medicare Part D has become active, a centralized database of Medicare enrollees’ pharmacy utilization is expected to become available to researchers. We also did not have access to Medicare utilization records for Medicare HMO enrollees. As a consequence, our HCC severity score was calculated based solely on VHA utilization data and likely underestimates, and may introduce bias into, the true HCC severity scores for the veterans examined in this study. For example, HMO enrollees overall had an average of 2 fewer VHA diagnoses during the study period than FFS enrollees overall did (5.4 ± 7.6 diagnoses vs 7.5 ± 9.5 diagnoses, respectively) and were less likely to be in the highest HCC severity score category (16.2% vs 20.4%). Whether this reflects a true difference in health status or preferential use of non-VHA healthcare by HMO enrollees is unclear.

	We used each veteran’s VHA priority level as a covariate to partially adjust for their level of access to VHA pharmacy services. VHA pharmacy access is a complex issue and is affected by a variety of factors, including overall extent of service-connected disability, income level, whether pharmacy care is specifically for a service-connected condition, prisoner-of-war status, and whether pharmacy care is for other specifically authorized conditions. These factors affect both the level of copayments and the out-of-pocket maximum veterans incur. Our 3 specific priority level groupings were chosen to broadly capture variability in VHA pharmacy access (see also Stroupe et al15), and successfully differentiated among all 3 groupings in terms of overall likelihood of use and total pharmacy cost among VHA pharmacy users (Table 3).

	For HMO enrollees, we assigned the pharmacy benefit level associated with the basic plan for the veterans’ managed care contract. In cases where contracts offered multiple plans, this assignment may have underestimated the pharmacy benefits available to the enrolled veterans. This underestimation would likely reduce the differences among our HMO enrolled groups (ie, with high, low, and no pharmacy benefits), suggesting that our findings may be conservative.

	Finally, our study is cross-sectional. Veterans who enroll in different types of Medicare plans may be fundamentally different in their illness burden or preferences for receiving care. We controlled for many individual characteristics in these analyses that previously were related both to veterans’ utilization of VHA care and their selection of healthcare plans. Consequently, we feel that the potential for bias in our analyses was ameliorated.

	Despite these limitations, our work has immediate and significant relevance in the context of the recent major programmatic changes in Medicare. According to the 2006 Medicare Payment Advisory Commission report, 

Report to the Congress: Increasing the Value of Medicare

 the majority of new PDP plans offered under Medicare Part D would have qualified as high-benefit plans using the definition in our analyses. Consequently, increased enrollment in PDPs by VHA-Medicare dual enrollees may have profound and immediate implications for the VHA pharmacy service. First, the availability of PDPs may significantly reduce the demand for VHA pharmacy services among current VHA users, as well as lessen the likelihood that current nonusers will turn to the VHA system as their provider of pharmacy care. On the other hand, a substantial percentage of veterans who do enroll in PDPs are likely to exceed the cap on the initial standard pharmacy benefit ($2250 for CY 2006). This is particularly likely to be true for veterans with complex medical conditions. These veterans will then become responsible for 100% of the next $2400 in their medication costs and will have a strong incentive to turn to the VHA for these pharmacy services. Alternatively, these veterans may be the VHA users most likely to split their pharmacy care, obtaining some medications from the VHA and the rest from their Medicare PDP, in order not to exceed their cap. Either of these cases affects the pattern and magnitude of VHA pharmacy use and may increase the potential for disjointed continuity of care. Further work is needed to examine the impact of the new Medicare coverage on actual medication use by veterans.

: From the Division of Management, Policy, and Community Health (ROM), University of Texas School of Public Health, Houston; Baylor College of Medicine (LAP, JAD), Houston; the Michael E. DeBakey VHA Medical Center (LAP, JCH, JAD), Houston, TX; the School of Medicine (MMB), University of Miami, Miami, FL; the College of Pharmacy and Health Sciences (NIO), Texas Southern University, Houston; the Centers for Medicare & Medicaid Services (IIW), Baltimore, MD; and the College of Pharmacy (MLJ), University of Houston, Houston, TX.

: This research was supported by the Department of Veterans Affairs, Health Services Research and Development Service grants IIR 02-081, IIR 02-083, and HFP 90-020, and by National Institutes of Health grant R01 AG19284-01. Drs Davila and Johnson are career development awardees of the Department of Veterans Affairs Health Services Research and Development Service. Dr Byrne is a career development awardee of the National Cancer Institute. Dr Petersen is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar (grant 045444) and an American Heart Association Established Investigator Awardee (grant 0540043N).

Greater Medicare managed care benefit levels reduce both the likelihood and magnitude of Veterans Health Administration pharmacy use by Medicare dually enrolled veterans.

VHA Pharmacy Use in Veterans With Medicare Drug Coverage

 To confirm the cost savings in a randomized clinical trial of the Vermedx Diabetes Information System (hereafter referred to as the Diabetes Information System [DIS]) in independently collected data using claims paid by a managed care insurer for patients with and without DIS participation.

Longitudinal analysis of paid claims with concurrent and historical controls from October 2002 through October 2007.   Methods: Using locally weighted smoothing functions and linear regression analysis before and after commencement of the DIS, we compared the total claims paid per member per month for 153 patients using the DIS versus 870 control patients.

For DIS patients, paid claims increased at a rate of $8.30 (95% confidence interval [CI], $1.12-$15.48) per month before the DIS started compared with −$3.92 (95% CI, −$9.50 to $1.67) after commencement of the DIS (P = .008). For control patients, the slope changed from $6.80 (95% CI, $3.78-$9.82) to $3.16 (95% CI, −$1.06 to $7.38) (P = .17). After commencement of the DIS, the slope of the claims in the DIS group is significantly lower than that of the control group (−$3.92 vs $3.16, P = .046). The mean estimated savings range from $504 per patient in year 1 of operations to $3563 in year 4. The cumulative net savings reach $8134 in 4 years.

Participation in the DIS is associated with substantial reductions in claims paid, net of the costs of the intervention. The cost savings reported in the randomized clinical trial of the DIS are reproduced in an independent data set.

Diabetes mellitus continues to be a major source of morbidity, mortality, and healthcare utilization.

The Vermedx Diabetes Information System is a low-cost provider-friendly decision support system for the primary care of adults with diabetes.

The system results in improved care and reduction in the total costs of care.

Net savings range from $504 per patient in year 1 of operations to $3563 in year 4.

Although effective therapies are available, diabetes care remains far from optimal,

 and the morbidity, mortality, and economic burdens of the disease continue to grow. The estimated cost of diabetes mellitus in the United States in 2007 was $174 billion, including $116 billion in healthcare utilization and $58 billion in reduced productivity.

The Vermedx Diabetes Information System (hereafter referred to as the Diabetes Information System [DIS]),  originally the Vermont Diabetes Information System, is a laboratory-based registry and decision support system based on the chronic care model.

 It is designed for low-cost and easy integration into primary care. It requires no additional staff, office space, or capital investment by participating practices. The practices need not use office computers or electronic medical records, although the DIS can easily integrate with them. The DIS costs less than $50 per patient per year in most settings. A recent randomized clinical trial in 64 practices with 132 primary care providers and 7412 adults with diabetes showed significant improvements in clinical care (adherence to laboratory testing guidelines) (B.L. and C.D.M., unpublished data, 2008). Although no change in physiologic control was observed, the intervention group reported substantially lower hospital, emergency department, and physician utilization. Total savings were estimated at $2426 per patient per year (P = .03). One of the limitations of this analysis was that utilization was estimated from patient self-report.

Our objective was to confirm these findings using actual claims. We assessed the effects of the DIS on resource utilization using independently collected data from another source.

 The system receives laboratory results (glycosylated hemoglobin and cholesterol levels and kidney function) from clinical laboratories, maintains a registry, and produces reports for primary care providers and their patients. Reports are automatically generated whenever a laboratory test is completed and include flow sheets with guideline-based recommendations for the providers and alert letters for the patient (when results the patient (when results are above target). When test results are overdue, a reminder is sent to the patient and the practice. Population reports listing all the provider’s patients are sent to the provider quarterly, along with a report card indicating population-level performance. Reports are mailed to patients and are sent electronically or by fax to the practices. The system is not linked to any pay-for-performance incentives.

Vermont Managed Care, Inc (VMC ) is a physician hospital organization serving 31,673 covered lives in Vermont. Practices began participation in the DIS in May 2003 through February 2008. All adults with diabetes covered by VMC were included, regardless of which practice they attended. Adults with diabetes covered by VMC and receiving primary care from participating practices were enrolled in the DIS when their practice first participated. All other adults with diabetes covered by VMC were included as control subjects.

Claims paid for all covered services from October 2002 through October 2007 were aggregated per patient per month. Patient-months with no claims were set to zero if the patient was covered that month. For DIS subjects, each patientmonth was assigned a number relative to the time the patient was first entered into the DIS. The month of DIS enrollment was assigned as zero. Negative numbers represent the number of months before starting the DIS, and positive numbers represent the number of months in the DIS. For control patients, a random month was chosen during the time of coverage to represent the “zero month.”

The DIS and control patients were compared using Wilcoxon signed rank sum test for continuous variables or 

 test for categorical variables. The relationship between claims paid and time was estimated using nonparametric locally weighted smoothing functions

 and 2-piece ordinary least squares linear regression analysis with adjustment for clustering within members.

 <.05 represented statistical significance.

This study was performed in accordance with ethical research standards. The research protocol was approved by the University of Vermont Committees for the Protection of Human Subjects.

The utilization records of 153 patients participating in the DIS and of 870 control patients were analyzed. All claims paid for the period of analysis were available, with no missing data. The DIS subjects received 193 alert letters (0.8 per patient per year) and 764 reminder letters (3.3 per patient per year). Their providers received 818 flow sheets (3.6 per patient per year) and 725 reminders (3.1 per patient per year). Overall, the system generated 2500 reports (10.7 per patient per year). The control patients and providers received none. Fifty-five percent of the subjects were male. The mean age at enrollment was 52.4 years, with 7.8% being 65 years or older at enrollment. The mean period of enrollment was 44.5 months. The DIS patients had longer periods of observation and fewer paid claims per month (

). The number of subjects contributing data varied from 11 control and 21 DIS subjects at 48 months before the start of the DIS, to 860 control and 131 DIS subjects at the start of the DIS, to 89 control and 18 DIS subjects at 48 months after the start of the DIS.

The nonparametric curve of claims paid per member per month as a function of time relative to starting the DIS has a distinct inverted-U shape (

), with the peak at 9 months after the start of the DIS. For control patients, the curve continues to rise without decline.

Fitting a straight line to the data for the period before the start of the DIS (

) shows that paid claims increased at a rate of $8.30 (95% confidence interval [CI], $1.12-$15.48) per month. A second line that was fit to the data after the DIS started decreased, with a slope of −$3.92 (95% CI, −$9.50 to $1.67) per month. The difference between the 2 slopes is highly statistically significant (P = .008). For control patients, the slope changed from $6.80 (95% CI, $3.78-$9.82) to $3.16 (95% CI, −$1.06 to $7.38) per month (P = .17). The slopes of the 2 lines for the earlier period are not significantly different ($8.30 vs $6.80, P = .70).

A later phase was assessed (after the start of the DIS or the random control date). At that time, the slope of the claims in the DIS group is significantly lower than that of the control group (−$3.92 vs $3.16, P = .046).

We estimated the savings due to the DIS by subtracting the claims paid during the DIS from the claims expected based on extrapolation of the control experience (

). The savings grow larger each month as the 2 data lines diverge. In the first month of the program, the difference in claims paid is negligible. By the end of the first quarter of operations, DIS patients had $30.48 less in claims than other patients with diabetes. In the first year, the mean estimated savings is $504 per patient. Savings in the following years are $1524 in year 2, $2543 in year 3, and $3563 in year 4. For the 12 months ending 32 months after the program commencement (the period analyzed in the randomized clinical trial), the net reduction in claims paid is $2203. The cumulative net savings reach $8134 in 4 years.

Repeat analyses included patient age, insurance plan, and length of observation as covariates and excluded high-cost patient-months and patients with high total costs. These analyses gave very similar results.

These data from an independent data source support the findings of the randomized clinical trial of the DIS in which the intervention was associated with savings of $2426 per patient per year after 32 months. The comparable savings from the present analysis total $2203. Start-up costs of $5000 to $15,000 per laboratory are usual. The ongoing operation costs of the DIS total less than $50 per patient with diabetes per year. Although the randomized clinical trial data were unlikely to be confounded by differences in the randomized groups or by secular trends in costs, the data were limited by patient recall as the source of utilization estimates. Although potentially confounded, the claims analysis uses objective claims data that are not subject to recall bias. The similarity of the results of 2 analyses supports the conclusion that the DIS is responsible for the observed reductions in costs.

The mechanism whereby the DIS reduces utilization is unknown. In the randomized clinical trial, glycemic control, cholesterol level, blood pressure, and self-care behaviors were largely unchanged. Therefore, the savings seen in that study, and again herein, are unlikely to be related to the prevention of vascular complications. It is possible that letters from the DIS reduce patients’ concerns about being ignored or lost to follow-up. With confidence that the provider is paying attention to their needs, patients may be more willing to accommodate or tolerate a minor symptom versus obtaining unscheduled care from the provider, emergency department, or hospital. Further research regarding patient perspectives on decision support and reminder systems, as well as the DIS in particular, is warranted.

Because the DIS does not directly change the flow of patients through the practice or require data entry, new staffing, or capital equipment, it is well accepted by primary care practices. Likewise, patients are generally pleased with the service. Less than 3% of patients opt not to participate in the DIS.

What would be the economic effect of wide deployment of the DIS? The number of US adults diagnosed as having diabetes in 2007 was 17.4 million.

Assuming that only 75% receive primary care and 5% of those patients will refuse the service, and conservatively estimating savings at $2000 per patient per year after deducting the costs of the service, healthcare utilization could be reduced by $24 billion per year in America.

This analysis is subject to certain limitations. The sample is small and represents 1 insurer in 1 region. Paid claims per month are variable, with positive outliers. However, eliminating extreme outliers had little effect on the results. As in any nonrandomized study, the observed associations may be  confounded by unmeasured differences between the 2 groups. The results seen herein do not apply to other disease management, cost reduction interventions, or conditions.

In conclusion, participation in the DIS is associated with substantial reductions in claims paid, net of the costs of the intervention. Savings range from $504 per patient in the first year of participation to $3563 in the fourth year. Therefore, the cost savings reported in the randomized clinical trial of the DIS are reproduced in an independent data set.

: From the Department of Medicine (BL, CDM), University of Vermont, Burlington, and Vermont Managed Care, Inc (KZ, BHD, JAD, CRF), Colchester.

: This work was funded in part by the National Institute of Diabetes and Digestive and Kidney Diseases (R01 DK61167 and K24 of this work.

: Drs Littenberg and MacLean hold equity in Vermont Clinical Decision Support, LLC, which distributes the Vermedx Diabetes Information System. Mr Zygarowski, Ms Drapola, Dr Duncan, and Mr Frank are

employees of Vermont Managed Care, Inc, which purchases Vermedx Diabetes Information System services for its subscribers.

Concept and design (BL, CDM, KZ, BHD, CRF); acquisition of data (BL, CDM, KZ, CRF); analysis and interpretation of data (BL, CDM, BHD, JAD, CRF); drafting of the manuscript (BL, JAD); critical revision of the manuscript for important intellectual content (BL, CDM, KZ, BHD, JAD, CRF); statistical analysis (BL, CRF); provision of study materials or patients (KZ, CRF); obtaining funding (BL); and administrative, technical, or logistic support (KZ, BHD).

: Benjamin Littenberg, MD, Department of Medicine, University of Vermont, 371 Pearl St, Burlington, VT 05401. E-mail: benjamin.littenberg@vermedx.com.

1. Saaddine JB, Cadwell B, Gregg EW, et al.

 Improvements in diabetes processes of care and intermediate outcomes: United States, 1988-2002. 

 Economic costs of diabetes in the U.S. in 2007 [published correction appears in Diabetes Care. 2008;31(6):1271]. 

3. Wagner EH, Glasgow RE, Davis C, et al.

 Quality improvement in chronic illness care: a collaborative approach. 

4. MacLean CD, Littenberg B, Gagnon MS, Reardon M, Turner PD, Jordan C.

 The Vermont Diabetes Information System (VDIS): study design and subject recruitment for a cluster randomized trial of a decision support system in a regional sample of primary care practices. 

 Diabetes decision support: initial experience with the Vermont Diabetes Information System. 

 Regression standard errors in clustered samples. 

 Passive consent for clinical research in the age of HIPAA. 

CLINICAL

This analysis of paid claims from a physician hospital organization demonstrates that the Vermedx Diabetes Information System improves healthcare costs for adults with diabetes.

The Vermedx® Diabetes Information System Reduces Healthcare Utilization

 To evaluate a telephone-operated, interactive voice response (IVR) system designed to collect use-of-care data from patients with major depression (UAC-IVR).

 Patient self-reports from repeated IVR surveys were compared with provider records for 3789 patients with major depression at 41 clinical sites participating in the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial.

 UAC-IVR responses were examined for consistency and compared with provider records to compute reporting biases and: intraclass correlation coefficients. Predictors of inconsistent responses and reporting biases were based on mixed logistic and regression models adjusted for need and predisposing and enabling covariates, and corrected for nesting and repeated measures.

 Inconsistent responses were found for 10% of calls and 21% of patients. Underreporting biases (−20%) and moderate agreement (intraclass correlation of 68%) were found when UAC-IVR responses were compared with medical records. IVR reporting biases were less for patients after 3 calls or more (experience), for patients with severe baseline symptoms (motivation), and for patients who gave consistent IVR responses (reliability). Bias was unrelated to treatment outcomes or demographic factors.

 Clinical managers should use IVR systems to collect service histories only after patients are properly trained and responses monitored for consistency and reporting biases.

The feasibility of interactive voice response (IVR) systems to collect use-of-care data was assessed in a large clinical trial (STAR*D) involving 41 clinics and 4041 patients with major depression.

Moderate intraclass correlation and underreporting biases were found when patient responses were compared with medical records.

Reporting biases varied with baseline symptoms and IVR experience, but not treatment outcomes, demographic characteristics, or care attitudes.

Clinical managers should use IVR systems to collect service histories only after patients are properly trained and responses monitored for consistency.

Provider records often are considered the gold standard to monitor patient service histories, measure use of care, and to compute healthcare costs. However, accessing such records from diverse healthcare providers and in the wake of Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations can present many challenges, as files often are inaccessible

 and vary by format, completeness, and data accuracy.

 Thus, plan administrators, policy makers, and research investigators often turn to self-reporteddata when provider records are not available.

In this study, we evaluate the performance of an interactive voice response (IVR) system that collected healthcare utilization and costs information from a computerized script administered by phone (UACIVR) for the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study.

 STAR*D followed approximately 4000 patients who were being treated for nonpsychotic major depressive disorder (MDD) by 400 clinicians at 41 sites in both specialty and primary care settings, and in both the public and private sectors. To capture use-of-care information, study subjects were asked to dial a centralized number, listen to instructions, and answer computer-scripted questions by pressing keys on a touch-tone telephone pad.

As a data collection tool, IVR systems have gained both public

 acceptance. Compared with personal interviews, IVR responses are associated with lower collection costs, greater patient convenience, and fewer transcription errors.

 These systems also allow for remote data access, automated scoring,

 and opportunities for self-disclosure of sensitive information.

 Furthermore, IVR technology has been applied to studies on alcohol use,

 Good reliability has been reported when IVR results are compared with responses from written questionnaires and personal interviews.

 A high correspondence has been found between psychiatric diagnoses based on the Primary Care Evaluation of Mental Disorders (PRIME-MD) screening instrument using IVR technology and those obtained using the Structured Clinical Interview for DSM-IV (SCID-IV) interview.

 The specificity and sensitivity of an IVR mental health screener for identifying anxiety and depressive disorders, obsessive-compulsive disorders, eating disorders, and alcohol use disorders also have been demonstrated.

Prior studies have not focused on IVRs as a data collection tool to measure patient total use of care. In this study, we evaluated STAR*D’s new use-of-care survey, UAC-IVR, for both consistency and reliability. Consistency was determined by comparing responses to questions that asked if any care was used (yes/no) with questions that asked how much care was used (greater than zero/none). Reliability was assessed by comparing survey responses with provider records.

 are described elsewhere. Briefly, subjects signed an institutional review board—approved informed consent form and were followed through prospective and sequenced treatments for MDD. Patients who responded to treatment or achieved remission were followed for an additional year. Data were collected from both providers and patients. Patient information was solicited from written questionnaires, face-to-face and telephone interviews, and patient-initiated IVR calls administered by Healthcare Technology Systems, Inc. in Madison, Wisconsin. STAR*D research staff helped patients make calls at baseline, after 6 weeks at each treatment level, at the end of each treatment level, at monthly intervals during the 12-month follow-up, and at study exit. To make a call, patients first dialed a toll-free number using a touch-tone telephone. The caller received recorded instructions, followed by a set of questions. After each question, the recorded message prompted patients to respond by pressing an appropriate number on the telephone keypad. The computer then recorded each response and automatically determined the next set of scripted questions to ask the patient.

Scripted questions covering patient use of healthcare during 90-day intervals are presented in the 

. Questions were derived from the Utilization and Cost Methodology (UAC).

 Each time subjects accessed the IVR server, the computer checked to see whether the use-of-care script had ran within the past 90 days. This strategy minimized risks of double-counting services from overlapping observation periods between IVR calls. Periods not covered by an IVR call were treated as missing.

Respondents were first asked whether they had used care during the past 3 months (yes or no). Patients who responded “yes” were subsequently asked how much care they had used. Patients were asked about using services classified by setting (outpatient clinic visits, emergency room visits, and inpatient days stayed) and by type (depression related, other-psychiatric, and general medical problems). For evaluative purposes, a response was considered “inconsistent” if the respondent answered “yes” to using care while subsequently reporting that “zero” days or visits were actually used. Responses that were not inconsistent were considered consistent.

To assess reliability, provider data were obtained from billing claims and medical charts for patients signing a medical release. STAR*D focused on depression-related care; thus, records for services not related to depression were generally unavailable, and these analyses were limited to depressionrelated outpatient visits only. Services were classified by Current

 the level 1 Healthcare Common Procedure Coding System,

Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition

 Abstractors counted the number of outpatient visits that were depression related (DSM-IV 296, 311) but not emergency room related (CPT 99281, 99282, 99283, 99284, 99285, 99288) for each 90-day period ending on the date of the respective IVR survey.

Patient demographic, education, earnings, employment, and health insurance information were taken from face-toface and telephone interviews. Patients rated (agreed or neutral/disagreed): “If I can get the help I need from a doctor, I believe that I will be much better able: (a) to make important decisions that affect my life and those of my family? and (b) to enjoy things that interest me?” Patients also rated (helpful vs neutral/not helpful): “the current overall impact of your family and friends on your condition?” Also collected using the IVR system were family size, health insurance status, Medicaid eligibility, and mental and physical functioning based on the Medical Outcome Study 12-item short form.

For purposes of these analyses, treatment outcomes were based on the 17-item Hamilton Rating Scale for Depression (HRSD

 administered by telephone at the end of the first treatment step (citalopram), and on the written, patient selfreported, 16-item Quick Inventory of Depressive Symptomatology (QIDS-SR

 administered at baseline and at each STAR*D clinic visit. Treatment outcomes were computed as (1) remission defined by an HRSD

score of 7 or lower at exit from the first treatment step, (2) remission defined by a QIDS-SR

 score of 5 or lower, and (3) response defined by a reduction in QIDS-SR

 score from baseline of 50% or more. Patients with missing HRSD

 scores were not considered to have achieved remission.

Interactive voice response use-of-care responses were evaluated for both consistency and reliability. To assess reliability, counts of depression-related, nonemergency outpatient visits based on IVR responses were compared with provider records that spanned comparable time periods to compute bias (mean difference) and intraclass correlation from 2-way mixed models.

The associations of selected predictor variables with response inconsistency were computed from 3-level mixed logistic models. Similarly, the associations of selected predictor variables with reporting biases were computed from 3-level mixed regression models. Both sets of models were computed using Hierarchical Linear Modeling software,

 where level 1 is IVR calls, level 2 is individual patients, and level 3 is study sites. Both models corrected for facility nesting and repeated measures with random-effects terms. Estimates of the association for each selected predictor variable were adjusted, in turn, for the mean-centered values of a given set of covariates. Based on traditional theory,

 these covariates included need (baseline QIDS-SR

, age), predisposing (graduated high school, Hispanic, African American, sex), and enabling (married, enrolled in private health insurance plan, and employed) variables. An additional covariate was added: the order of the call. To account for complex error distributions, significance tests were based on robust estimates of standard errors.

STAR*D enrolled 4041 subjects, of whom 94% (n = 3789) completed an IVR use-of-care script, making 9864 calls, or 2.6 calls per patient (SD = 1.7, range = 1-8) (

). Among the 3789 subjects who completed the script, 17% (655 of 3789) were African American (excluding Hispanic black); 12% (464 of 3784) were Hispanic; 63% (2377 of 3788) were female; 34% (1270 of 3785) were married; 57% (2157 of 3784) were employed; 88% (3315 of 3784) had a high school diploma, General Educational Development (GED) certification, or higher; 12% (458 of 3691) had Medicaid coverage; and 51% (1911 of 3715) had private health insurance. The mean age was 41.2 years (SD = 13.2 years).

Use-of-care information obtained from IVR calls is summarized in 

. All patients reported using depressionrelated care on at least 1 call. For all calls activating the use-of-care script, 50% (4921 of 9864) reported depressionrelated care, 36% (3545 of 9864) reported general medical care, 12% (1137 of 9864) reported other psychiatric care, 2% (166 of 9864) reported an inpatient stay for depression, and 3% (311 of 9864) reported an inpatient stay for general medical purposes.

Table 2 also lists by setting the number of sites, subjects, and IVR calls that contained inconsistent use-of-care responses. There were 1069 instances of an inconsistent response, among 944 of 9864 (10%) IVR calls, from 778 of 3789 (21%) participants, at 39 of 41 (95%) study sites. By comparison, 14% (537 of 3745) of patients gave inconsistent employment and earnings responses.

Among the 944 calls with at least 1 inconsistent use-ofcare response, 832 (88%) involved only 1 response, 99 (10%) involved 2 responses, and 13 (1%) involved 3 inconsistent responses during the same call. Among the 778 participants who gave inconsistent responses, 648 (83%) made 1 call, 99 (13%) made 2 calls, 26 (3%) made 3 calls, and 5 (1%) made 4 or more calls involving at least 1 inconsistent response.

Adjusted odds ratios for predictors of inconsistent response by IVR call are listed in 

. The likelihood of making an inconsistent response tended to decline as patients gained experience with IVR calling. Whether this trend reflects patient learning or sample selection biases cannot be determined. Patients who consented to release their medical records to study investigators were slightly less likely to provide inconsistent responses, though the effect was not statistically significant. Patients providing inconsistent employment and earnings data were neither more nor less likely to provide inconsistent use-of-care responses on their IVR calls.

There were no significant associations between treatment outcomes (response or remission) and the likelihood of giving inconsistent use-of-care responses. However, patients were actually less likely to give inconsistent responses when they reported more severe baseline depressive symptoms or poorer mental functioning. On the other hand, patients with worse physical functioning were more likely to provide inconsistent responses. The different roles for mental versus physical health on response consistencies suggest that psychiatric symptoms may be motivating respondents to complete study surveys accurately, whereas poor physical functioning may be related to the capacity of patients to respond.

Patient income, employment, insurance status, and health attitudes were not related to response consistency. On the other hand, respondents who were older, female, Hispanic, African American, less educated, or married were more likely to make inconsistent choices on any given IVR call than their counterparts.

Interactive voice response reporting biases for outpatient depression-related care were determined by subtracting counts computed with IVR responses from counts based on medical records. Estimates were taken from 6858 complete and consistent IVR calls representing 2677 participants for whom medical records were available at 39 clinic sites. Overall, IVR responses underreported depression-related treatment visits by 19.2%: 2.1 visits with IVR versus 2.6 visits with the medical record (Δ = −0.50 visits ± .045; 95% confidence interval [CI], −0.59, −0.42; 

 <.001) with an intraclass correlation of 0.49, based on a 2-way mixed-effects model. Adjusting for core factors and facility nesting, IVR underreporting bias was 34.2% (adjusted Δ = −0.89 visits ± 0.12; 95% CI, −0.66, −1.13; 

 <.001). Computing total visits across all IVR calls, respondents understated visits compared with provider records by 19.2%: 5.31 visits with IVR versus 6.60 visits with the medical record (Δ = −1.29 visits ± 0.14; 95% CI, −1.57, −1.02; 

 <.001), with an intraclass correlation of 0.68.

Selected predictors of IVR reporting biases are listed in 

. A positive effect means the predictor was associated with less bias because counts of outpatient visits computed from IVR responses understated counts computed from medical records. Conversely, negative effects reflect greater disparities in visit counts between survey responses and medical records.

Interactive voice response biases tended to get worse for patients who used more care (based on medical records), made inconsistent responses on other IVR calls, had less severe baseline symptoms for depression, were uninsured, and were not on sick leave. On the other hand, there was no evidence that reporting biases were associated with treatment outcomes or patient demographic characteristics, income, employment status, or attitudes about care. Overall, these data suggest patients may be more careful when responding to survey questions when they have more severe symptoms. However, the positive association between giving inconsistent responses and reporting biases suggests these problems are related. Finally, IVR biases diminished with experience (IVR call order), but only after the third call. In fact, underreporting bias got worse between the first and second call.

Patients often seek care from diverse, off-network, and out-of-plan healthcare providers. Such information is important to clinicians preparing treatment plans, administrators monitoring healthcare costs, and scientific investigators conducting cost-outcome studies. However, access to off-plan medical records is complicated by HIPAA regulations and patients who often refuse to grant access to such records.1 For STAR*D, 94% of study enrollees (3789 of 4041) completed a use-of-care IVR call, while 77% (3116 of 4041) granted access, with only 66% (2677 of 4041) actually releasing records. In many cases, patient self-reports may be the only practical source of information decision makers have to determine use from all of the patient’s care providers.

This study evaluated a computer-scripted, IVR-formatted survey, the UAC-IVR, designed to collect self-reported use of health services. When compared with actual medical records, IVR responses tended to underreport depression-related outpatient visits by 20%, but with moderate agreement at 0.68 intraclass correlation. These findings, however, are comparable to other means of collecting self-reported health data, such as Van den Brink et al’s weekly/monthly diaries for healthcare products (

; Ritter et al’s mailed questionnaire and 6-month reporting period for outpatient (

; Raina et al’s telephone survey of seniors covering a 1-year observation interval for hospital care (intraclass correlation coefficient [ICC] = 0.50) and outpatient services (ICC = 0.25-0.41)

; and face-to-face structured interviews of veterans with mood disorders measuring outpatient encounters within 90-day intervals (ICC = 0.74).

Underreporting biases present problems for clinicians evaluating the service histories of individual patients. However, these biases are stable across demographic (sex, race, ethnicity) and patient outcome groups where plan administrators may use IVR data to make between-group comparisons. That is, differences between groups will be unbiased whenever the under-reporting biases in each group cancelled out in the difference. Our findings contrast with those of Wallihan et al, who found that African Americans were more likely to understate ambulatory care visits during telephone surveys over a 1-year period,

 but parallel those of Rozario et al, who found no racial effects.

Response inconsistencies did vary by demographic characteristics. For instance, men made fewer inconsistent responses than women. This finding is consistent with the observations of Ritter et al,

who found, with smaller sample sizes, that men had numerically but not statistically fewer discrepancies than women.

Patients who presented with more severe depressive symptoms also made fewer inconsistent responses and had fewer reporting biases. These findings are consistent with those of Raina et al,

 who found smaller reporting biases when patients claimed poorer baseline health status. Small reporting biases leads to speculation that sicker patients, presumably in search of symptom relief, may have been better motivated to provide more accurate information.

There are several limitations to the study. The reliability of self-reported use of care was based on data from 2677 participants (from among the original 3789 patients who completed at least 1 use-of-care IVR call) who signed releases and from whom medical records were obtained. Differences between these samples are described elsewhere.

 On the other hand, our estimates may hold external validity because patients were drawn from diverse clinical settings across the United States. Response inconsistencies may not necessarily reflect bad data, as patients may have reported zero volume in order to correct a prior “yes” response. Response biases were computed only for depression-related outpatient visits. Finally, this study compared responses from only 1 survey mode: IVR. Reporting differences have been observed between mail and Web-based surveys versus phone

Decision makers planning to use IVR systems to collect information on patient service histories should consider the following. First, IVR systems should be designed to alert patients whenever their responses are inconsistent and allow them to make corrections. The IVR systems also should record each occurrence, as inconsistent responses are related to reporting biases and may be symptomatic of other reporting problems. Second, response consistency improved and reporting errors eventually diminished as responders gained experience. Thus, several practice runs with feedback are highly recommended before using IVR data for decision-making purposes. Third, for accurate information, responders should be given reminders that the information being collected over the IVR call is important. Finally, IVR reporting errors must be weighted against inaccessible medical records, data collection efficiency, flexible response times for responders, and biased samples when plan enrollees are excluded from analyses because they were unwilling to share off-plan medical records.

: From the Department of Psychiatry (TMK, MHT, AW, KS-W, AJR), University of Texas Southwestern, Dallas; the Veterans Health Administration (TMK), Washington, DC; the Department of Psychiatry (MF), Massachusetts General Hospital, Boston; Healthcare Technology Systems, Inc (JHG, JCM), Madison, WI; and the Department of Epidemiology (SRW), University of Pittsburgh, Pittsburgh, PA.

Moderate underreporting biases were found when patient responses to an interactive voice response system were compared with medical records in the STAR*D clinical trial.


March 2009

List of Peer Reviewers (2008)

VHA Pharmacy Use in Veterans With Medicare Drug Coverage

The Vermedx® Diabetes Information System Reduces Healthcare Utilization

Voice Response System to Measure Healthcare Costs:A STAR*D Report

Patient Factors Associated With Following a Relocated Primary Care Provider Among Older Adults

Curtailing Laboratory Test Ordering in a Managed Care Setting Through Redesign of a Computerized Order Form

Medicare Part D Coverage Gap and Diabetes Beneficiaries

Knowledge, Attitudes, and Preferences Regarding Advance Directives Among Patients of a Managed Care Organization