To validate a risk stratification scheme using computerized pharmacy data to predict emergency hospital utilization for persistent asthma.

The development sample consisted of 1079 HMO members aged 18 to 56 years with persistent asthma. The scale used medication cut-points as predictors for next-year emergency hospital utilization in a stepwise logistic regression model. Prediction properties were evaluated in a validation sample of 24 370 patients aged 18 to 56 years in a separate persistent-asthma database.

Increasing use of β-agonists (odds ratio [OR] of 2.2 for 5-13 vs 0-4 canisters; OR of 2.4 for >13 vs 5-13 canisters) and oral corticosteroids (OR of 2.6 for >2 vs 0-2 dispensing events) in the first year independently predicted emergency hospital utilization in the second year. Assigning 1 point for exceeding each of the above 3 medication thresholds led to a 4-level medication intensity scale that was significantly (

< .0001) related to validated measures of asthma symptom severity, asthma control, and asthma quality of life in the development sample. In the validation sample, this scheme identified a high-risk group that was 6 times more likely than the low-risk group to require subsequent emergency hospital care, with overall sensitivity of 65% and specificity of 54%. This scale did not perform as well as a scale based on both baseline emergency hospital care and pharmacy data.

This simple risk stratification scheme can be used for populations with persistent asthma for whom computerized pharmacy data, but not computerized prior utilization data, are available.

Asthma is a common chronic medical condition that exacts a high human and economic cost in our society.

The prevalence and impact of asthma, along with the availability of effective controller therapy, make it an appropriate disease for population management, which involves disease identification, risk stratification, and therapeutic intervention in specified populations. Risk stratification is used in population management to identify those asthmatic patients who are most likely to experience morbidity and resource utilization, and for whom targeted intervention should reduce these risks. In addition, risk adjustment is necessary when using databases to evaluate the effects of interventions on asthma outcomes.

Several prior studies developed and validated risk stratification schemes based on electronic data to predict subsequent emergency hospital utilization for asthma.

However, all of these schemes included prior emergency hospital utilization as an important component of the risk prediction algorithm. Computerized pharmacy data are becoming increasingly available, but frequently they are not linked to hospital utilization data. Even if utilization data that could be linked to pharmacy data were available, it would be simpler to be able to use only 1 computer system for risk stratification.

We are aware of 3 prior studies that attempted to predict subsequent emergency hospital care based on stratification of computerized pharmacy data. Two were published only in abstract form,

and the other one was not validated in a separate population.

Thus, we thought it would be important to develop and validate a risk stratification scheme based on computerized pharmacy data alone that would predict subsequent emergency hospital utilization and identify patients for targeted intervention. In addition, we compared the performance of this medication-derived scale with that of a previously validated scale that included baseline emergency hospital utilization as well as pharmacy data.

This study is a retrospective cohort database study using 1 sample (the development sample) to develop the risk stratification scheme and a separate sample (the validation sample) to validate it. The study was approved by the Northern California, Northwest, and Southern California Regional Kaiser-Permanente institutional review boards.

Surveys were sent in August 2000 to a random sample of Kaiser-Permanente Medical Care Program adult members aged 18 to 56 years from the Northern California (n = 3072), Northwest (Oregon and Washington) (n = 543), and Southern California (n = 3251) regions who were diagnosed as having persistent asthma in 1999. The diagnosis of persistent asthma was based on the National Committee for Quality Assurance Health Plan Employer Data and Information Set (HEDIS) criteria

: (1) 4 or more asthma medications dispensing events, or (2) 1 or more emergency department visits or hospitalizations with a principal diagnosis of asthma, or (3) 4 or more asthma outpatient visits with 2 or more asthma medication dispensing events. The upper age of 56 years was chosen to reduce the inclusion of patients whose primary disease was chronic obstructive pulmonary disease, as was done in the HEDIS cohort.

Completed surveys were returned between August and October 2000 by 4175 members (61%), of whom 3765 (90%) answered yes to the question, "Have you ever been told by a doctor that you have asthma?" Computerized pharmacy information and year 2001 asthma hospitalization and emergency department information were available for 1079 patients who met the HEDIS persistent-asthma criteria in 2000 and who were continuously enrolled in 2000 and 2001. These patients represent the development sample for this study.

The survey included 3 validated asthma tools: (1) the Mini Asthma Quality of Life Questionnaire,

a disease-specific quality-of-life questionnaire with a 2-week window and in which higher scores indicate better quality of life; (2) the Asthma Therapy Assessment Questionnaire,

a validated asthma- control tool that has 5 levels, corresponding to 0 to 4 asthma-control problems as assessed over the prior 4 weeks; and (3) the Asthma Outcomes Monitoring System,

a symptom intensity scale that assesses symptoms over the previous 4 weeks.

Survey records were matched using a unique medical record number to year 2000 and 2001 computerized pharmacy records. These data included canisters per year of short-acting β-agonists, canisters per year of inhaled corticosteroids (not weighted for potency or puffs per container), and dispensings per year of oral corticosteroids. The 1079 patients in the development sample also had information on year 2001 asthma emergency department (ED) visits and hospitalizations available from linked hospital and outpatient visit databases.

Development of the Medication Intensity Scale

Candidate predictors for the medication intensity scale were the number of year 2000 (1) β-agonist canisters, cut at more than 0, more than 1, more than 2...and more than 17 per year; (2) inhaled-corticosteroid canisters, cut at more than 0, more than 1, more than 2...and more than 17 per year; and (3) oral-corticosteroid dispensings, cut at 1 or more, 2 or more, 3 or more, and 4 or more. The outcome was emergency hospital utilization for asthma during the year 2001, defined as at least 1 hospitalization or ED visit for asthma (

International Classification of Diseases, Ninth

code 493.xx), expressed as a yes/no variable. Stepwise logistic regression was used to identify independent predictors from among the above candidate predictors. In this analysis, the predictor with the strongest relationship to the outcome entered the model first. Subsequently, only predictors independently related at 

<.05 entered the model. Risk points then were assigned based on the relative odds ratios (ORs) of the predictors in the final logistic regression model.

The validation sample was derived from patients in the Southern California Kaiser Permanente asthma database. This database makes use of administrative data to define patients as having persistent asthma if they meet 1 or more of the following criteria in a 12-month period:

A hospital discharge principal diagnosis of asthma (

An ED visit with asthma as the principal diagnosis (

Four or more dispensings of any combination of inhaled β-agonists, inhaled corticosteroids, other inhaled anti-inflammatory drugs, and oral leukotriene modifiers, excluding oral corticosteroids (which may have been used for illnesses other than asthma) and oral theophylline (for which specific information was not available in this database).

These criteria are similar to the HEDIS criteria except for not capturing patients with 4 or more outpatient visits and 2 or more medications, because number of outpatient visits was not available in this dataset. Subjects in the validation sample were patients aged 18 to 56 years (to conform to the adult HEDIS population) who were included in the asthma database during both 2002 and 2003.

The utilization information included canisters of β-agonists and dispensings of oral corticosteroids in 2002 and 2003 as well as asthma ED visits and hospitalizations during 2002 and 2003.

Relationships between the medication intensity scale levels and the Mini Asthma Quality of Life Questionnaire, the Asthma Therapy Assessment Questionnaire, and the Asthma Outcomes Monitoring System were evaluated by means of 1-way analysis of variance. The outcome to be predicted in the development and validation samples was emergency hospital care during the second year of observation. Emergency hospital care was defined as 1 or more asthma hospitalizations or ED visits as a yes/no variable. Prediction properties of the scale were assessed by calculation of sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic (ROC) curve for various cut-offs of the scale. The latter was derived from the C statistic of a logistic regression model, with second-year emergency hospital care as the outcome and first-year scale cut-offs as the predictors.

The predictive properties of the medication scale in the current validation sample were compared with prediction properties in this sample of the previously validated risk stratification scheme that includes baseline- year emergency hospital care.

In this scheme 2 points are assigned for baseline-year emergency hospital care; 1 point for any baseline-year dispensings of oral corticosteroids; and 1 point for dispensing of 15 or more β-agonist canisters in the baseline year. This leads to scale scores of 0-4.

All analyses were conducted using SAS software, version 8.2, for Windows (SAS Institute Inc, Cary, NC). All 

values are 2 sided, and the term "significant" refers to a 

Characteristics of patients in the development and validation samples are compared in 

. Patients in the development sample were somewhat older, more likely to be female, and more likely to require emergency hospital care than patients in the validation sample (Table 1).

Increasing use of inhaled β-agonists and oral corticosteroids were significant independent predictors of future (2001) emergency hospital care for asthma in the development sample (

). Two β-agonist thresholds (>4 vs 0-4 canisters and >13 vs 5-13 canisters) and 1 oral-corticosteroid threshold (>2) were independent predictors of subsequent emergency hospital care. No inhaled-corticosteroid predictor entered the model. Noting the similar ORs associated with exceeding each of the β-agonist thresholds and the oral-corticosteroid threshold, we defined a simple 4-level risk score that assigned 1 point for each risk threshold that was exceeded (Table 2). Thus, 1 point was conferred by more than 4 β-agonist canisters, another point for more than 13 β-agonist canisters, a third point for more than 2 dispensings of oral corticosteroids, and no points if none of the above thresholds were reached, leading to a total score of 0-3 for each patient.

The risk of year 2001 emergency hospital care approximately doubled with each increasing medication scale level from 0-2 and tripled at level 3 in the development sample (

). Higher risk scores also were significantly associated with stepwise decreases in the Mini Asthma Quality of Life Questionnaire scores, indicating lower quality of life; increases in the Asthma Therapy Assessment Questionnaire scores, indicating less asthma control; and increases in the Asthma Outcomes Monitoring System scores, indicating greater symptom severity (

Table 3 also shows the proportion of validation-sample patients in each level of the medication intensity scale. The validation sample had fewer patients in risk levels 1-3 (47.0%) than the development sample (62.2%). With each level increase in scale score, the risk of year 2003 emergency hospital care for patients in the validation sample increased by 50% (risk levels 0- 1) to 100% (risk levels 1-2 and 2-3). Patients in the highest risk group were more than 6 times as likely as patients in the lowest risk group to require emergency hospital care in 2003 (Table 3).

Prediction performance of the medication intensity scale for emergency hospital care during the second year of observation in the development and validation samples is shown in 

. For all cut-offs, sensitivities, positive predictive values, and areas under the ROC curve were lower in the validation sample than in the development sample, whereas specificities and negative predictive values were higher. However, a reasonable balance of sensitivity (64.6%) and specificity (54.1%) could be obtained with a medication intensity scale cutoff of more than 0 in the validation sample, and identification of more than 25% of the high-risk patients could be accomplished with nearly 90% specificity by using a cut-off of more than 1 in the validation sample. A relatively high positive predictive value (26.8%) compared with the baseline risk (6.1%) and a very high specificity (99.1%) could be obtained with a medication intensity scale cut-off of more than 2, although this stratum encompassed only slightly more than 1% of the total number of patients (Table 3).

The proportions of patients in the validation sample in each of the previously defined scale levels (using both pharmacy and baseline emergency hospital care data) were as follows: 57.9% (risk level 0), 28.6% (risk level 1); 5.5% (risk level 2); 7.0% (risk level 3); and 1.0% (risk level 4). Sensitivity, specificity, positive predictive value, negative predictive value, and area under the ROC curve of any risk points (ie, >0) all were somewhat higher with the previously defined scale, which included baseline emergency hospital utilization, compared with the current scale derived from pharmacy data only, but the absolute values of these differences were less than 7%. Somewhat larger sensitivity and positive predictive value differences as well as a larger difference in areas under the ROC curves between the 2 schemes were seen using scale values of more than 1 (Table 5).

An important component of population management is risk stratification, and risk adjustment is important in observational database research that evaluates asthma interventions. Previously validated risk stratification schemes have included prior ED visits or hospitalizations as important predictors of subsequent utilization.

 However, because such information is less available than computerized pharmacy data, we felt it was important to develop a risk stratification scheme that did not depend on knowing patients' prior emergency hospital utilization.

Among patients in the validation sample, the currently developed scheme identified 12% as having a risk score of more than 1 (see Table 3); this group accounted for more than 25% of the patients who subsequently required emergency hospital care (see Table 5, "Sensitivity"). The false-positive rate (1 minus the specificity) was only approximately 11%. A gradation of risk was identified with increasing risk points, with a 50% to 100% increase in the risk of subsequent emergency hospital care with each additional point (Table 3). More than 1 in 4 patients in the highest risk group subsequently had emergency hospital care in the validation sample (Table 3).

The medication intensity scheme developed in this study has 2 major strengths. First, it was validated in a larger separate sample seen later in time. It was expected that the prediction performance in the validation sample would be less strong than that in the development sample (Table 5), both because the scheme was developed from the data in the development sample and because the incidence of the outcome was higher in the development sample than in the validation sample. However, as discussed below, the prediction properties of the scale in the validation sample appear clinically useful. Second, because it requires only β-agonist canister and oral-corticosteroid dispensing information, and includes only 3 criteria, the current medication intensity scale should be relatively simple to obtain and apply.

There are 2 main weaknesses of the current scheme. The first is that it missed approximately 35% of the patients who subsequently required emergency hospital care (1 minus sensitivity of score > 0, Table 5). This could be because patients with generally controlled asthma still may experience acute severe episodes. Alternatively, lower perception of dyspnea could explain why some patients with low chronic medication utilization experience acute episodes.

This study cannot differentiate between these alternatives or other explanations for the above findings. The second potential weakness is that 73% of highest risk patients did not require emergency hospital care in the next year (1 minus positive predictive value of score > 2, Table 5). (However, this is probably a less important disadvantage, because the survey data [Table 4] suggested that patients in the higher risk groups were experiencing more symptoms and decreased quality of life, which makes targeting such patients for intervention a fruitful effort.)

This study provides a method of risk adjustment for use in observational database studies when prior hospital utilization data are not available. Studies that assess pharmacologic interventions by using computerized pharmacy data could benefit from the use of the current scale for baseline risk adjustment. Such baseline risk adjustment also may be useful in assessments of quality of asthma care that use computerized pharmacy data.

The relationship of β-agonist use and oral-corticosteroid use to subsequent emergency hospital care was expected based on a number of prior studies.

Inhaled corticosteroids did not appear to affect the risk of subsequent emergency hospital care when β-agonist and oral-corticosteroid use also were accounted for. These results seem contrary to prior data suggesting that inhaled-corticosteroid use reduces the risk of subsequent emergency hospital care.

However, controller use can act as an independent severity marker.

In addition, inhaled corticosteroids do not lead to controlled asthma in all patients.

If inhaled corticosteroids are effective, this should result in less use of β-agonists. If inhaled corticosteroids are not effective enough, this should result in more use of β-agonists and possibly oral corticosteroids as well. Thus, β-agonist use and oral-corticosteroid use indirectly reflect inhaled-corticosteroid 

in treated patients, which is apparently more important in predicting subsequent utilization than inhaled corticosteroid use alone.

Asthma severity and asthma control are 2 related but separate constructs.

Severity can be defined clinically in untreated patients,

but to assess severity in treated patients, the Global Initiative for Asthma guidelines recommend consideration of the medication level.

The medication intensity scale defined in this study appears to be a severity marker because it is associated with increased hospital utilization and it is inversely related to asthma control and quality of life.

One prior published study used computerized pharmacy data to develop a pharmaceutically based severity stratification scheme for an asthmatic population.

The 5 levels tested were determined based on focus groups and defined based on the number of β-agonist canisters dispensed per year (cut-point of 4 canisters), other asthma medications besides systemic corticosteroids, and systemic corticosteroids (at least 1 prescription dispensed). The scheme was evaluated in relationship to acute bed days during the 12 months after the period when the severity level was assigned. Only the top 2 levels were associated with significantly increased hospital bed days, and validation in another population was not confirmed. Two other asthma severity stratification schemes using computerized pharmacy data have been proposed, but results have been published only in abstract form.

Because prior emergency hospital utilization is the strongest risk factor for subsequent utilization,

it is not surprising that a risk stratification scheme that included both baseline emergency hospital utilization and pharmacy data performed better than the current scheme based on pharmacy data alone, especially regarding the sensitivity and positive predictive value of a score more than 1 (Table 5). Thus, if baseline emergency hospital utilization data are available, the risk stratification system based on both types of data should be chosen. However, if baseline emergency hospital utilization data are not available or are difficult to obtain, the current medication-derived scheme should be useful because a reasonable balance of sensitivity (65%) and specificity (54%) could still be achieved with a score greater than 0, and a sensitivity of 27% with a specificity of 89% could be achieved with a score greater than 1. Which cut-off of the scale to use would depend on the purpose of the risk stratification exercise. If it was to identify patients for intervention, the intensity and expense of the intervention could determine the choice of cut-off. For a low-intensity intervention, a score greater than 0 may be chosen because of highest sensitivity (65%), with an acceptable false-positive rate (1 minus the specificity) of 46%. For a medium-intensity intervention, a cut-off of greater than 1 provides lower sensitivity (27%) but much higher specificity (89%). For a very intense intervention, a cut-off of greater than 2 identifies 1% of the population with a nearly 27% risk of subsequent emergency hospital care, with 99% specificity.

This study does have several potential limitations. It did not capture unscheduled outpatient visits other than ED visits, although ED visits would be the most expensive type of outpatient unscheduled asthma visits. Administrative data do not usually capture potentially important risk factors such as lower pulmonary function,

The study also did not capture clinical predictors such as symptom severity

although the current study demonstrates a strong relationship between the medication intensity scale and these measures.

A final issue is the external validity of this study. Patients receiving care from Kaiser Permanente are demographically similar to the working family populations from which they are drawn (unpublished data). However, the asthma hospitalization and ED utilization rate in the validation sample was quite low compared with levels reported in other populations.

This might be related to the access of this population to routine care, telephone interactions, urgent-appointment clinics, and specialty care.

The results of this study should be generalizable to West Coast adult populations with persistent asthma (as defined by database criteria) who have similar access to healthcare. Further studies will be necessary to confirm its generalizability to other populations.

In summary, we developed and validated a simple and practical 4-level risk stratification scheme based on computerized pharmacy data for use in management of asthma populations and for risk adjustment in database studies. We hope that this scheme will prove useful for identifying patients in need of targeted intervention and that future outcome studies will show reduced morbidity in higher risk patients who receive such targeted intervention.

From the Departments of Allergy at San Diego (MS, RSZ), Sacramento (AL), and Vacaville (GM), Calif, the Center for Health Research, Portland, Ore (WMV), and the Care Management Institute, Oakland, Calif (DM, TBS, MSJ), Kaiser Permanente Medical Care Program; the Division of Allergy-Immunology, Department of Pulmonary and Critical Care Medicine, University of Pennsylvania School of Medicine, Philadelphia, Pa (AJA); and the Department of Epidemiology, Harvard School of Public Health, Boston, Mass (EFC).

This research was supported by the Kaiser Permanente Medical Care Program.

Address correspondence to: Michael Schatz, MD, Chief, Department of Allergy, Kaiser Permanente Medical Center, 7060 Clairemont Mesa Blvd, San Diego, CA 92111. E-mail: michael.x.schatz@kp.org.

Articles

METHODS

Development and Validation of a Medication Intensity Scale Derived From Computerized Pharmacy Data That Predicts Emergency Hospital Utilization for Persistent Asthma

To assess healthcare use among veterans with diabetes mellitus (DM) enrolled in a Department of Veterans Affairs (VA) Care Coordination Home Telehealth (CCHT) program during 24 months and to contrast this utilization with the service use of a comparison group of veterans with DM not enrolled in the program. 

Two-year, retrospective, concurrent matched cohort study design. 

The VA CCHT program included older veterans with type 2 DM at high risk for multiple VA inpatient and outpatient visits. Healthcare utilization (hospitalizations, length of stay, and outpatient visits by type) was assessed at baseline and at 24 months after intervention for the treatment (n = 400) and comparison (n = 400) groups. Propensity scores were used to improve the balance between the treatment and comparison groups. A difference-in-differences approach was used to control for selection bias and for intervening time factors. 

Two years after enrollment, the treatment group exhibited a statistically significant reduction in the likelihood of allcause and DM-related hospitalizations. In a subgroup analysis in which we controlled for patients' baseline glycosylated hemoglobin levels, the treatment group had a lower likelihood of having any care coordinator-initiated primary care clinic visits (in which the care coordinator initiated referral to primary care based on health information received from patients' CCHT technology). 

After controlling for selection bias and for intervening time factors, the VA CCHT program reduced avoidable healthcare services for DM (such as hospitalizations) and reduced care coordinator-initiated primary care clinic visits. 

Diabetes mellitus (DM) is associated with high rates of morbidity and costs for the Department of Veterans Affairs (VA).

reported that veterans with DM received 30% of all VA pharmacy prescriptions. Diabetes mellitus that is not managed appropriately can lead to adverse outcomes such as limb amputations, and veterans with DM who have had limb amputations experience 1.6 times as many hospitalizations as veterans without DM.

Many veterans with DM face healthcare access barriers such as having to travel long distances to VA medical centers to use outpatient services.

During the past decade, the VA has undergone a tremendous structural transformation from a hospital-based system to a system geared to greater ambulatory care. A more recent development has been the patient-centered care coordination perspective, which aims to reduce healthcare access barriers for veterans and to provide care in a more efficient manner for community-dwelling veterans with chronic diseases who have complex needs.

This extends the notion of disease management by more efficiently integrating the various healthcare needs of veterans with disabling chronic diseases (such as DM) with system resources. Care coordination aims to unify care so that the amount and timing of care for each veteran are appropriate for that veteran and not simply based on customary population- based rules that may not fit the individual veteran's needs.

Home telehealth technologies, which integrate information and communication technologies through communication services such as messaging devices (interactive caller ID-type devices) and videophones, allow the veteran and his or her care coordinator to maintain direct communication.

Care coordination and supportive home telehealth technologies may be especially helpful for people who travel long distances for care or who experience lengthy clinic wait times.

A principal goal in the VA Care Coordination Home Telehealth (CCHT) program is to reduce avoidable and costly VA healthcare services such as hospitalizations. Two observational studies

of veterans with DM examined healthcare use 1 year before and 1 year after enrollment into a VA CCHT program. Multivariable results indicated a significant reduction in the proportion of patients who were hospitalized and a reduction in the mean length of stay for those who were hospitalized. Although the findings from these 2 studies were informative, they used a simple pre-post design that may be substantially affected by selection bias or by regression to the mean, statistical phenomena occurring whenever a nonrandom sample is derived from a population and 2 flawed correlations are assessed.

Indeed, many disease management investigations have faced considerable limitations such as regression to the mean, selection bias, poor follow-up, limited settings, and publication bias.

More rigorous research is needed to assess the quality of programs such as the VA CCHT.

The present study was designed to assess healthcare service use in an ethnically diverse group of veterans with DM who were enrolled in a VA CCHT program during 24 months and to contrast this utilization with the service use of a comparison group of veterans with DM who did not receive any intervention. The objective of this study was to rigorously test the effectiveness of the VA CCHT intervention to determine if there was a reduction in avoidable, costly services during 24 months by (1) using propensity scores to improve the balance between the treatment and comparison groups and (2) using a difference-in-differences (DiD) approach to control for regression to the mean and for selection bias. 

The VA CCHT program was implemented at 4 VA medical centers in a Veterans Integrated Service Network that covered most of Florida, Puerto Rico, and southern Georgia. The predominant type of home telehealth technology used was a messaging device. This device operates using basic telephone service and an electrical outlet. Patients used this messaging device daily to answer scripted questions about their DM symptoms and health status. The program consisted of nurse care coordinators (registered nurses or advanced registered nurse practitioners) who used disease management protocols to manage treatment and to educate the veterans about their disease to prevent more costly interventions (hospitalizations or emergency department [ED] visits). The care coordinators monitored responses daily and made clinical judgments regarding whether a telephone call should be placed to the patient or an appointment should be made with the patient's physician. In the instances in which a telephone call was made, it lasted on average 15 to 30 minutes. The duration of the telephone call depended on the complexity of the issue. The care coordinators performed multiple tasks that might include, but were not limited to, performing patient assessments, placing new orders for patient medications, helping patients manage their medications, scheduling new appointments in particular VA clinics, reminding patients of their clinic appointments, and assisting with technology difficulties. In rare circumstances, a telemonitor and a videophone with 2-way audio/video connectivity were used for weekly contact. 

We used a retrospective, concurrent matched cohort study design that was institutional review board approved. The VA CCHT program included veterans with DM who were at high risk for expensive, multiple VA inpatient and outpatient visits, including ED visits. Veterans with DM were eligible for the program if they had 2 or more VA hospitalizations or 2 or more VA ED visits in the 12 months before enrollment. Veterans needed access to a working telephone line and had to be noninstitutionalized to be enrolled. The treatment and comparison groups were matched on the basis of the treatment group members' study enrollment dates, so that both groups had identical distributions of enrollment and service periods.

Both groups had to remain enrolled in the VA for the entire 24-month observation window. Three comparison group participants were randomly selected for each member of the treatment group to ensure an adequately sized comparison group. To improve the match between the treatment and comparison groups, we used propensity scores by (1) estimating a model of the probability that a patient "selects" into the treatment group as opposed to the comparison group, (2) dividing our sample into the quintiles of the predicted propensity scores distribution (<20%, 20% to <40%, 40% to <60%, 60% to <80%, and ≥80%), and (3) randomly sampling equal numbers of individuals in the treatment and comparison groups from each quintile. Although the application of propensity scores improved the balance between the treatment and comparison groups, there remained a few statistically significant differences between the groups, as summarized in 

. To control for these remaining differences, we used a DiD approach to measure the treatment effect, as explained herein. 

Any simple pre-post comparison of use for a treatment group may be biased by regression to the mean in healthcare use. Regression to the mean in healthcare use refers to the tendency of high users in any single period to exhibit lower use in subsequent periods (ie, for their healthcare use to regress to the mean).

 Regression to the mean can lead to faulty conclusions and to a misunderstanding of test results.

Because the treatment group was chosen based on high levels of prior use, we might expect the treatment group to exhibit lower use in subsequent periods strictly from regression to the mean, independent of any intervention effect. Consequently, failure to correct for regression to the mean in use may cause researchers to incorrectly attribute reduction in use to the intervention.

In addition, because veterans with DM were not randomly assigned to the treatment and comparison groups, selection bias could be present. Selection bias occurs when some poorly observed factor influences the assignment of the patient to the treatment group vs the comparison group, as well as the healthcare outcomes of the patient. 

To control for any regression to the mean and selection bias, we used a DiD evaluation design. The DiD approach has long been used in studies of labor economics

 and has recently been applied to health services research.

The DiD approach seeks to measure a treatment effect while accounting for any pretreatment differences (ie, selection bias) between the treatment and comparison groups, as well as any intervening time factors (ie, regression to the mean). The concept of the DiD approach is illustrated in the 

, which shows a hypothetical example of pretreatment and posttreatment outcomes in the treatment and comparison groups before and after an intervention. Before the intervention, the difference between the treatment and comparison groups measures any innate or intrinsic difference between the 2 groups. In the Figure, this is indicated by "Comparison Group/Pre"minus "Treatment Group/Pre." As shown in the Figure, the comparison group has higher outcome values than the treatment group before the intervention. 

Following the intervention, the difference between the treatment and comparison groups measures the treatment effect plus intrinsic difference. In the Figure, this is indicated by "Comparison Group/Post" minus "Treatment Group/Post." To calculate the treatment effect alone, we must subtract the intrinsic difference between the 2 groups from the combined treatment effect plus intrinsic difference. The intrinsic difference is indicated by the difference between the treatment and comparison groups before the intervention, while the treatment effect plus intrinsic difference is indicated by the difference between the treatment and comparison groups following the intervention. So, to calculate the treatment effect alone, we subtract the difference between the treatment and comparison groups before the intervention from the difference between the treatment and comparison groups following the intervention. Therefore, the treatment effect is measured as the difference between 2 differences, hence the term 

. By obtaining the treatment effect alone and by eliminating any intrinsic difference between the treatment and comparison groups, we controlled for observed differences between the treatment and comparison groups (Table 1) and for any unobserved differences that might bias the treatment effect. 

Service use outcomes were measured at baseline and at 24 months after enrollment in the VA CCHT program. We derived service use from the National Patient Care Database, a Veterans Health Administration database that consists of inpatient and outpatient data. Inpatient service use included hospitalizations (all cause vs DM related) and length of stay (all cause vs DM related). Outpatient service use was measured as ambulatory care clinic visits and was categorized as (1) ED (all cause vs DM related), (2) care coordinator-initiated primary care clinic visits (in which the care coordinator initiates the referral to primary care based on health information received from the patient's home telehealth technology), (3) podiatry, (4) ophthalmology, or (5) diabetes specialty clinic. Because we used the Veterans Health Administration database for our data, we used 

codes to calculate the Dartmouth-Manitoba comorbidity index developed by Romano et al.

We added an additional code for depression but used it as a dummy variable. We collected data on the participants' age, marital status (married or not married), ethnicity (white, Hispanic, black, or other), facility site (A, B, C, or D), and service-connected disability (none, 10%-49%, or ≥50% based on disabilities that were sustained or aggravated during military service). These sociodemographic and comorbidity variables were used as covariates in the models. We obtained baseline glycosylated hemoglobin (A1C) levels from patient medical records for 1 site (facility site B). We conducted a subanalysis among this group and used these values as an important clinical control variable. 

We used logistic regression analysis for binary service use outcomes (eg, hospitalizations, ≥1 ED visits, and ≥1 care coordinator-initiated primary care clinic visits) and ordinary least squares regression analysis for continuous outcomes (eg, length of hospital stay). All statistical tests were 2-tailed, and the analyses were performed using SAS version 8.0 (SAS Institute, Cary, NC). Statistical significance was set at 

<.05. We used the estimator for the interaction effect by Ai and Norton

 for the logistic regression models because the interaction effect in nonlinear models must be calculated differently from that in linear models. 

Table 1 gives descriptive assessments of the treatment and comparison groups before the intervention. Before the intervention, there were no statistically significant differences in age, marital status, ethnicity, facility site, service-connected disability, hospitalizations (DM-related), length of stay (DM-related), or outpatient visits between the 2 groups. There were comorbidity differences between the treatment and comparison groups, with the treatment group reporting more comorbidities (1.4 vs 0.7). 

gives the service use multivariable preintervention and postintervention outcomes in the treatment and comparison groups. The treatment effect, which was measured using the DiD approach, is given in the column on the far right. These outcomes were adjusted for sociodemographic and health status characteristics. Two years after enrollment, there was a significant difference between the treatment and comparison groups in the likelihood of all-cause hospitalizations, decreasing in the treatment group from 38.8% to 30.0% (

= .01) and increasing in the comparison group from 31.2% to 33.1% (

= .61). The treatment group had a significant reduction in DM-related hospitalizations during 24 months from 35.3% to 26.9% (

= .02). The comparison group demonstrated little change in DM-related hospitalizations. The DiD estimate of -9.1 percentage points was significant at 

= .02, suggesting that the intervention reduced hospitalizations by almost 25%. 

A -6.1-percentage point change in DM-related hospitalizations was attributable to the intervention (

= .08) (Table 2). Care coordinator-initiated primary care clinic visits increased by 8.7 percentage points under the intervention (

= .04), while podiatry visits decreased by 6.2 percentage points (

Although the results in Table 2 suggest that the intervention dramatically increased ED visits, this result is an artifact of the inclusion criteria for the treatment and comparison groups. Because the treatment group enrolled a disproportionate share of patients with DM with multiple hospitalizations, the comparison group enrolled a disproportionate share of patients with DM with multiple ED visits to meet the inclusion criteria. This resulted in 98% of the comparison group having an ED visit before the intervention, thereby rendering any meaningful comparison impossible. 

gives results only from site B, at which baseline A1C levels for the treatment and comparison groups were available for inclusion in our models. During 24 months, the treatment group experienced a significant decrease in all-cause hospitalizations from 60.3% to 40.7% (

= .03). The comparison group showed a slight increase from 36.8% to 38.8% (

= .84). As measured by our statistical models, the difference between these 2 groups was -18.0 percentage points (

= .055), indicating that the intervention reduced hospitalizations. The treatment group showed a decrease in DM-related hospitalizations from 54.6% to 37.0% (

= .04), but a statistically significant treatment effect was not detected. 

In terms of the care coordinator-initiated primary care clinic visits, the treatment group experienced a significant reduction during 24 months, decreasing from 59.0% to 21.0% (

<.001), while the comparison group experienced a reduction that approached significance, from 38.0% to 22.6% (

= .06) (Table 3). The estimated intervention effect was -21.0 percentage points (

= .03), indicating that the intervention reduced the number of patients requiring care coordinator-initiated primary care clinic visits. Patients' baseline A1C levels had a statistically significant main effect on hospitalizations, such that patients with higher baseline A1C levels had a greater odds of having at least 1 hospitalization at 24 months after the intervention. 

We found a significant difference between the treatment and comparison groups in the proportions with at least 1 hospitalization (all cause) during 24 months, with the treatment group exhibiting a significant reduction. There were significant differences between the treatment and comparison groups for care coordinator- initiated primary care clinic visits, with the treatment group experiencing a slight increase in visits and the comparison group exhibiting a slight decrease in visits. 

We conducted a subanalysis for 1 site in which we controlled for baseline A1C levels. We found a significant difference between the treatment and comparison groups for all-cause hospitalizations. The treatment group experienced a significant reduction in the proportion with at least 1 hospitalization during 24 months. There was a significant difference between the treatment and comparison groups for care coordinator-initiated primary care clinic visits. The treatment and comparison groups experienced reductions in care coordinator-initiated primary care clinic visits, but the reduction was larger in the treatment group. 

This retrospective study examined the effectiveness of a VA CCHT program for a large group of ethnically diverse patients with DM by examining healthcare service use outcomes at 24 months after the intervention. In the absence of a randomized controlled trial, our quasiexperimental design attempted to overcome methodological shortcomings by using the following 2 important state-of-the-art procedures: (1) propensity scores to improve the balance between the treatment and comparison groups and (2) a DiD approach to evaluate treatment effectiveness. 

During 24 months, the treatment group experienced a significant reduction in the proportion with at least 1 hospitalization. These findings using propensity scores and a DiD approach support the pre-post observational findings of previous studies

that showed a significant reduction in hospitalizations during 12 months. Our results suggest a reduction in hospitalizations by the VA CCHT program during 24 months associated with better home management of patients with DM. 

In our subgroup analysis, we examined 1 of 4 sites in greater detail. These investigations yielded an intriguing finding with reference to the care coordinator-initiated primary care clinic visits. The treatment group showed a stronger significant decline in care coordinator-initiated primary care clinic visits than the comparison group. In a matched cohort study of the VA CCHT program for patients with DM, Chumbler et al

reported that at 12 months the treatment group experienced a significant increase in initial care coordinator-initiated primary care clinic visits, suggesting that veterans' health status was monitored and clinical needs were met just in time before health deterioration. During 24 months, however, our results show a decrease in the need for these appointments. With daily monitoring via the home telehealth technology, care coordinators may have been able to identify subtle health changes, assist patients in managing their health problems, and resolve these problems before they became serious enough for a care coordinator-initiated primary care clinic visit. The explanation of these findings is congruent with the tenets of the chronic care model.

That is, with the paradigm shift to coordinated care and the increase in patient self-management, DM symptoms are brought under better management, requiring fewer care coordinator- initiated primary care clinic visits. 

Contrary to our findings in the subanalysis sample, there was a significant increase in the likelihood of care coordinator-initiated primary care clinic visits in the overall sample. Unfortunately, we do not have available data to definitively understand why these findings occurred. It may be that the site at which we conducted the subanalysis contained more motivated staff, who more carefully followed the protocols and performed better triage, referring only those patients who required the care of a VA primary care physician. It is possible that the program was established differently at site B and comprised care coordinators who were more experienced. Future research should further examine these important managerial and clinical issues. 

Some limitations in our study deserve mention. Although patients were comparable with respect to age, ethnicity, service-connected disability, DM-related hospitalizations, and DM-related length of stay, they were not comparable with respect to comorbidities. Although we took careful steps to use the same inclusion criteria to retrospectively obtain a comparison group that was comparable to the treatment group, the treatment group tended to be selected based on hospitalization criteria, while the comparison group was chosen largely based on ED visit criteria. Conceptually, our DiD approach should correct for these differences. Although our propensity scores and DiD approach were chosen to help compensate for differences, they cannot automatically account for clinical variables that are potentially important to a patient population with DM such as body mass index, insulin use, low-density lipoprotein cholesterol levels, systolic and diastolic blood pressures, and time since diagnosis.

Our study included limited sociodemographic information, such as the presence and level of social support and whether patients had insurance other than VA coverage, which may have affected our outcome findings. Although the ED visits reported in the comparison group appeared to be an artifact of selection bias, the treatment group had high rates of hospitalizations and ED visits. This is not due to the fact that the treatment group was selected for hospitalizations, whereas the comparison group was selected for ED visits. Rather, it may be attributable to the fact that the comparison group was selected primarily on the basis that the treatment group contained most of the available patients who had multiple hospitalizations. Unfortunately, we do not have access to data on the number of contacts or the duration of the contacts between the care coordinators and the patients. Further research might incorporate such information to examine the associations between CCHT correspondence and change in service use. 

Costs are of significant concern when implementing and evaluating new programs. Our data precluded us from assessing the cost-benefit ratio or the cost-effectiveness of the CCHT program for veterans with DM. We did not have access to program development or implementation costs at the 4 sites. To obtain a more comprehensive evaluation of the program effectiveness, future research should collect program and service use costs for the treatment and comparison groups. 

Our study was implemented in 1 Veterans Health Administration geographical region; thus, generalizability may be an issue. The patients included in this VA CCHT program and the sites at which it was implemented differ notably from those in previous disease management-oriented investigations, which were primarily executed in managed care and other private insurance-type settings. 

However, the region chosen for our study includes a large, ethnically diverse population with DM with existing management issues and who needed more intensive follow-up than the traditional medical care system provided.

Our results can be informative as similar programs and interventions are planned to be implemented throughout the VA and outside the VA. 

Our findings suggest that the VA CCHT program can minimize avoidable healthcare service use at 24 months after implementation, as evidenced by the reduction in hospitalizations in the overall group and in the subanalysis group and by the reduction in care coordinator-initiated primary care clinic visits in the subanalysis group. These results are based on the use of a well-matched comparison group and a DiD approach that controlled for selection bias, avoiding the pitfalls that are common in disease management program investigations. 

From the Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System (TEB, NRC, WBV, RJB, HQ); Veterans Affairs Health Services Research and Development Stroke Quality Enhancement Research Initiative (TEB, NRC, WBV, RJB); Departments of Health Services Research, Management and Policy (NRC), Epidemiology and Health Policy Research (WBV), and Aging and Geriatrics (RJB), University of Florida, Gainesville; and Veterans Health Administration Office of Care Coordination, Washington, DC, and Lake City, Fla (RK). 

This research was funded by the Veterans Affairs (Veterans Integrated Health System-8) Community Care Coordination Service and was supported by the Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System. 

This study was presented at the 24th Veterans Affairs Health Services Research and Development National Meeting; February 16, 2006; Arlington, Va. 

Address correspondence to: Tracey E. Barnett, PhD, Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System, 1601 SW Archer Rd, 151B, Gainesville, FL 32608. E-mail: tracey.barnett@va.gov. 

MANAGERIAL

The Effectiveness of a Care Coordination Home Telehealth Program for Veterans With Diabetes Mellitus: A 2-Year Follow-up

To measure the timing and frequency of follow-up care during the initiation phase of antidepressant therapy, and to compare the typical pattern of care with current product guidelines. 

The study included 84 514 adult and pediatric patients who started a new course of antidepressant therapy for any indication between July 2001 and September 2003. Patients were members of a large managed care organization in the northeastern United States. Ambulatory visits during the first 12 weeks of treatment were identified using medical claims data. Outcome measures were time to first follow-up visit, frequency of follow-up visits, and percentage of patients receiving recommended levels of care. 

During the first 4 weeks of treatment with antidepressants, only 55.0% of patients saw a healthcare provider for any purpose, and only 17.7% saw a provider for mental healthcare. Ambulatory visit rates during the first 4, 8, and 12 weeks were significantly lower than the minimum levels recommended in product labeling (

<.0001). Only 14.9% of patients received the Food and Drug Administration-recommended level of follow-up care during the first 4 weeks, 18.1% at 8 weeks, and 22.6% at 12 weeks. 

Adults and children who begin a new course of antidepressant therapy tend to receive far less monitoring than is recommended in current product labeling. Given safety concerns during the initiation phase, earlier and more frequent follow-up care appears desirable. Further research is needed to identify the most cost-effective schedule of care. 

In October 2004, the Food and Drug Administration (FDA) directed manufacturers to include a new "black box" warning and expanded warning sections in the labeling for all antidepressant products.

The new labeling warns healthcare providers about the increased risk of suicidal thoughts and actions ("suicidality") in children and adolescents who take antidepressant medications for any indication, including major depressive disorder and obsessive-compulsive disorder (OCD). The FDA also developed a patient medication guide on the risk of suicidality and the precautions that can be taken by patients, parents, and caregivers.

These regulatory actions were based on a combined analysis of adverse events in 24 clinical trials that studied the use of antidepressant drugs in pediatric populations. When results were pooled across all drugs and all indications, the analysis showed a small but statistically significant increase in suicidality risk for antidepressants (compared with placebo) during the first few months of treatment.

The new warning labels advise healthcare providers to monitor pediatric patients closely for signs of clinical worsening, suicidality, or unusual changes in behavior, especially during the first few months of a new course of antidepressant therapy or after a change in dose.

The warning labels recommend a specific pattern of followup care: at least weekly face-to-face visits during the first 4 weeks of treatment, visits every other week during the next 4 weeks, a visit at 12 weeks of treatment, and visits as indicated clinically beyond that point. Additional contacts by telephone as needed are encouraged between the face-to-face visits. More concentrated monitoring during the early stages of treatment is recommended because there is some evidence that the risk of suicidality may be higher during the first few weeks of treatment.

Although the risk of suicidality in children and adolescents has been the primary focus of public and regulatory attention, the FDA also has begun to address the possibility of an increased risk of suicidality in adult users of antidepressants.

Warnings about this risk were first introduced into the labeling of selected products in March 2004, and they were expanded and applied to all antidepressant products in October 2004.

The new product labels recommend "close monitoring" of adult patients for signs of clinical worsening or suicidality, especially during the first few months of a new course of therapy for depression, and they recommend an intensive pattern of follow-up visits similar to the pattern recommended for pediatric care.

 In June 2005, the FDA announced that it had begun a comprehensive analysis of adult clinical trial data (similar to the combined analysis of pediatric trial data) in an effort to create a stronger evidentiary basis for future regulatory actions in this area.

The new suicidality warnings build on a broader framework of clinical practice guidelines for the use of antidepressants in the treatment of depression, anxiety disorders, and other conditions. These guidelines have been developed most extensively for the treatment of depression in adults,

whereas relatively few have been developed for the pharmacological treatment of anxiety disorders

or for pediatric use of antidepressants for any indication. Guidelines for the treatment of depression call for close monitoring of patients during the acute phase of treatment, which generally lasts a few months and ends with remission of symptoms.

The primary goals of monitoring are to assess the effectiveness of therapy, identify side effects, monitor the patient's condition, adjust therapy as needed, and promote adherence.

Guidelines for the treatment of anxiety disorders also call for close monitoring of patients who receive medication therapy.

Guidelines vary widely in the recommended frequency and timing of follow-up visits for patients who begin a new course of antidepressant therapy. The American Psychiatric Association recommends a vigorous schedule of monitoring during the acute phase of treatment for depression–at least weekly contacts in routine cases and multiple contacts per week in more complex cases.

Monitoring contacts are defined broadly to include face-to-face visits, telephone contacts, or contacts with other knowledgeable clinicians, depending on the circumstances.

In an earlier set of guidelines, the Agency for Health Care Policy and Research recommended face-to-face visits every 10 to 14 days during the first 6 to 8 weeks of treatment for patients with less severe depression, and weekly visits in more severe cases.

Other clinical guidelines recommend follow-up contacts on a weekly, biweekly, or monthly basis, depending on a variety of factors.

 The FDA guidelines fall roughly in the middle of this range–weekly face-to-face visits during the first month, tapering to monthly face-to-face visits by the third month. 

Many studies have examined patterns of treatment response to antidepressant medications, but only a few have reported data on the typical frequency of follow-up visits for adult or pediatric patients who initiate antidepressant therapy in usual practice.

The limited available data suggest that follow-up visit rates tend to be low,

but none of these studies provide a direct measure of the concordance between usual practice and the new FDA guidelines. The most commonly used measure of visit frequency was developed by the National Committee for Quality Assurance to assess the pharmacological management of depression.

As defined in Health Employer Data and Information Set (HEDIS) specifications, "optimal practitioner contact" frequency is a minimum of 3 follow-up visits for mental healthcare during the first 12 weeks of a new treatment episode; at least 2 of the measured visits must be face-to-face, and 1 may be a telephone contact.

This metric is useful for evaluating the overall quality of care, but it is not well suited to measuring the concordance of current practice with the new FDA guidelines for antidepressant products. The HEDIS metric is limited to patients with depression (rather than any condition treated with antidepressants), it provides no information on the follow-up care received by pediatric patients (because the metric is defined only for adults), and it provides no information on the time course of visits during the first 12 weeks of care. Without more comprehensive benchmark data, it is difficult to gauge whether the pattern of care defined in the new product labeling is feasible and realistic in the context of current practice. 

The objectives of this study were to measure the timing and frequency of follow-up visits during the initial phase of antidepressant therapy, and to compare the typical pattern of care with the pattern recommended in product labeling. The design of the study mirrored the scope and content of the new FDA guidelines. The study measured the time course of face-to-face visits for any patient (adult or pediatric) who began using an antidepressant medication for any treated condition, and it compared the typical pattern of visits with the time course recommended in product guidelines. Although the new guidelines apply equally to all pediatric users, we measured visit frequency separately for 3 pediatric age groups (0-6 years, 7-12 years, and 13-18 years), because of the wide variations in treated conditions and incidence of use across these groups. 

Patients were participants in medical and pharmacy benefit plans administered by a large managed care organization in the northeastern region of the United States; the medical plans included coverage for mental healthcare. The study population included all plan participants who had combined eligibility for medical and pharmacy benefits at any time during the study period, which extended from January 1, 2001, through December 31, 2003. A total of 2 411 316 plan participants met the criteria for inclusion. 

Medical utilization data (including mental health claims data) were drawn from an administrative claims database maintained by the managed care organization. Drug utilization data were drawn from a prescription claims database maintained by Medco Health Solutions, Inc, the pharmacy benefits management company that manages the prescription benefit plans for this population. Medical and pharmacy claims data were integrated at the patient level; personal identifying information was not retained. 

Patients were initially identified for the study sample if they filled a prescription for an antidepressant during an identification period that extended from July 1, 2001, through September 30, 2003. Prescriptions for all classes of antidepressants were included in the analysis (see the 

available online at www.ajmc.com). A prescription was considered the start of a 

 if no antidepressant prescriptions had been filled for the same patient during the prior 180 days (a 6-month negative medication history). The 

for the new episode was the fill date of the prescription at the start of the episode. Patients were included in the final study sample if they had continuous benefits eligibility during the 6 months before the index date and the 3 months after the index date. A total of 84 514 patients with new episodes of antidepressant therapy met the criteria for inclusion in the study sample. A small percentage of patients (4.9%) had 2 or more new episodes during the identification period; these recurrent episodes were included in the final study sample. As defined here, a new episode is a new course of therapy for any condition treated with antidepressants; this definition accords with the scope of the new warning labels for these products. 

For each patient in the study sample, medical claims were used to track ambulatory follow-up care during a 12-week analysis period after the index date of the new episode. Ambulatory care included office visits and outpatient visits; telephone visits were not included. Follow-up visits were identified at 2 levels of specificity: all ambulatory visits and mental health visits. 

Ambulatory visits were identified by place-of-service codes for office and outpatient visits (Appendix). All visits were included, no matter what diagnostic coding was used in billing. This measure provides an upper bound for follow-up visits where the patient's response to antidepressant therapy may have been addressed. All face-to-face visits provide an opportunity for follow-up care, even if the visit was scheduled or coded for another purpose. 

Mental health visits were identified by Current Procedural Terminology (CPT) codes for evaluation and management visits in conjunction with mental health diagnostic codes, or by CPT codes for psychiatric visits (Appendix). This measure provides a lower bound for follow-up visits where the patient's response to antidepressant therapy was likely addressed. The measure is likely to underestimate the actual number of follow-up visits, because some mental health visits may not be coded as such. 

For each patient starting a new episode of therapy, the patient's age and sex were derived from an eligibility database maintained by the managed care organization. Patient age was measured at the index date of the new episode. 

Analyses were conducted separately for children (age =18 years) and adults (age =19 years). For each age group, 3 sets of measures were derived from the claims data: 

The percentage of patients who had at least 1 follow-up visit during a specified interval after the start of a new episode. This percentage was measured at 1, 2, 3, 4, 8, and 12 weeks after the index date. 

August 2006

Development and Validation of a Medication Intensity Scale Derived From Computerized Pharmacy Data That Predicts Emergency Hospital Utilization for Persistent Asthma

The Effectiveness of a Care Coordination Home Telehealth Program for Veterans With Diabetes Mellitus: A 2-Year Follow-up

Frequency of Follow-up Care for Adult and Pediatric Patients During Initiation of Antidepressant Therapy

Economic and Clinical Outcomes of a Physician-led Continuous Quality Improvement Intervention in the Delivery of Percutaneous Coronary Intervention

Continuous Quality Improvement Initiatives in the Cardiac Catheterization Laboratory: Can We Have Our Cake and Eat It Too?

Polypharmacy With Oral Antidiabetic Agents: An Indicator of Poor Glycemic Control