To identify cases of inappropriate physician prescribing in a managed care setting in Israel that may have resulted from misuse of magnetic-stripe membership cards. 

Retrospective drug utilization analysis of electronic patient prescription data. 

In a managed care setting involving approximately 1000 physicians who issue approximately 1.4 million prescriptions annually, the rate of prescription of sex-specific drugs to patients of the opposite sex for which the drugs are indicated was evaluated for 2003. The categories of drugs included in the analysis were drugs for the treatment of benign prostatic hyperplasia or erectile dysfunction that were prescribed to women, as well as oral contraceptives, vaginal pessaries, hormone therapy, or raloxifene hydrochloride prescribed to men. 

Throughout the study year, 193 different physicians wrote 341 prescriptions that matched the drug inclusion criteria for 210 different patients. The most frequently observed scenario involved the prescription for women of selective Ã¡-blockers, including alfuzosin hydrochloride, tamsulosin hydrochloride, and terazosin hydrochloride, that are indicated exclusively for the treatment of benign prostatic hyperplasia. 

The electronic patient record system used in the health maintenance organization studied was programmed to block the prescription of sex-specific drugs for patients of the opposite sex for which they are intended unless proper authorization has been obtained. Furthermore, periodic investigation into prescription impropriety may be easily accomplished through the implementation of pharmacoepidemiological methods commonly used in drug utilization studies. 

Health insurance fraud involving inappropriate physician prescribing of drugs has become an issue of significant concern in the managed care setting internationally. In Israel, one aspect of this matter involves the misuse of member identification cards for the illegitimate acquisition of prescription drugs. Under the Israeli National Health Law, all Israeli citizens are enrolled in 1 of 4 health maintenance organizations (HMOs) that deliver a uniform, government-defined benefits package. This study was conducted in the Leumit Health Fund (an HMO), which serves approximately 700 000 members throughout the state of Israel (approximately 10% of the population) and involves approximately 1000 physicians who issue approximately 1.4 million prescriptions annually. Leumit Health Fund has implemented a computerized electronic patient record (EPR) system that requires patients to present a magnetic-stripe membership card to be electronically registered by the physician before receiving treatment. In addition to recording the medicines prescribed during the session, Leumit Health Fund's EPR system generates prescriptions on HMO prescription forms that entitle members to purchase drugs for a copayment, which generally provides a significant savings to the patient. Leumit Health Fund employees who receive coverage from this HMO also benefit from a 60% discount on the copayment. Because HMO membership is acquired on an individual basis and not on a family basis, situations occur in which only one member of a family is covered by Leumit Health Fund, with the remainder of the family being covered by 1 or more of the other 3 HMOs. There is concern that Leumit Health Fund employee members may be requesting physicians to prescribe drugs for next of kin who are not covered by Leumit Health Fund, to receive the employee discount or to avoid paying a replacement fee for a family member's lost membership card. Such behavior results in fraudulent data entries into the true cardholder's EPR of diseases diagnosed, drugs prescribed, and other information. Furthermore, because this information will not be documented in the patient's file, future practitioners will be denied what is potentially essential information concerning the patient's history. The objective of this study was to evaluate the prevalence of illegitimate prescriptions for patients involving physician complicity, while exploring the feasibility of implementing pharmacoepidemiological methods of drug utilization analyses as managerial tools. 

This analysis evaluated the rate of prescription of sexspecific drugs to patients of the opposite sex for which the drugs are intended. The categories of drugs included in the analysis appear in the 

Using a prescription database generated by EPR data, all prescriptions written throughout 2003 for patients of the opposite sex for which the target drugs are intended were identified and analyzed. 

The results of the study are presented in the Table. Throughout the study, 193 different physicians wrote 341 prescriptions that matched the drug inclusion criteria for 210 different patients. The most frequently observed scenario involved the prescription for women of selective Ã¡-blockers, including alfozosin hydrochloride, tamsulosin hydrochloride, and terazosin hydrochloride, that are indicated exclusively for the treatment of benign prostatic hyperplasia (BPH). All cases observed involved women diagnosed as having a lower urinary tract dysfunction during the visit in which these drugs were prescribed. Other cases included prescriptions for oral contraceptives written by a male physician for himself whose wife is not covered by the HMO, as well as oral contraceptives for a male regional manager. In view of these findings, the EPR system used in the HMO was programmed to block the prescription of sex-specific drugs for patients of the opposite sex for which they are intended. 

This study in a nationwide HMO demonstrates that the suspicion concerning illegitimate prescriptions for patients involving physician complicity was not unwarranted. The 141 cases involving men being prescribed vaginal preparations or oral contraceptives are sufficient to adequately substantiate this suspicion. The cases in which women with lower urinary tract conditions were prescribed drugs licensed exclusively for the treatment of BPH raise a significant policy issue for the HMO. Although there have been reports in the literature on the use of these drugs in women for these conditions,

these drugs have not been approved by the Israel Ministry of Health for indications other than BPH. Consequently, should a physician wish to prescribe these drugs for conditions other than BPH, special authorization must be requested on an individual basis justifying this off-label use. Accordingly, the HMO bears no obligation to cover treatment with these drugs in these women. Although the HMO faces dilemmas of this type on a daily basis, our experience has been generally limited to requests for off-label use of antineoplastic drugs for types of cancers for which they are not indicated. However, because cancer drugs are ordered and approved on an individual basis, no possibility exists that they will be dispensed for an off-label use without the proper authorization. This is not the case with the drugs for the treatment of BPH. Because these drugs can be freely prescribed and dispensed without gatekeeper intervention, off-label prescribing can go undetected. This issue is particularly pertinent because the selective Ã¡- blockers observed include expensive drugs that would not have been authorized unless the less expensive alternatives available were found to be ineffective or contraindicated in the specific patient. Although the observed incidence of inappropriate prescribing was low, the findings of this study should serve as a warning signal to the HMO that increased vigilance is warranted in monitoring the use of drugs with potential off-label abuses. 

Routine prescription monitoring using pharmacoepidemiological methods should be implemented in the HMO studied. Furthermore, the HMO medical division must formulate an official pharmacy policy that addresses the therapeutic and reimbursement issues raised by this study. 

We thank Raphael Cayyam, MD, for reviewing the manuscript before submission and for his helpful comments. 

From the Leumit Health Fund, Tel Aviv, Israel. 

Address correspondence to: Natan R. Kahan, RPh, MHA, Kupat Cholim Leumit, PO Box 3512, Petach Tikva, Israel 49134. E-mail: 

Prospective study comparing hyoscyamine, doxazosin, and combination therapy for the treatment of urgency and frequency in women. 

2. Kakizaki H, Ameda K, Kobayashi S, Tanaka H, Shibata T, Koyanagi T. 

Urodynamic effects of Ã¡-blocker tamsulosin on voiding dysfunction in patients with neurogenic bladder. 

Articles

CLINICAL

A Pharmacoepidemiological Approach to Investigating Inappropriate Physician Prescribing in a Managed Care Setting in Israel

Recently there has been tremendous growth in the number oflay-press articles and peer-reviewed journal articles reportingextraordinary improvements in health status and financial outcomesdue to disease management (DM) interventions. However,closer scrutiny of these reports reveals serious flaws in researchdesign and/or analysis, leaving many to question the veracity of theclaims. In recent years, there have been numerous contributions tothe literature on how to assess the quality of medical researchpapers. However, these guidelines focus primarily on randomizedcontrolled trials, with little attention given to the observationalstudy designs typically used in DM outcome studies. As such, generalguides to evaluating the medical literature are inadequate intheir utility to assist authors and readers of DM outcomes research.The purpose of this paper is to provide authors with a clear andcomprehensive guide to the reporting of DM outcomes, as well asto educate readers of the DM literature (both lay and peerreviewed) in how to assess the quality of the findings presented.

Until recently, disease management (DM) haslargely been able to avoid scrutiny of its methodsfor assessing effectiveness in attaining positivehealth and financial outcomes. Unfortunately, this hasled to the reporting of incredible achievements in thelay and industry press that have left many questioningthe veracity of these claims.

Similarly, DM outcome studies in peer-reviewed literaturehave reported extraordinary results, at times asa consequence of poor study design. Some basic issuesinclude the use of a pre-post study without a controlgroup, or the misguided application of a more robustdesign; not addressing biases that may threaten thevalidity of the results; inadequate description ofresearch methods or characteristics of the population;and inappropriate use or lack of statistical analysis.

Attempts have been made recently to address theseshortcomings publicly.

However, a more methodicalapproach to both designing and reviewing DM evaluationand research studies is needed.

In recent years there have been numerous contributionsto the literature on how to assess the quality ofmedical research papers, with perhaps the mostnotable among them being a series of articles appearingin the 

However, these guidelines focus primarilyon randomized controlled trials (RCTs), withlittle attention given to the observational study designstypically used in DM outcome studies. As such, generalguides to evaluating the medical literature are inadequatein their utility to assist authors and readers ofDM outcomes research. Therefore, the purpose of thispaper is to provide authors with a clear and comprehensiveguide to the reporting of DM outcomes, as wellas to educate readers of the DM literature (both lay andpeer reviewed) in how to assess the quality of the findingspresented.

Fundamentally, the objective of any thoughtfulcritique is to ascertain whether the reported resultsrepresent an unbiased estimate of a treatment effect, orwhether they were influenced by factors other than theintervention. To make this determination, one mustconsider 2 major elements in any evaluation or researchstudy; the 

presents a framework for assessingthe quality of the study design and analysis used inRCTs and observational studies. As shown, manymethodological issues overlap, while others are specificto the given design category. The items are ordered temporallyto coincide with each phase of the study.

The 2 predominant categories of study design relevantto DM research are 

(generally referred toas an observational study design).

The most basicdifference between these 2 categories lies in how subjectsare assigned to the study. As the name implies, inthe RCT, individuals are randomly assigned either to atreatment or a control group, thereby giving each personan equal probability to be chosen for the intervention.Conversely, in an observational study design,eligible individuals are not randomly assigned to thetreatment or thecontrol group. InDM, patients and/ortheir physicians arecommonly allowedto decide who willparticipate in programinterventions.This type of assignmentprocessresults in a nonrandomdistribution ofindividuals to theintervention andnoninterventiongroups.

The value of arandomized assignmentprocess is thatall variability is distributedequally betweenthe twogroups.

Variabilitycomes in 2 forms:observed and unobserved.Observedcovariates are characteristicsthat canbe measured by theanalyst via sourcessuch as claims, medicalrecords, memberfiles, or surveyreports; and unobservedcovariates areall other characteristicsnot captured orrecorded. Althoughobserved covariatesare used for ensuringthat subjects in the 2groups are similar onbaseline characteristics(eg, age, sex, diseasestatus), it is leftto the process of randomization to ensure that unobservedcharacteristics are similar in both groups as well.Observational study designs are susceptible to bias preciselybecause they cannot control for unobservedcovariates, and therefore cannot provide unbiased estimatesof treatment effect.

The launching point for any study regardless of designcategory (RCT or observational) is a definition of thestudy population. It is important that the individuals eligiblefor inclusion in the study be representative of thepopulation to which the findings will be applied. Forexample, many studies exclude women, the elderly, orindividuals with multiple illnesses.

This obviously limitsthe ability to generalize results across patients outsideof this study population.

In DM, programparticipants typically are not representative of the generalpopulation with the disease. By design, programadministrators target those patients who are either thesickest or at the highest risk of utilizing services.Therefore, it is important for the researcher and thereader of the DM outcomes literature to recognize thelimitations of generalizability of the study findings. Agood definition of the study population would include adescription of the inclusion/exclusion criteria, and clinical/demographic characteristics of both the treatmentand control groups.

The second attribute of study design to consider isthe process by which individuals engage in either thetreatment or nontreatment group. Strict adherence tothe assignment process is absolutely crucial in an RCT.As stated earlier, the basic tenet behind randomizationis that it distributes unobserved variation evenlybetween groups. Imagine if the assignment processallowed a patient's physician to determine study participation.Bias would be introduced if that physicianrelied on personal judgment to determine whether thepatient should or should not be included. Studies inwhich the process of random assignment was inadequatelydescribed or not described at all have beenshown to exaggerate the size of the observed treatmenteffect.

In observational studies (DM studies in particular),assignment is usually determined through self-selection.Individuals eligible for the study or programintervention are invited to participate. The factorsdetermining why a given individual chooses to participatewhile another individual does not are at the crux ofthe issue that differentiates RCTs from observationalstudies. It has been well demonstrated

that myriadfactors (eg, belief systems, enabling factors, perceivedneed) help explain why and how individuals accesshealthcare and perform health-related behaviors.Collecting as much information as possible on those eligibleindividuals who choose to participate as well as onthose who decline participation may assist theresearcher in identifying those differential characteristics.Similarly, unusual features of one group or anothermust be described for readers.

Assessing comparability between the study groupand the control group on baseline characteristics is thenext element of study design to consider. Baseline comparabilityof groups is an essential step in determining acausal link between study or program intervention andoutcome.

Most DM programs are currently being evaluatedusing a pre-post design with no control group. Themost basic limitation of this design is that without acontrol group for which comparisons of outcomes canbe made, several sources of bias and/or competingextraneous confounding factors offer plausible alternativeexplanations for any change from baseline.

Advocates of this approach argue that most threats tovalidity are nullified by using the entire population inthe analysis.

However, unless some basic factors arecontrolled for, such as case mix and turnover rate, biasstill remains a significant concern. Even with these controllingvariables in place, the pre-post method can beconfounded with environmental changes unrelated tothe DM program interventions.

Given these concerns, it is absolutely necessary todevelop a control group with which comparisons can bemade. Considering that DM programs or their payersare not likely to withhold potentially beneficial interventionsfrom eligible individuals by assigning them tothe control group, statistical methods can be used tomatch participants to historic controls.

That said,some studies have included control groups in their evaluations.

In both RCTs and observational studies, comparabilitycan only be assessed on observedcharacteristics. Therefore, it is extremely importantthat the research study include either a table or detaileddescription of demographic and clinical attributes of thetreatment and control groups. If cohorts differ onimportant observed baseline features, causal inferencesabout the program impact will be limited.

Determining whether an adequate number of individualswere included in the study is the next designfeature to review. Four interrelated parameters have aneffect on the conclusions that are attained from a typicalstatistical test

or the number of observations, subjects,or cases under study.

or alpha. This is the probabilitythat the observed result is due to chance alone.

or the probability that a difference will beobserved when it actually occurs.

This is the magnitude of change between2 groups or within 1 group, before and afterthe intervention.

Using this logic, the sample size of the study must besufficiently large to reduce the effect size necessary todemonstrate statistically significant findings. To put itsimply, studies that use large samples require a smallereffect size to show statistical significance. Thus, from anevaluation perspective, DM programs should strive toenroll as many participants as possible and identify anequal or larger number of controls.

Similarly, analyseson subgroups can be carried out only if their sample sizeis sufficiently large, irrespective of the overall studypopulation size. Study reports should identify the samplesize for treatment and control groups as well as forany subgroups analyzed. Significance levels for all findingsshould be clearly reported.

Study duration is the next important factor in DMprogram evaluations and is interrelated with 2 other elementsthat impact the validity of the findings:dose/response and loss to follow-up (attrition). It is generallyagreed that it takes at least 6 months after DMprogram commencement until behavioral changesbegin to take effect (dose/response). Therefore, significantchanges in healthcare utilization or monetary outcomesmay not be realized within the first year. Studiesreporting immense decreases in utilization and costs ina short-duration study (less than 1 year) must be viewedwith suspicion (especially if the study does not includea control group, or if the cohorts are not comparable atbaseline). The most likely bias in this scenario is regressionto the mean.

Attrition from a DM program via disenrollment inarguablyimpacts results negatively. Participants who donot achieve the maximum benefit from the intervention(eg, improved self-management of their disease,improved knowledge of how to access appropriatehealth services)

may continue to exhibit behaviorsthat run contrary to the program objective. Therefore,it is imperative that studies include a description of thepopulation that did not complete the prescribedlength/amount of treatment. Two methods that can beused to adjust for attrition are survival analysis

The next important, yet often overlooked, aspect ofDM program evaluations is the intervention itself. It ismostly assumed that the treatment is robust, and thatany change noted in the outcomes are causally linkedto that treatment. However, rarely is the interventiondescribed in enough detail to allow readers to decidefor themselves if there is sufficient evidence to drawthis conclusion.

Moreover, specific outcome measuresdirectly related to that intervention should beincluded. For example, if psychosocial models are usedto change health-related behaviors, then analysesshould be performed, and reported, to assess the relativechange in those behaviors. Without such information,the reader is left to question the causal impact ofthose interventions.

A treatment effect may or may not be evidenced,depending on the choice of outcomes. Most often in DMprogram evaluations medical cost is chosen as the primaryend point. However, cost is an ill-advised outcomevariable because it is influenced by changes in the unitcost of services, members'financial share of the medicalexpense, introduction of new technologies, and soforth—variables outside of the DM program intervention.

It is for this reason that disease-specific utilizationmeasures should be used as indicators of programsuccess.

Although rising costs may be due to manyuncontrolled-for variables, a decrease in utilization ismore likely evidence of a DM program's intervention. Bymeasuring the specific utilization variables that a DMprogram intends to impact directly, the evaluationshould draw the appropriate conclusions from the dataanalysis.

The final study design element for consideration inRCTs only is blinding the patient, provider, and analystto group assignment. Blinding alone eliminates theintroduction of several biases that may invalidate thestudy findings. Failure to use blinding in RCTs has beenshown to overstate treatment effects.

This issue is notrelevant to observational studies since researchers havelittle control over program participation.

The second major area related to the quality of evaluationor research findings is the rigor and applicabilityof the data analysis. Accuracy of data sources is the firstpoint of concern. Most DM programs rely on largeadministrative databases (medical claims and membershipfiles) for retrieving information on diagnostic measuresto identify suitable participants, baselinecharacteristics, quality indicators, and utilization andcost values. These data sources are notoriously inaccurate.

The influence of data inaccuracy on outcomescan be decisive. For example, in one study comparingthe ability to predict mortality after coronary arterybypass surgery, the predictive ability based on dataderived from medical records was significantly betterthan that based on administrative data.

Therefore, adescription of how validation of data accuracy wasaccomplished must be presented in studies that rely onadministrative data for any aspect of the researchendeavor.

Next, the group on which the analysis was performedshould be clearly identified. In RCTs, it is common toassess outcomes of all participants assigned to a givencohort, as opposed to evaluating outcomes only of thosewho received the treatment. The former is called theintent-to-treat (ITT) analysis, and the latter is referredto as a treatment-received (TR) analysis. The ITT analysispreserves the value of randomization (by equally distributingobserved and unobserved covariates betweenthe cohorts); however, causal inferences can be madeonly about the effects of being assigned to a given treatment,not receipt of that treatment. This method is usefulon a policy level, where forecasts of outcomes can bemade assuming the program will be implemented on alarge-scale basis.

In DM programs, individuals self-selectto participate in the program and thereby limitthe analysis to the TR method. Predictive risk-adjustedmodels should be used to improve the process by whichsuitable participants are identified, while establishing ameans to provide a more accurate description of eligibleindividuals. If the tool has high sensitivity (accuratelyidentifying people who meet the eligibility criteria) andspecificity (accurately identifying people who do notmeet the eligibility criteria), the researcher may feelmore confident in using the ITT method for comparingoutcomes between cohorts.

The final step in reviewing the soundness of a study'sdata analysis is consideration of the application of statistics.A comprehensive discussion about the types ofstatistical models and analyses required to evaluate programeffectiveness is unfortunately beyond the scope ofthis paper. However, 2 books, 

provide a good introduction to medical statisticsfor the interested reader with a basic understandingof the field.

In DM outcome studies,multiple regression analysis almost always is required toestimate the independent effect of covariates on theoutcome and to test whether the model provides additionalprognostic value. These models also form thebasis for most risk-adjustment tools. Two importantvariables that should be included in any comparativeanalysis are severity and case-mix adjustment.

Especially in pre-post designs, tracking the population'scase mix and severity level over the course of the studywill assist in determining whether the program had atreatment effect or whether population dynamics influencedoutcomes. Several diagnostic groupers can bereadily used for this purpose (eg, diagnosis-relatedgroups, ambulatory care groups), as well more simplemethods such as counts of comorbid conditions. Thesevariables should be included in the regression model asadjusters in the assessment of a treatment effect.

values and/or 95% confidence intervalsshould be stated for each outcome variable. While thisstatement may appear superfluous, many studies eitherdo not include any levels of significance, or they provideinexact measurements. For example, while the generalconsensus is to report significant 

values at <.05, severalstudies report values at 

<.10. This is potentiallymisleading to the inattentive reader, who may draw thewrong conclusions based on these values. Similarly,many journals require that researchers report exact 

values instead of "NS"(nonsignificant). This allowsreaders to decide for themselves how much stock to putin that actual value, as opposed to a predeterminationby the authors on their behalf.

Confidence intervals give an estimated range of valueswithin which the unknown population parametermay lie. Using the mean as an example, we can calculate,based on the sample data, an estimated range ofvalues within which we believe (with a given level ofconfidence) that the population mean may exist. Thewidth of the confidence interval generally gives us someinsight as to the accuracy of the estimate. A wide intervalmay indicate large variability in the dataset, or maybe a result of having a very small number of study participants.In cases where parametric statistics cannotprovide confidence intervals, bootstrapping is a viableand suggested option.

When the outcome variable isdichotomous (eg, yes/no, 0/1), the proportion of individualswith the outcome should be provided, alongwith the associated odds ratios. An additional measurethat can be used to assess the effect of introducing theintervention is called the "number needed to treat"(NNT)

The NNT provides an estimate of the number ofpatients that must be treated to prevent 1 adverse outcome.While not widely used, this may be a very suitablemeasure for assessing DM program effectiveness.

Conceptually, the basic premise of a sensitivityanalysis is that subjects in observational studies differfrom those in RCTs in their recruitment to the treatmentgroup. Although all individuals in a RCT have a50/50 chance of being assigned to the treatment group,observational studies are limited by self-selection bias.Sensitivity analysis therefore provides an estimate forhow far this bias must diverge from the 50/50 split of anRCT to raise concerns about the validity of the studyfindings (A. Linden, J. Adams, N. Roberts, unpublisheddata, 2004). Observational studies that fail to include asensitivity analysis inhibit the reader's ability to judgethe strength of the evidence that support a treatmenteffect.

The presentation of data analyses performed isessential to any research, whether it be an RCT orobservational study. Two basic tables should be commonplacein any paper. These are (1) a display of baselinecharacteristics of the groups under comparison and(2) outputs from statistical analyses, including modelparameters and estimates.

presents a modified table from an article byLinden et al

in which participants in a congestive-heart-failure program were compared with the entireunmanaged congestive-heart-failure population and witha control group matched on propensity score. Includedin the table are the major elements discussed in thispaper. Baseline characteristics are presented above thedotted line, and outcome measures are shown below it.Sample sizes are noted, as well as group means and standarderrors. 

values are noted for each pairwise comparison.Although this table is meant for illustrative purposesonly, it serves as a basic template for presenting comparisongroup characteristics in a clear and concise manner.

presents results from a Cox-regression survivalanalysis by Linden et al

in which age and sexappear to be significant predictors of hospitalization.Each unit increase in a patient's age was expected toincrease the risk of hospitalization by 2.6%, while beingfemale reduced the risk of hospitalization by nearly 8%.Also presented are 

values and 95% confidence intervals.Regardless of statistical model used in the dataanalysis, tables with a similar structure should be presentedto the reader.

This paper has provided in some detail a comprehensiveguide to the reporting of DM outcomes, includingimportant elements of both study design and dataanalysis. The information presented herein should beused as an educational tool to enable readers of the DMliterature to independently assess the quality of theresearch findings presented in the lay press and thepeer-reviewed literature. This guide also should be usedby DM researchers in developing DM evaluation plansand reporting findings. Raising the standards by whichDM program outcomes are evaluated should result inimproved quality of peer-reviewed and lay publicationson the subject, and the healthcare community's confidencein the veracity of these reports.

From Linden Consulting Group, Portland, Ore, and Oregon Health Science University,School of Medicine, Department of Preventive Health/Preventive Medicine, Portland, Ore(AL); and the Providence Health System, Portland, Ore (NP).

Address correspondence to: Ariel Linden, DrPH, President, Linden Consulting Group,6208 NE Chestnut St, Hillsboro, OR 97124. E-mail: ariellinden@yahoo.com.

1. Nurse-driven CHF program cuts hospitalization by 87%. 

MANAGERIAL

A User's Guide to the Disease Management Literature: Recommendations for Reporting and Assessing Program Outcomes

To demonstrate the potential effect of deploying asentinel system that scans administrative claims information andclinical data to detect and mitigate errors in care and deviationsfrom best medical practices.

Members (n = 39 462; age range, 12-64 years) of amidwestern managed care plan were randomly assigned to anintervention or a control group. The sentinel system was programmedwith more than 1000 decision rules that were capable ofgenerating clinical recommendations. Clinical recommendationstriggered for subjects in the intervention group were relayed totreating physicians, and those for the control group were deferredto study end.

Nine hundred eight clinical recommendations wereissued to the intervention group. Among those in both groups whotriggered recommendations, there were 19% fewer hospitaladmissions in the intervention group compared with the controlgroup (

< .001). Charges among those whose recommendationswere communicated were $77.91 per member per month (pmpm)lower and paid claims were $68.08 pmpm lower than among controlscompared with the baseline values (

= .003 for both). Paidclaims for the entire intervention group (with or without recommendations)were $8.07 pmpm lower than those for the entire controlgroup. In contrast, the intervention cost $1.00 pmpm,suggesting an 8-fold return on investment.

Ongoing use of a sentinel system to prompt clinicallyactionable, patient-specific alerts generated from administrativelyderived clinical data was associated with a reduction inhospitalization, medical costs, and morbidity.

Governmental and private initiatives in theUnited Kingdom and the United States

havecalled for expanded use of information technologyin healthcare and for the incorporation of clinicaldecision support systems

to detect and alert physiciansto potential medical errors and deviations from bestmedical practices. Previous authors have demonstratedthe value of decision support systems in guiding thechoice of antibiotic therapy

and in monitoring the useof potentially nephrotoxic drugs.

A review of 65 suchsystems demonstrated improved physician compliancewith treatment guidelines in two thirds of studies.

However, all previously reported decision support systemshave been deployed within a hospital setting orwithin an integrated delivery system in which electronichealth record systems provide the backbone of clinicalinformation.

We developed a clinical decision support system thatuses the clinical information contained in administrativeclaims data from physicians, hospitals, pharmacies,and clinical laboratories to identify common errors incare and departures from widely accepted clinicalguidelines. Therefore, the system can operate in anyfee-for-service clinical environment or other environmentin which encounter data are reported withoutrequiring any cooperation from the caregiver communitybeyond responding to clinical recommendations asthey are issued. Because of its ongoing vigilance in monitoringpatient information and spontaneously contactingtreating physicians, we applied the term 

to distinguish its function from point-of-caredecision support.

The claims-based system we developed is deployedin the care of approximately 4 million Americans whoare enrolled in health plans funded by large employers,managed care organizations, federal employee programs,Medicare managed care programs, and Medicaidprograms. In this report, we describe the results of a 12-month randomized prospective study to test thehypothesis that a claims-driven decision support systemcould increase compliance with evidence-based practices and effect improvements in patient outcomes asmeasured by decreased hospitalization and attendantcost. As subgroup analyses, we examined the causes ofhospitalization for any recommendation issued withsufficient frequency to yield adequate study power.

Classification of Diseases, Ninth Revision

The sentinel system was designed as a rule-basedartificial intelligence engine combined with an automaticmessage generator that conveys clinical recommendationsand supporting literature to treating physicians.Software coding was programmed in a combination ofC++ and Visual Basic (Microsoft Corporation, Redmond,Wash) that draws data from an Oracle 8i-based datawarehouse (Oracle Corporation, Redwood City, Calif).Daily data inputs include physician-generated insuranceclaims, hospital discharge and outpatient claims,laboratory claims and laboratory test results, and pharmacyclaims. Information is coded in 

codes, Logical Observation Identifier Names and Codes,and National Drug Code identifiers.

Candidate clinical issues for inclusion in the clinicalrules engine were identified from national multicenterclinical trials, guidelines published by the federal governmentor medical specialty societies, and Food andDrug Administration-approved pharmaceutical labelingby an in-house committee of physicians, working inconsultation with health plan medical directors and apanel of medical consultants. Typical issues targeted bythe rules engine include the following: (1) absence ofangiotensin-converting enzyme inhibitor therapy inpatients with congestive heart failure

and in those whomeet the Heart Outcomes Prevention Evaluation(HOPE) trial criteria,

(2) absence of &#946;-blocker use inpatients who have had myocardial infarction,

(3)absence of anticoagulation in patients with atrial fibrillationand structural heart disease,

and (4) absenceof documented laboratory monitoring in patients takingwarfarin sodium, glitazones,

and other medicationsthat require specific laboratory tests. Depending on theclinical urgency of the issue, notification may be byphysician-to-physician telephone contact, fax, or letter.

The criteria that trigger the engine to issue a specificrecommendation are multidimensional decisionmatrices that take into account all available informationon each patient. For instance, the criteria surroundingthe use of &#946;-blockers after myocardial infarctionattempt to identify known contraindications for &#946;-blocker therapy, such as severe bronchospasm andadvanced heart block in the absence of a pacemaker.Should any of those contraindications be identified, therecommendation is not issued.

The system contains more than 1000 decision matricesthat, when triggered, result in the transmission of acommunication to the treating physician, recommendingthat he or she consider a particular avenue of care.Because a claims-based system such as this will neverhave as much information about the patient as thetreating physician, all recommendations make clearthat the communication is merely for the physician'sconsideration and that there may be mitigating circumstancesthat might render the recommendation inappropriate.For this reason, such communications areinterchangeably referred to as care considerations.

The study was performed among the commerciallyinsured population of a university-affiliated managedcare plan, in which patients were free to choose theirprimary care physician and to move their care from oneprimary care physician to another. The protocol wasapproved by the human subjects committee of thehealth plan before inception and was in accord with theDeclaration of Helsinki with regard to protection ofhuman subjects. In the annual enrollment form, all planmembers provided written informed consent to theanalysis of their healthcare data and to the medicalmanagement and quality improvement activities of thehealth plan. Eligible subjects were notified by letter thatthe health plan was undertaking a quality improvementstudy and that they might be randomly allocated to aprogram in which their physicians might receive informationrelevant to their care and were given a telephonenumber to call if they wished not to be includedin such a study. To maximize the study power, interventionand control group members consisted of allhealth plan enrollees who were between the ages of 12and 64 years and had incurred at least 1 physician claimor 1 pharmacy claim in the 12 months before enrollment.Patients were assigned to an intervention or acontrol group, using an individually assigned randomnumber. Assignment occurred on a single date at studyinception. Neither patients nor treating physicians wereinformed of the allocation, although it is likely thatthose physicians who received communications aboutspecific patients surmised that those patients were partof the intervention group.

The study was designed to measure potential differencesin cost of care and to measure the extent to whichphysicians comply with clinical prompts issued in thismanner. The sample size required was calculated to havea power of greater than 80% (&#946;= .2) to detect a 5% differencein claims cost and compliance with recommendationsbetween the 2 groups at a type I error rate of .01(2-sided). We determined that subgroup analysis (withthe same study power) was feasible for any recommendationissued to at least 100 intervention and controlgroup subjects in which the adverse event rate in controlsexceeded 40%.

All analyses were performed on an intentionto-treat basis. For each group, individual members'claims were measured monthly during thestudy and expressed in terms of dollars permember per month (pmpm). All claims were alsobroken down into type of service, including inpatient,outpatient, professional services, and pharmacy.For each group, the totals and means werecalculated for total claims and claims by type ofservice. We also compared the costs for the interventiongroup participants who triggered careconsiderations vs controls who triggered butwere not issued care considerations.

To compare differences in total admissions andtotal days in the hospital, 2×2 tables were createdcontaining the numbers admitted and the numbersnot admitted in the intervention and controlgroups (as well as for days in the hospital and daysnot in the hospital) and were tested by &#967;

test.Differences in the rates of hospital admissions per1000 subjects were tested by 

test. Differences inmean charges and paid amounts in the interventionand control groups were tested by 

test,weighted for the number of months under observation.

< .02 was required for rejection of thenull hypothesis. As a safety measure, a maskedanalysis was conducted at 6months to assess whether therewas any difference (

< .05) inthe number of deaths betweenthe 2 groups.

A total of 39 462 subjectswere initially enrolled in thestudy (

). At baseline,no significant differences wereobserved in age or sex betweenthe intervention and controlgroup subjects (

).Eighty-four percent of control group members and 85%of intervention group members remained in the studyfor the entire 12 months (Figure 1). Approximately halfof the early withdrawals occurred in the first month,comprising individuals enrolled and randomized inDecember who chose another health plan for the studyyear. Therefore, 90% of control and intervention groupmembers who began the study remained under observationfor the entire study. During the study year, no differencein mortality was observed between theintervention group and the control group. The data availableto us identified hospital mortality, including deathon arrival to the hospital, and mortality documented bythe health plan.

Nine hundred eight clinical recommendations wereissued to the intervention group: 395 of those involvedthe recommendation to start a medication, 150entailed the suggestion to stop a medication, and 790suggested a test or procedure (

). Compliancewith care considerations that entailed starting a newdrug was measured by ascertaining the date the drugwas dispensed at the pharmacy. Physicians compliedwith 24% of these "add-a-drug"recommendations inthe intervention group. In the control group, physiciansspontaneously instituted the treatment thatwould have been recommended in 17% of instances inwhich the recommendation was triggered but notissued. This 42% relative difference in compliance wasstatistically significant (

= .007). The same clinicaltriggers that generated add-a-drug recommendationswere measured at baseline, and identical rates ofspontaneous resolution of those clinical situations inthe absence of intervention were noted in those subjectsultimately assigned to the intervention groupand to the control group (18% for both).

As seen in Table 2, more recommendations weretriggered for the intervention group than for the controlgroup. Although this difference was statisticallysignificant (

< .01), it was associated with recommendationsto obtain laboratory tests. This recommendationcategory is unrelated to differences in costor hospital utilization (

). We believe that themost likely explanation for this is that the care considerationengine was run weekly on the interventiongroup patients, whereas it was run at the end of thestudy on the control group patients. This was to avoidthe potential ethical problem of withholding actionableclinical information from control group participants.

Compliance with recommendations could notaccurately be assessed in the case of recommendationsto discontinue a medication, because discontinuancedoes not result in a medical order. Compliancewith recommendations for laboratory testing was notascertainable, in part because approximately onethird of laboratory testing, including all inpatient laboratorytesting, in this care system did not generate adiscrete claim.

Overall, there were 115 more hospital admissionsin the control group than in the intervention group(

). Most of this difference (96 admissions)occurred in the 5% of subjects who triggered recommendations.No meaningful differences were observedin the 95% of the population that did not trigger recommendations.Among all those triggering recommendationsin the intervention and control groups,19% fewer hospital admissions per 1000 subjectswere observed in the intervention group (

< .001).Because intervention group members who were admittedexperienced a slightly longer stay, the overall differencein hospital bed days per 1000 subjects wasonly 8% (

For total charges and paid claims, differencesbetween the intervention and control groups were notsignificant at baseline but became highly significantduring the study (

). Although charges in thestudy year compared to the baseline were higher inboth groups, charges in the intervention group were$18.62 pmpm (95% confidence limits, $12.00-$25.00pmpm) lower and paid claims were $8.07 pmpm (95%confidence limits, $5.00-$11.00 pmpm) lower. The differencein charges was observed for all categories ofcare except pharmacy claims, in which a small differencewas observed in favor of the control group (pharmacyclaims increased slightly for both groups frombaseline to the study year because of changes in reportingacross the health plan). This increase in observedpharmacy related charges is attributable to implementationof the add-a-drug recommendations. Multivariateanalyses that included baseline utilization as an independentvariable did not attenuate this effect.

Examination of Those Who Triggered Care Considerations.

Examination of the subgroup that triggeredcare considerations reveals a more striking differencebetween the intervention group and controls. 

demonstrates that a far greater financial differencebetween the intervention group and controls is observablein the subpopulation of those who triggered careconsiderations. Comparing the study year results withbaseline values, the intervention group patientsincurred $77.91 pmpm (95% confidence limits, $26.00-$130.00 pmpm) less in charges and $68.08 pmpm (95%confidence limits, $39.00-$98.00 pmpm) less in paidclaims than controls (

= .003 for both). There was nostatistically significant difference between interventionand control group subjects who did not trigger careconsiderations.

Examination by Type of Care Consideration

When differences in utilization are analyzed bytype of care consideration issued (Figure 2), the mostpronounced difference is associated with those patientswho triggered recommendations to add a specific medication.A smaller but meaningful difference was identifiablein patients who triggered recommendations tostop specific medications. No significant difference wasidentified in cost or utilization among patients who triggeredrecommendations for laboratory or diagnosticprocedures.

Analysis of Heart Outcomes Prevention Evaluation

The only clinical recommendationthat was issued with sufficient frequencyto enable subgroup analysis of outcomes was the recommendationto prescribe angiotensin-converting enzymeinhibitors in patients meeting the HOPE trial criteria.

Inthese 311 patients (156 intervention and 155 controlgroup subjects), a $510.00 pmpm difference in totalcharges was observed between the control and interventiongroups (date not shown), as well as a difference of 12hospital admissions per 100 patients (

Using a Claims Data-Based Sentinel System to Improve Compliance With Clinical Guidelines: Results of a Randomized Prospective Study

The Department of Veterans Affairs (VA) Pharmacy BenefitsManagement Strategic Healthcare Group (VA PBM) oversees theformulary for the entire VA system, which serves more than 4 millionveterans and provides more than 108 million prescriptions peryear. Since its establishment in 1995, the VA PBM has managedpharmaceuticals and pharmaceutical-related policies, includingdrug safety and efficacy evaluations, pharmacologic managementalgorithms, and criteria for drug use. These evidence-based practicespromote, optimize, and assist VA providers with the safe andappropriate use of pharmaceuticals while allowing for formularydecisions that can result in substantial cost savings. The VA PBMalso has utilized various contracting techniques to standardizegeneric agents as well as specific drugs and drug classes (eg, antihistamines,angiotensin-converting enzyme inhibitors, alphablockers,and 3-hydroxy-3-methylglutaryl coenzyme A reductaseinhibitors [statins]). These methods have enabled the VA to saveapproximately $1.5 billion since 1996 even as drug expenditurescontinued to rise from roughly $1 billion in fiscal year (FY) 1996 tomore than $3 billion in FY 2003. Furthermore, the VA PBM hasestablished an outcomes research section to undertake quality improvementand safety initiatives that ultimately monitor anddetermine the clinical impact of formulary decisions on the VA systemnationwide. The experiences of this pharmacy benefits program,including clinical and contracting processes/procedures andtheir impact on the VA healthcare system, are described.

Prescription drugs constitute a large percentage ofthe country's overall healthcare spending, as evidencedby double-digit growth rates in the last6 years.

The continued escalation in prescriptionconsumption and spending has gained the attention ofpolicymakers, health plan sponsors, healthcare professionals,and patients. Many US citizens now receivepharmaceuticals through some form of pharmacy benefitsmanagement to help achieve cost-effective and high-qualitypharmaceutical care. The Department of VeteransAffairs (VA), an organization dedicated to improving thehealth of our nation's veterans, finds itself at the vanguardof this movement as it has the responsibility toprovide all "needed"care to enrolled patients underPublic Law 104-262, the 

The term "needed"carerefers to "services that will promote, preserve, and restorehealth?including treatment, procedures, supplies,and services,"which encompasses providing and dispensingpharmaceuticals.

As a national healthcare organization, the VA offersa broad array of pharmacy benefits to its patient population(

). And, like many other healthcare organizations,the VA has seen its pharmaceutical budget risedramatically in recent years. For example, drug expendituresin fiscal year (FY) 2003 increased to more than$3 billion from $1.5 billion in 1999 (

), largelydriven by rising enrollment of new patients in the VAsystem. The VA began a national pharmacy benefitsmanagement program in 1995 to reduce geographicvariability of access to pharmaceuticals across the system,improve the distribution of pharmaceutical agents,promote appropriate drug therapy, and reduce inventorycarrying and drug acquisition costs. This articledescribes the cornerstones of this pharmacy benefitsprogram. We address the question, posed in 1996 bySchulman and colleagues, of how to evaluate pharmacybenefits managers by making explicit the experiences,processes, and procedures that form the core of theVA's Pharmacy Benefits Management Strategic HealthcareGroup (VA PBM).

Before 1995, 173 individual VA facilities managedtheir own pharmaceutical coverage policies via localpharmacy and therapeutics committees. The VA DrugProduct and Pharmaceuticals Management division,based in Hines, Illinois, managed and monitored drugusage and purchasing for those facilities, but had no utilizationoversight responsibilities. However, inSeptember 1995, Dr. Kenneth Kizer, the VA's formerUndersecretary for Health, established the VA PBM,directing it to establish a national formulary; managepharmaceutical costs, utilization, and related outcomes;and oversee pharmacologic guideline development ofcommon diseases within the VA system. To do so, clinicalpharmacists employed at the Drug Product andPharmaceuticals Management division headquarterscollaborated with a newly established consultative bodyof 11 field-based VA physicians and 1 physician fromthe Department of Defense. This group became the VAMedical Advisory Panel. This combined entity wouldlater evolve into the VA PBM.

Formulary decisions fall under the purview of 2groups: the VA PBM and the Veterans Integrated ServiceNetwork (VISN) formulary leaders. The VISN formularyleaders manage the pharmacy benefit for VA's 21regional care systems. Each VISN includes a variety ofhealthcare centers such as tertiary facilities, ambulatory-care centers, and associated community clinics.Each VISN's formulary committee collaborates with thepharmacy and therapeutics committees of its healthcarecenters to allow for integrative, multilevel decisionmaking. Thus, VISNs and local facilities can communicateand provide guidance to the VA PBM, and viceversa, on policies determining drug use within the VAsystem.

The VA PBM also collaborates with the VA NationalAcquisition Center for pharmaceutical contracting,establishes clinical guidelines for the VA andDepartment of Defense, develops criteria for use fornew pharmaceutical agents, and maintains a nationwidepharmacy database of patient/provider-specificprescription utilization information for research purposes.We discuss the history and issues in developmentof each of these below.

The VA PBM created a VA national formulary toaccomplish 2 main goals: to minimize interfacility variationin access to pharmaceuticalsand to useVA buying power toleverage contract pricesfor drugs. These accomplishmentsoccurred insteps. First, in 1996,each VISN established aregional formulary for itsfacilities, thereby reducingsome variation in thepharmacy benefits package.Then, in 1997, thefirst national formularywas established byassimilating these VISNformularies into a unifiedlist of drugs. Thisgreater uniformityallowed access to thesame VA pharmaceuticalbenefits nationwide.From a system standpoint,this standardizationnot only defined thecore national pharmacybenefits package, butalso provided leveragefor bulk purchasing, andwith that, contractingwithin drug classes whenappropriate. However, to offset concerns about overlycentralizing the process and thereby missing opportunitiesfor VISNs to tailor their formularies to local needs,the VA national formulary policy allowed for VISNs toadd (but not subtract) pharmaceuticals as needed tomatch the needs of local patients and resources. In July2001, the system further evolved to eliminate local drugformularies.

Currently, the VA maintains the VA national formularyand VISN formularies. VISN formularies may list additionaldrugs and dosage forms not found on the VAnational formulary, provided they do not conflict with theVA national formulary (ie, if a mandatory national standardizationcontract deems an entire VA national formularydrug class as "closed,"VISNs may not addnoncontracted drugs to the VISN formulary). In addition,a VISN may not add a new molecular entity approved bythe Food and Drug Administration (FDA) to its formularyuntil completion of a national review with specificauthorization to do so provided by the VA PBM and VISNformulary leaders. Access to nonformulary drugs occurslocally by either national, VISN, or local criteria.

Standardizing the VA national formulary evokedscrutiny at first, not praise. Veteran, commercial, clinical,and patient advocacy groups expressed concernsabout the VA's changes in formulary management. Aseries of hearings at the congressional level ensued, andin 1999 and 2000, the Senate and House Committees onVeteran Affairs each requested an outside review of theVA national formulary. The House requested an assessmentby the US General Accounting Office (GAO), andthe Senate by the National Academy of Sciences'Institute of Medicine. These assessments would determinethe clinical and economic integrity of the VAnational formulary and the formulary managementprocess. Areas of focus included access to formularyand nonformulary drugs, an assessment of the "restrictiveness"of the VA national formulary compared withother government and nongovernment formularies, andthe impact of the VA national formulary on the qualityof care.

The Institute of Medicine report, released in 2001,concluded that the VA national formulary was not"overly restrictive"with respect to "formulary size andquality, coverage of drugs in different classes, timelinessof new drug additions, fairness and responsiveness ofthe nonformulary exceptions process, and sensitivity oftherapeutic interchange policies and procedures."

Per the Institute of Medicine report, from July 1997through July 1999, only 0.4% (2385 out of 570 937) ofall complaints to patient representatives about the VAinvolved a pharmacy issue.

On the other hand, the Institute of Medicine foundinconsistencies in nonformulary use and in guidance forinstituting therapeutic interchanges. It also noted thelack of data, both within and external to the VA, measuringthe impact of changes in utilization on quality ofcare and patient satisfaction. Thus, the Institute ofMedicine recommended reducing inconsistenciesacross VISN formularies through more centralized managementand by establishing a more timely assessmentprocess for newly approved drugs. They further recommendedmore research initiatives to generate betterdata on the safety and efficacy of drug therapy representedon the VA national formulary.

The VA largelyagreed with the Institute of Medicine recommendationsand had already begun to take steps to improve suchshortcomings.

The second study, conducted by the GAO, noted thatVA national formulary drugs accounted for 90% of outpatientprescriptions dispensed between October 1999and March 2000 and that the VA national formulary metveterans'needs. However, the GAO reported that localformularies did not always include drugs listed on theVA national formulary, that sites sometimes did not usepreferred drugs, and that a wide variation existed in thenumber of drugs added to VISN formularies (rangingfrom 5 to 63 over a 3-year period, from June 1997 toMarch 2000). The GAO also observed variations in nonformularywaiver procedures and in the time needed toprocess such requests. For example, per prescriber selfreports,the GAO found average approval times rangingfrom minutes (22%) to a few hours (18%) to more than1 week (60%). They concluded that the VA national formularysystem would benefit by having mechanisms toensure uniform access to similar drugs across the system,VISN compliance with designated drugs, andimproved nonformulary drug approval processes.

In response to these reports, the VA PBM institutedseveral changes. First, the elimination of local facilityformularies helped reduce variance within individualVISNs. Thus, no additions or substitutions could occurwithout VISN approval. Second, to limit varianceacross VISNs in terms of adding new products, the VAPBM implemented a national review process for allnew molecular entities and any other drug deemedimportant for VA patient care. Third, to ensure a timelyreview of these new products, the VA PBM removedthe 1-year waiting period before consideration of anew product for formulary addition. Fourth, the VAPBM revised its nonformulary policy by requiringquarterly reports of nonformulary approval rates andaverage review time for such requests (with the goal ofa maximum 96-hour turnaround time for nonurgentrequests). Last, the VA PBM began to develop anexpanded research agenda to measure the effects ofthe VA national formulary on patient outcomes andsafety.

Standardization of pharmacy benefits through contractingallows for uniformity of generic and brandedproducts as well as price reductions due to bulk purchasing.Other benefits of standardization include easeof access and the ability to provide identical multisourcemedications from any VA pharmacy or consolidatedmail outpatient pharmacy, reducing the risk of patientconfusion resulting from different generic or brandedproducts. For instance, in the case of generics, facilitiesmay have dispensed different shapes, colors, and formulationsof pharmaceuticals depending on where patientssought care within the VA system. Moreover, pharmaceuticalstandardization improves drug inventoryprocesses and can reduce inventory management costs.

The most important and most contentious aspect ofcontracting lies in competitive bidding among brandedproducts within drug classes. Through competitive sourcing, and in collaboration with the National AcquisitionCenter, the VA PBM estimates that the VA achieved over$1.5 billion in total cost avoidance via national contractingefforts cumulatively from FY 1996 to FY 2003 (Figure1). This estimate has not been subjected to recent externalreview, though an earlier assessment by the Instituteof Medicine noted an approximate $100 million in savingsdue to closed and preferred drug classes from 1997through 1999.

The Institute of Medicine also found thatcontract awards generally reduced pricing by 16% to41%.

As an added benefit, renewable options for up to 4years have likely led to a reduction in the number of localtherapeutic switch programs, which occurred frequentlyin the past as companies tried to undercut each other'sprices in local VA markets.

Contracts have a compelling effect on adherence inthe VA. After awarding a national standardization contract,VISNs encourage providers to switch to the contractedagent if clinically appropriate. This, in turn, leadsto significant drug acquisition cost savings. As an example,after competitive bidding for proton pump inhibitorsin 2001, 95% of patients switched to the contractedagent within 6 months. Patients tolerated this switchwell, with only 5% needing another proton pumpinhibitor due to suboptimal response or intolerance.This therapeutic interchange generated more than $45million in cost savings in FY 2001. 

lists examplesof individual drugs/drug classes under contract withinthe VA from FY 2000 until FY 2003 and correspondingcost avoidances and expenditures (in-house data, thePharmacy Benefit Management Strategic HealthcareGroup).

The VA utilizes contracting techniques other thancompetitive bidding to reduce prices. First, the federalsupply schedule limits by law the price of brand nameproducts to 76% of the nonfederal average manufacturer'sprice. Secondly, the VA PBM works closely with theNational Acquisition Center to negotiate performance-basedincentive agreements (where the VA and themanufacturer directly negotiate a price reduction) andto establish generic multisource contracts. This hassaved at least $572 million from FY 1996 through FY2000 when comparing actual expenditures with estimatedexpenditures.

Through careful competitive bidding, performance-basedagreements, and other cost-saving measures (eg,promoting use of generic drugs when possible), theaverage acquisition cost per 30-day supply of drugs hasremained relatively steady (ranging between $13.00and $14.50 since FY 1999) compared with the averageyearly percent increase in VA pharmaceutical expendituresover the past 5 years (Figure 1 and 

). Drivers of drug costs in VA include increased numbersof users, drug cost inflation, utilization of new andmore expensive drugs, and an increase in the intensityof treatment for certain diseases and conditions (eg,diabetes, hypertension, hyperlipidemia). Despite thepresence of these cost-augmenting factors, the VA hasdemonstrated the ability to stay within its fixed budgetand direct savings to local VA medical centers for alternatepatient care functions. This opportunity providesincentive for local compliance with national contractingdecisions.

Competitive bidding and the subsequent closing ofselected drug classes can strain the VA's relationshipswith pharmaceutical manufacturers, who sometimesonly reluctantly accept the "all-or-nothing"concept ofdoing business across the VA. Indeed, since the competitivebidding program began in 1996, in nearly all of thesolicitations VA has attempted, 1 or more manufacturershave invoked their right under the federal acquisitionregulations to protest the solicitation, particularly if theyhave a sizable share of the market basket. The GAOadjudicates these protests and, to date, has ruled in favorof the VA in all cases. As protests can slow the process,the VA PBM takes this into account in the solicitation-planningstages. The VA has observed an increasingnumber of nuisance protests filed by manufacturers whoultimately choose to submit noncompetitive bids for VAcontracts. Additionally, some of these companiesattempt to delay or derail the contracting process by lobbyingformulary decision makers, patient advocacygroups, and congressional staff. Thus, legal and/or lobbyingactivity has become an expected occurrence in theVA's formulary management planning.

A primary objective addressed by the VA PBMentailed developing pharmacologic management guidelinesfor the most prevalent and most costly diseasestates observed in the VA population.

Over time, however,other entities within VA and the Department ofDefense have assumed these tasks, with VA PBM clinicians(physicians and pharmacists) taking on eithercentral or peripheral roles in this larger process. Sincethat transition has occurred, the VA PBM has focusedmore on evidence-based drug monographs for specificpharmaceuticals or drug classes and on developingappropriate criteria for use for various formulary andnonformulary agents.

These clinical documents andalgorithms of drug use rely on a consistent evidence-basedapproach that takes into account published clinicaltrials and careful cost-effectiveness analyses so thatwhen peer-reviewed by the Medical Advisory Panel,VISN formulary leaders, and field-based experts (therapeutic advisory groups), decisions do not depend ondrug acquisition costs alone and may even favor use ofhigher cost pharmaceuticals where evidence supportsbetter patient outcomes. The finalized drug and disease-state policies, together with utilization statisticsfrom its national prescription utilization database,assist the VA in advancing its national purchasingpower to contract for quality drug products at competitiveprices and to help ensure equal access to specificdrugs for specific conditions.

In 1996, Schulman et al stated that "pharmaceuticalbenefits managers rarely do original research on theclinical effectiveness of products before considering theproducts for inclusion in a formulary. These organizationscannot track the effect of their interventions onoverall health outcomes?and cannot link patients'medical claims data to prescription drug information."

The VA PBM recognized the need for formalassessment of its activities, and dedicated resourcesand personnel for this purpose. The VA has placed ahigh degree of importance on computer technology andcreating an integrated computerized medical record foreach patient. Most outcomes assessment addressesquality-improvement and patient safety initiativesusing a pharmacy database developed by the VA PBM.This patient/provider-specific database includes informationon all outpatient drugs dispensed from any VApharmacy and provides a detailed profile of medications,dosing, quantities, and drug costs. Prescriptionuse can be tracked on a macro (national, VISN, or facility)or micro (individual patient and provider) level.Further, when required, merging this data with largerVA administrative and clinical databases can providefurther information such as diagnosis, hospitalization,comorbidity, and laboratory data. In addition, the VAPBM utilizes ProClarity

&#8218;Professional data managementsoftware (version 5.0, ProClarity Corporation,Boise, Idaho) to create views of relational pharmacydatabases, allowing quick queries of data on selectedpharmaceuticals for questions requiring urgentresponses.

Within the VA, the VA PBM has spearheaded nationwidemonitoring and management of clinical pharmacyand pharmacy-related patient outcomes. The VA PBM'soutcomes research group designs formal research evaluationslooking at safety, appropriateness of use, effectiveness,and cost-effectiveness of prescription drugs inthe veteran population for retrospective, real-time, andprospective analyses. The VA PBM, along with other VAresearchers, have published several efforts in thisregard, as the following examples illustrate.

First, the VA PBM registry system tracks pharmaceuticalagents that have a suspected or identified safetyconcern. Several features of the VA provide auniquely powerful and sensitive laboratory environmentfor conducting postmarketing surveillance activities.The VA essentially mirrors a closed, staff-modelhealth maintenance organization that provides continuouscare for a large cohort of patients with significantburden of illness; members stay in the plan for extendedperiods of time (often decades); medication use ishigh, reflective of the sicker population; and centraltracking of all medications by the VA PBM allows it tofollow trends as well as track adverse outcomes.

Furthermore, quality-improvement projects such asdatabase reviews and appropriateness-of-use evaluationscapture and quantify the present state of drug interactions,drug switching patterns, inappropriate and indiscriminateuse of agents, drug-induced adverse events,treatment diagnoses, related diagnoses, medicationcompliance, and geographic variation in the utilizationof medications.

Several of the quality-improvementdatabase reviews began as patient safety initiatives.These and other projects formed the foundation for, andwill continue under, a new patient safety center ofinquiry known as the VA Center for Medication Safety, ajoint project funded for 3 years by the VA's NationalCenter for Patient Safety and the VA PBM.

As a corollary to its ability to monitor drug utilization,the VA PBM has formalized processes to improvepatient safety by utilizing educational tools and continuedmonitoring to ensure appropriate use. In additionto the standard educational tools, the VA PBM's Website (http://www.vapbm.org) improves access tonational directives on drug use, pharmacotherapeuticguidance, medication safety topics, and other pharmacotherapeuticinformation specific to the system.These resources increase the quality of care by directlycommunicating information to providers. This informationis updated on an as-needed basis as newinformation become available in published, peer-reviewedliterature.

For cases where concomitant therapy has resultedin adverse interactions and clinical sequelae, the VAPBM has taken the initiative to intervene at theprovider level by forwarding information letters to prescribers through the VISN formulary committees.Several interactions have been addressed, including (1)concomitant use of selective serotonin reuptakeinhibitors with codeine, where the former maydecrease the metabolism of codeine to morphine byinhibiting cytochrome P-450 2D6, thereby decreasingits therapeutic benefit

; (2) the potential interactionsbetween protease inhibitors and the hydroxymethylglutarylcoenzyme-A reductase inhibitors(statins) lovastatin and simvastatin, where proteaseinhibitors may inhibit statin metabolism, leading tohigher drug concentrations and, possibly, an increasedrisk for myopathy and rhabdomyolysis

; and (3) thecombination of statins and fibrates, which has thepotential to induce rhabdomyolysis and death.

Next, in monitoring new drugs used increasinglythroughout the VA system, the VA PBM conducted ananalysis on ziprasidone, an atypical antipsychotic,which can cause potential cardiovascular risks throughprolonged QTc intervals, especially in patients whohave cardiovascular disease, use concomitant drugsthat can prolong QTc, or take drugs that inhibit themetabolism of ziprasidone.

The evaluation trackedpatients with a diagnosis of schizophrenia and lookedat histories of cardiovascular disease, antipsychoticswitching patterns, discontinuation rates of ziprasidone,and concomitant use with contraindicated medications.The VA PBM also initiated a database reviewof oxycodone, a drug with abuse potential. Resultsprompted the VA PBM to develop national criteria foruse as well as an employee education program, and toinstitute an automated system to periodically reviewpatients who obtained a prescription for any formulationof oxycodone nationwide.

Moreover, the VA PBM conducts drug use evaluationsto analyze adverse drug events, assess compliancewith predetermined criteria and standards of practice,and estimate related pharmaceutical costs.

February 2005

A Pharmacoepidemiological Approach to Investigating Inappropriate Physician Prescribing in a Managed Care Setting in Israel

A User's Guide to the Disease Management Literature: Recommendations for Reporting and Assessing Program Outcomes

Using a Claims Data-Based Sentinel System to Improve Compliance With Clinical Guidelines: Results of a Randomized Prospective Study

Pharmacy Benefits Management in the Veterans Health Administration: 1995 to 2003

National Trends in the Provision of Smoking Cessation Aids Within the Veterans Health Administration

Sex Disparity in the Management of Dyslipidemia Among Patients With Type 2 Diabetes Mellitus in a Managed Care Organization