Thomas Delate, PhD

The purpose of this study was to identify and describe patient rationales for filling prescriptions at an out-of-plan pharmacy (OOPP).

A cross-sectional survey conducted in February 2013 among a random sample of 1000 patients.

Adult Kaiser Permanente Colorado (KPCO) patients who had a prescription electronically issued to an OOPP in November 2012 were surveyed. The study questionnaire was developed using items obtained from the literature and prepared de novo, as needed. The questionnaire included items regarding whether the electronic prescription issued to an OOPP was filled; if filled, which OOPP was used; factors that may have influenced the use of an OOPP; and the patient’s ability to afford medications. Responses to the survey were tabulated and reported as percentages.

The survey response rate was 38%. Respondents (N = 382) had a mean age of 61 years, 35% were males, and anti-hypertensives were their most common OOPP prescription. Overall, 330 (86%) respondents reported that they had their prescription filled at an OOPP. The most commonly reported OOPPs utilized were supermarket pharmacies (42%). Factors that influenced the decision to use an OOPP included the prescription being less expensive (58%), the OOPP had a discount generic prescription program (57%), and the OOPP’s location was convenient (44%). Thirty-nine percent of respondents reported that using an OOPP helped them afford their prescriptions.

Prescription cost and pharmacy convenience were identified as the most significant drivers of OOPP use. Future research should be conducted to assess the health-related consequences of OOPP use.

		The rate of prescriptions electronically issued to out-of-plan pharmacies (OOPPs) has risen considerably, but factors driving OOPP use are largely unknown.

		A random sample of patients who had a prescription electronically issued to an OOPP were surveyed.

		The most common factors reported that influenced the decision to use an OOPP included that the prescription was less expensive at the OOPP (58%), the OOPP had a discount generic prescription program (57%), and the OOPP’s location was convenient (44%).

		Out-of-plan pharmacy use may result in less expensive medications and increased convenience for patients; however, the risks to patients of using an OOPP are unclear.

Health plans, whether they are managed care organizations, accountable care organizations, or other types of care networks, often require their members to utilize in-plan pharmacies to receive subsidized prescription medications. Filling all prescriptions at the same pharmacy has been shown to decrease the risk for potentially harmful drug combinations.

 Plan members may also fill prescriptions at unsubsidized prices at out-of-plan pharmacies (OOPP).

 However, seeking prescription medications from multiple pharmacies, sometimes called “pharmacy shopping,” may contribute to medication-related problems, such as an increased risk for adverse drug interactions.

 The term “pharmacy shopping” suggests that this behavior is motivated by the goal of identifying the lowest medication cost. While actual patient-reported reasons for this OOPP use have not been well studied,

 some evidence does support medication cost as a factor. Discount generic prescription programs (DGPs) were introduced at several large chain pharmacies in 2006 and offer very lowcost generic prescription medications.

 Accordingly, the rate of prescriptions electronically issued to OOPPs has risen considerably since the introduction of DGPs.

 The influence of other factors driving OOPP use (eg, convenience, customer service, pharmacy loyalty) are largely unknown.

Information regarding the rationale for OOPP use is necessary in order to identify opportunities to improve patient safety, healthcare quality, and potentially member satisfaction. The objectives of this study were to describe if and where prescriptions electronically issued to an OOPP were filled, and the rationales for OOPP use among patients of an integrated health system with in-plan pharmacies.

This was a cross-sectional survey conducted in February 2013 among a random sample of 1000 patients from Kaiser Permanente Colorado (KPCO) who had a prescription electronically issued to an OOPP in November 2012. Kaiser Permanente Colorado is a group model, not-forprofit, integrated health system with more than 500,000 members in the Denver/Boulder metropolitan area that uses an electronic medical record (EMR) (Epic Clarity, Madison, WI) with e-prescribing capabilities. The e-prescribing system allows for the identification of whether a prescription was electronically issued for an in-plan pharmacy or OOPP. Data for this study were obtained from queries of the KPCO integrated, electronic administrative databases; the EMR; and a 12-item questionnaire mailed to the random sample of patients. This study was sponsored by the KPCO Pharmacy Department and all aspects were reviewed and approved by the KPCO Institutional Review Board prior to data collection.

Patients were eligible for inclusion in the survey if: (1) they were 18 years or older as of November 1, 2012; (2) they had a prescription electronically issued to an OOPP between November 1, 2012, and November 30, 2012; (3) their prescription was not for an over-the-counter, injectable, or infused medication or medical supply; (4) they had continuous KPCO membership for the 180 days prior to the prescription being electronically issued; (5) they were not residing in a skilled nursing facility at the time of the prescription; and (6) they were alive as of January 20, 2013.

A total of 12,827 prescriptions for 7841 patients were electronically issued for an OOPP in November 2012. Of these, 7531 (58.7%) prescriptions for 4177 (53.3%) patients were eligible for the survey. A random sample of 1000 eligible patients and their first eligible prescription electronically issued to an OOPP in the study month (heretofore termed the index prescription) was generated for the survey mailing.

The primary outcome was the group of patient-reported factors that influenced the use of an OOPP. Secondary outcomes included an assessment of patient-reported information on the OOPP used, patients’ ability to afford medications, and pharmacy services use behaviors. In addition, a comparison of the characteristics of responders and nonresponders was performed.

The study questionnaire was developed at KPCO using items obtained from the literature

 as needed to assess patient rationales for OOPP use. The questionnaire included information on the patient’s index prescription along with the name of the prescriber, whether the index prescription was filled or not, which OOPP was used, factors that may have influenced the use of an OOPP, ability to afford medications, and general health perceptions. The questionnaire items were assessed for interpretability (ie, reading level, jargon, etc), content validity, face validity, and reliability among a sample of KPCO pharmacy department staff. No outstanding issues were identified among the items.

The questionnaire and a cover letter explaining the purpose of the survey, along with a return envelope, were mailed via the US Postal Service to the random sample during the third week of January 2013. A reminder postcard was mailed approximately 2 weeks after the initial questionnaire. HealthCare Research, Inc (Denver, Colorado) conducted the survey and collected information from the returned questionnaires. Each questionnaire included a unique identifier to allow for the linking of the survey results with patient characteristics.

Patients’ demographic and health characteristics were identified using KPCO electronic administrative databases, including ambulatory visit, pharmacy use, membership databases, and the EMR. Administrative data collected included patient date of birth, KPCO membership status, sex, index prescription information (eg, index prescription drug class, in-plan medication purchases), socioeconomic demographics, and health plan information (eg, Medicare status).

As this was a descriptive study, no a priori sample size calculation was performed. The types of OOPP reported as used to purchase the index prescription were categorized as: (1) big box (eg, Walmart, Kmart); (2) discounter (eg, Sam’s Club, Costco); (3) chain (eg, CVS, Walgreens); (4) supermarket (eg, Krogers, Safeway); and (5) other (eg, independent pharmacy, Veterans Administration). Medicare status was categorized as Medicare beneficiary with a coverage gap, Medicare beneficiary without a coverage gap, or non-Medicare. Health plan type was categorized as traditional HMO or other (eg, preferred provider option, high deductible). Medications were classified into therapeutic drug classes based on their National Drug Code. A chronic disease score (CDS), a validated measure ranging from 0 to 35 with a higher score representing an increased burden of chronic disease,

was calculated from medication purchases during the 180 days prior to November 1, 2012. The count of in-plan prescription medication purchases during this time period was calculated. Patient race was categorized as white, other, or unknown/unreported.

Responses to the survey were tabulated and reported as percentages. Patient characteristics were reported as means/medians and percentages, as appropriate. Differences between respondents and nonrespondents were examined using c2 tests of association/Fisher’s exact test and t tests, as appropriate. A 2-sided alpha was set at 0.05.

A total of 4177 patients with a prescription electronically issued to an OOPP in November 2012 were eligible for this study. A random sample of 1000 of these patients were surveyed; 382 returned the questionnaire with usable data, for a survey response rate of 38.2%. Patients were stratified by survey response status into survey respondents and nonrespondents.

Survey respondents and nonrespondents were similar on sex distributions, count of previous prescription purchases at KPCO pharmacies, and socioeconomic indicators (all 

). However, respondents were more likely to be older and Medicare members and to have maintained plan membership during the 2 months following their index prescription order date (all 

 <.001). In addition, respondents were more likely to be white and less likely to be Hispanic than nonrespondents, and there were differences between groups across index prescription medication categories (all 

 <.05). The majority of respondents (n = 194, 51.5%) reported that they perceived their general health to be very good/excellent; this information was unobtainable for nonrespondents.

Overall, 330 (86.4%) respondents reported that they had purchased their index prescription at an OOPP (

). Their prescription being less expensive at the OOPP (n = 190, 57.6%), the OOPP having a discount generic prescription program (n = 188, 57%), and the OOPP’s convenient location (n = 145, 43.9%) were reported most frequently as factors influencing respondents’ decisions to choose their OOPP. In addition, 61 respondents (18.5%) reported that receiving extra incentives, such as fuel discounts, was a factor. The most commonly utilized OOPPs were supermarket (n = 140, 42.4%) and big box (n = 86, 26.1%) (

Almost one-fourth of respondents (n = 91) reported that they had used 2 or more OOPPs within the past year (Table 2). Less than 30% of respondents reported that they informed their Kaiser pharmacist (n = 95, 29.8%) or non-Kaiser OOPP pharmacist (n = 72, 22.6%) about their prescriptions filled at other pharmacies. Among respondents who reported using 2 or more OOPPs in the past year plus a Kaiser pharmacy (n = 77), 39% (30/77) reported informing their Kaiser pharmacist about their OOPP use.

While nearly a quarter (n = 87, 23.1%) of respondents reported that it was somewhat or very difficult for them to afford their prescriptions in 2012 (Table 2), 123 (39.4%) respondents indicated that using an OOPP helped them afford prescriptions they otherwise would not have been able to afford. Also among respondents who reported that it was somewhat or very difficult for them to afford their prescriptions in 2012 (n = 87), 69 (79.3%) reported that using their OOPP helped them afford their prescription(s). Among respondents who reported that their prescription, being less expensive at an OOPP, was a factor for using an OOPP (n = 190), 42.1% (80) reported that using their OOPP helped them afford their prescription(s).

In this survey of patients who had a prescription electronically issued to an OOPP in November 2012, we found that the most salient patient-reported factors given for using an OOPP were lower cost and convenience. These findings suggest that even patients with a defined pharmacy prescription drug benefit view prescription drugs as a commodity. Thus, “pharmacy shopping” may be a viable strategy for them to obtain prescription medications at a lowest cost. Most patients in our survey reported utilizing supermarket and big box retail pharmacies, reinforcing cost and convenience as drivers of OOPP use. Little information exists in the literature about why patients use an OOPP, and which one.

 Nevertheless, our findings do support previous research that identified convenience and inexpensive medications as factors that patients value.

Prescription cost makes sense as a commonly reported determinant of OOPP use, as this reinforces the theory that patients are concerned with prescription affordability.

Nevertheless, patients in our study reported that other factors, such as relationship(s) with a pharmacy’s staff and a pharmacy’s customer service also contribute to OOPP use. It is interesting to note that lower cost may not necessarily be equivalent to prescription affordability; while 57.6% of our respondents reported that their prescription was less expensive at an OOPP, approximately one-third of our respondents indicated that utilizing an OOPP helped them to afford prescriptions that they otherwise would not have been able to afford. This suggests that OOPP use among some patients is equivalent to buyers searching for the best bargain (ie, pharmacy shopping),

 not an inability to pay for medications.

Additionally, our study found that convenience is an important factor influencing OOPP use. Respondents reported that they most commonly utilized supermarket pharmacies for medications electronically issued to an OOPP. As convenience is typically advertised as one of the benefits of utilizing a supermarket, this finding is not surprising and is consistent with previously published data that indicate supermarket pharmacies are the most common choice for prescriptions transferred to an OOPP.

 Our findings suggest that convenience is crucial for health systems with in-plan pharmacies to consider if their current prescription delivery model is to stay competitive in the changing healthcare landscape. Future research should be undertaken to assess whether more prescriptions would be filled at in-plan pharmacies if more in-plan pharmacy locations and/or longer pharmacy operating hours were available.

Our study found that approximately 14% of patients reported that they did not fill their prescription electronically issued to an OOPP. This proportion is numerically lower than but similar to that reported by Fischer and  colleagues in their study of 3 Massachusetts health plans with e-prescribing capabilities who identified that 22% of electronically issued prescriptions were not filled.

These findings suggest another potential medication-related problem: failure to fill a prescription electronically issued to an OOPP. It is unclear if patients do not fill their prescription because they perceive medication costs as too expensive, or because they are not interested in obtaining the medication, or for some other reason(s).

Other results from our survey are disconcerting as well: approximately one-fifth of respondents reported that they inform their OOPP pharmacists about medications they receive at other pharmacies. This is troubling because in-plan EMRs and OOPP pharmacy dispensing systems generally do not share patient-specific information unless a health plan contracts with a pharmacy benefit manager (PBM) to perform this service.

Thus, the burden of ensuring communication across pharmacies about a patient’s other medication use can rest largely with the patient. The potential for adverse medication outcomes with this behavior is substantial (eg, drug-drug interactions).

It is unclear whether in-plan prescribers should discuss this gap in patient care when providing prescriptions electronically issued to an OOPP, ensure that their health plan has adequate drug utilization management coverage across pharmacies, or place the onus on patients to ensure that all of a patient’s medication information is communicated to each of the patient’s pharmacists. One low-tech option would be for prescribers to provide patients with a list of their current medications to share with their OOPP pharmacist.

Coupled with our current findings and previous work that revealed that approximately 5% of prescriptions are electronically issued to an OOPP,

 OOPP use appears to be an entrenched activity among a minority of patients. A retrospective analysis conducted in the Netherlands identified that approximately 11% of 45,805 patients from one insurance company were identified as OOPP users.

 Additionally, Gatwood and colleagues reported from their survey of 414 patients in a university-affiliated general medicine clinic that approximately 32% of respondents engaged in OOPP use.

 Therefore, it appears that different health systems have different rates of OOPP use, but that such use is ingrained.

Our study had several limitations. First, there was a moderate survey response rate. We attempted to encourage as many responses as possible by delaying survey administration until after the holiday season and utilizing a reminder postcard for nonrespondents. However, since we asked patients questions pertaining to their health-related behaviors that could be uncomfortable to answer and/or construed as their health plan checking up on them, it is unsurprising that a majority refused to participate. Second, patients may have felt compelled to respond in the affirmative (whatever they perceived the affirmative to have been), since it was a survey from their health plan. We attempted to mitigate the potential to answer in the affirmative by wording our items to be as nonjudgmental as possible. Third, this survey was conducted in a health plan with its own pharmacies. Patients from health plans that utilize a network of pharmacies may have different rationales for use of an OOPP. In addition, our study did not examine medication-related outcomes for patients utilizing an OOPP. Future research is essential to determine the impact of OOPP use on medication-related outcomes. If such research shows harm from OOPP use, strategies to recapture OOPP-using patients into in-plan pharmacies will need to be developed.

In this survey of patients who had a prescription electronically issued to an OOPP, we identified cost and convenience as the most significant drivers of OOPP use. While factors such as lost revenue, impact on healthcare quality, and the inability to track appropriate medication use would appear to be harmful from the health plan’s perspective, the risks to patients for using an OOPP are not clear. Future research should be conducted to assess the health-related consequences of OOPP use.

 From Kaiser Permanente Colorado, Pharmacy Department, Aurora, CO (TD, SJB), Greeley, CO (APB), and Englewood, CO (DK)

 This study was funded in its entirety by the Kaiser Permanente Colorado Pharmacy Department. Preliminary findings from this study were presented at the 2013 Western States Conference in San Diego, CA, as part of Alexander Block’s PGY2 pharmacy residency requirements.

 The authors report no conflicts of interest.

Concept and design (TD, APB, DK, SJB); acquisition of data (TD); analysis and interpretation of data (TD, APB, DK, SJB); drafting of the manuscript (TD, APB, DK); critical revision of the manuscript for important intellectual content (TD, APB, DK, SJB); statistical analysis (TD); provision of study materials or patients (TD, DK); obtaining funding (TD); administrative, technical, or logistic support (DK); and supervision (TD).

 Thomas Delate, PhD, 16601 E Centretech Pkwy, Aurora, CO 80011. E-mail: 

1. Tamblyn RM, McLeod PJ, Abrahamowicz M, Laprise R. Do too many cooks spoil the broth? multiple physician involvement in medical management of elderly patients and potentially inappropriate drug combinations. 

2. Gatwood J, Tungol A, Truong C, Kucukarslan SN, Erickson SR. Prevalence and predictors of utilization of community pharmacy generic drug discount programs. 

3. Buurma H, Bouvy ML, De Smet PA, Floor-Schreudering A, Leufkens HG, Egberts AC. Prevalence and determinants of pharmacy shopping behavior. 

4. Varallo FR, Capucho HC, Planeta CS, Mastroianni Pde C. Safety assessment of potentially inappropriate medications (PIM) use in older people and the factors associated with hospital admission. 

5. Delate T, Albrecht G, Olson K. Out-of-plan pharmacy utilization by members of a managed care organization. 

Articles

Prescription cost and pharmacy convenience were identified as the most significant drivers of out-of-plan pharmacy use.

Out-of-Plan Pharmacy Use: Insights Into Patient Behavior

To determine the positive predictive values of inpatient and outpatient 

 codes and status code V12.54 for identifying confirmed history of stroke or transient ischemic attack (cerebral event) among patients within a managed care organization.

Inpatient hospital claims and outpatient visit records were used to identify patients with

codes (430.XX to 438.XX) or status code V12.54 in the primary or secondary position recorded between January 1, 2001, and December 31, 2009. A standardized chart abstraction tool was used by trained chart abstractors blinded to the coding to confirm the cerebral event and classify stroke type. Positive predictive values (PPVs) were calculated for each code based on care setting.

 A total of 4689 patients with 10,376 unique stroke codes recorded in the administrative data were reviewed. Of these, 2785 (59.4%) patients had a confirmed cerebral event. The codes with PPV less than 90% were 434.XX, 433 .X1, and V12.54 where codes were recorded in both the inpatient and outpatient settings. Overall, inpatient-only codes produced higher PPVs; however, relatively fewer events were captured in this setting.

codes 434.XX, 433.X1, and V12.54 had consistently high PPVs in identifying patients with a confirmed cerebral event. These codes could be used as part of a probabilistic approach to focus care activities on patients with the highest likelihood of a cerebral event.

		 An ideal cerebrovascular ICD-9 code would correctly identify a large number of patients with true cerebral events (ie, maintain a high positive predictive value [PPV]).

		Our results found that several stroke codes (433.X1, 434, and V12.54) had relatively high PPVs. n Incorporating diagnoses and health status administrative codes from the outpatient setting may improve the ability to identify patients new to a health plan with a confirmed stroke or transient ischemic attack (TIA).

		Using these codes to identify patients with true events may be a means to establish quality improvement programs and focus care activities on patients with prior strokes or TIAs.

		Individuals should be cautioned that even the codes with high PPVs may still result in false positives in up to 11% of cases.

Stroke is the fourth-leading cause of death, the most common neurologic disorder, and a leading cause of long-term disability in the United States.

 Despite widely available national clinical practice treatment guidelines for ischemic stroke, studies demonstrate that a significant proportion of these patients have uncontrolled modifiable risk factors and remain undertreated.

 Barriers to implementing treatment guidelines include patient, provider, and healthcare system factors. Developing validated disease registries may allow better tracking and management of patients with stroke.

International Classification of Diseases, Ninth Revision

) diagnosis codes, can be a powerful tool for identifying eligible patients for disease state registries. Healthcare organizations have developed stroke registries to address treatment gaps and improve quality of care, and support surveillance, epidemiologic and economic research for patients with ischemic stroke.

 diagnosis codes may not reliably identify patients with a confirmed stroke.

Studies evaluating the accuracy of cerebrovascular 

 codes have relied primarily on patients discharged following emergency department (ED) visits or hospitalizations.

 These data may not be available for patients transitioning to a new healthcare system. Patients new to a healthcare system and/or patients who were never treated in the ED or hospital could be seen in the outpatient setting where either self-report or clinical diagnosis of cerebrovascular disease is made. Thus, incorporating diagnoses and health status administrative codes from the outpatient setting may improve the ability to identify patients with a confirmed stroke or transient ischemic attack (TIA). The purpose of this study was to determine the positive predictive value (PPV) of cerebrovascular 

codes using data from both inpatient and outpatient care settings to capture patients, both incident and prevalent, with cerebral events including stroke or TIA. The information from this study may be useful for healthcare systems to develop and/or improve stroke registries.

This retrospective study was conducted at Kaiser Permanente Colorado (KPCO), an integrated healthcare delivery system providing services to more than 500,000 members at 24 medical offices in the Denver-Boulder metropolitan area. At KPCO, all providers utilize an electronic medical record (EMR), in which all office visit, vital sign, imaging, laboratory, hospital discharge summaries, pharmacy, etc, data are housed. An EMR has been used at KPCO since 1998. In addition to the provider interface, the EMR has a data repository that can be queried electronically to extract relevant data. The majority of KPCO patients receive prescription medications from in-house pharmacies for a co-payment. This study was approved by the KPCO Institutional Review Board with a waiver of informed consent.

All patients at least 18 years of age with at least 1 inpatient stay or outpatient medical office visit with an ICD-9 diagnosis code between 430.xx and 438.xx or status code V12.54 (Personal History of Stroke) (Appendix) in the primary or secondary position recorded between January 1, 2001, and December 31, 2009, were identified administratively from inpatient hospital claim databases and outpatient visit records from the EMR data repository. Medical information (eg, provider visit notes, hospital discharge summaries) in the EMR was available from January 1, 1998, onward. In some cases, scanned medical information was available prior to 1998.

While there is some disagreement for using all diagnosis codes versus limiting to primary and secondary positions,

 we used codes from both positions to maximize the yield of patients with which to examine for true strokes. Patients were categorized by the first 3 digits of the identified diagnosis code (eg, a patient with a diagnosis of 433.10 would be categorized as 433) or the status code alone (ie, V12.54) and the setting type where the code was recorded (inpatient only, outpatient only, or the same code recorded in both care settings). All of a patient’s cerebrovascular codes identified in the administrative data during the study period were captured; thus, a patient could have had more than 1 code category assigned in the inpatient and/or outpatient settings. There were no prespecified exclusion criteria.

Information from each eligible patient’s EMR was considered the standard for determination of a confirmed stroke or TIA. An abstraction tool using elements from the Rochester Minnesota Stroke study form

 was developed with input from KPCO neurologists. The tool was used to identify patients with a confirmed incident or prevalent stroke or TIA (cerebral event), identify the cerebral event date, and, if a confirmed stroke was found, classify the stroke type (ischemic, hemorrhagic, or unknown). For purposes of this manuscript, all cerebral events (including TIAs) are referred to as a stroke unless specifically describing/discussing TIAs. Identifying a confirmed stroke required sufficient clinical and/or radiologic evidence of diagnosis (focal neurological symptoms and/or computed tomography [CT] scan/magnetic resonance imaging [MRI] verifying ischemia, infarct, or hemorrhage). Recognizing the difficulty in objectively diagnosing TIA, a confirmed TIA was defined as a coded diagnosis with written confirmation by physician(s) in hospital progress/discharge note(s) or medical office note(s).

Given that secondary prevention management of TIA is similar to that of ischemic stroke,

 TIA was included with the ischemic stroke classification. Intracerebral and subarachnoid bleeding was classified as hemorrhagic stroke. The “unknown” classification was used when chart review revealed clinical evidence of stroke, but insufficient clinical or radiologic evidence was available to establish stroke type. Trained clinical pharmacy specialist reviewers used the abstraction tool and all available information in the EMR to determine whether patients had confirmed stroke and to classify stroke type. Reviewers were blinded to administrative 

 code(s). A kappa score was calculated for a random sample of 20 patients with the result of 0.8 indicating a high degree of inter-rater reliability for stroke confirmation.

 Additionally, the study investigators met with the reviewers to review the discrepancies and agree on how to code certain cases moving forward.

Prescriptions for antiplatelet medications (ie, ticlopidine [Ticlid], clopidogrel [Plavix], dipyridamole [Persantine], and dipyridamole/ ASA [Aggrenox]), use of imaging (CT and/or MRI scans), and outpatient neurology department visits within 180 days of initial cerebrovascular 

 code(s) were identified using queries of KPCO-integrated electronic databases. These stroke-related health service use indicators were used in conjunction with the study 

 codes to determine if the PPVs of administrative data to identify strokes could be improved.

 code were determined by dividing the number of confirmed stroke events by the total number of events recorded in the administrative data for the specified code. The primary study outcome was the PPV with 95% confidence intervals (CIs) for correctly identifying confirmed cerebral events.

 code in the setting where the code was recorded (ie, inpatient setting only, outpatient setting only, or the same code recorded in both settings). Secondary outcomes included assessments of PPV for identifying confirmed hemorrhagic and ischemic strokes, respectively for each hemorrhagic (430, 431, 432) and ischemic (433, 434, 435, 436, 437, 438, V12.54) code. Positive predictive values for these outcomes were determined by dividing the number of specific confirmed events (ischemic or hemorrhagic) by the total number of events recorded in the administrative data for the specified code. Subanalyses were performed to determine the robustness of PPV estimates under various scenarios.

 codes in conjunction with health service indicators was determined by calculating PPVs for each 

 code individually and in combination with various health service indicators. To accomplish this, only patients who were exposed to the health service indicator and had the 

 code diagnosis were used to calculate the PPV. To assess the impact of 

code position (primary or secondary), secondary position inpatient codes and patients with only a secondary position inpatient code were removed and PPVs recalculated. To assess the impact of including TIA, all patients with confirmed TIA were removed and PPVs recalculated. To assess the impact of the 

code 436 not being inclusive of stroke, because the code excludes “cerebrovascular accident (CVA) NOS, Stroke” as of October 1, 2004, the PPVs of code 436 recorded before and on/after this date were recalculated. To assess the impact of the use of codes with specific “infarction” terms, codes 433.XX and 434.XX were categorized individually by codes that contain the infarction term (ie, 433.X1 and 434.X1) or not and the PPVs were recalculated. Positive predictive values, were determined with SAS version 9.1.3 (SAS Institute Inc, Cary, North Carolina) using Proc Freq with weighting by the count of patients having a specific diagnosis and the exact binomial function to determine 95% CIs.

Patient characteristics were reported as means with standard deviations for interval-level characteristics. These characteristics were assessed for distribution normality and appropriate tests (eg, t test, rank-sum test) were used to assess differences between groups. To assess differences in proportions between groups on dichotomous characteristics, Pearson’s x

 test of association was utilized. A 2-sided alpha level was set at <.05.

A total of 4689 patients with 10,376 unique study administrative 

 codes were reviewed. Of these, 2785 (59.4%) patients had a cerebral event confirmed by EMR review. The majority of patients had 

 codes from the outpatient setting (82.6%) while 1.3% and 16.1% were from inpatient and both outpatient and inpatient settings, respectively (

). The most commonly identified cerebral event types were non-cardioembolic strokes (34.8%) and TIAs (31.1%). Cerebral event type was unknown in 15.4% of cases. Patients with confirmed cerebral events had a higher mean count of unique ICD-9 codes, were slightly older, more likely to have purchased a prescription antiplatelet drug, and more likely to have had CT or MRI imaging. Positive predictive values for “intracerebral hemorrhage”( 431), “acute but ill-defined cerebrovascular disease” (436), and “personal history of stroke” (V12.54) were greater than 90% when recorded in both the inpatient and outpatient settings but identified small numbers of patients; thus, associated 95% CIs were wide (

). “Occlusion of cerebral arteries” (434) recorded in either the inpatient-only or in both the inpatient and outpatient settings also achieved PPVs greater than 90% and identified large numbers of patients. Codes 434 and V12.54 in the outpatient-only setting identified the most patients and had reasonably high PPVs (both 89%). “Other and unspecified intracranial hemorrhage” (432) and “other and ill-defined cerebrovascular disease” (437) performed poorly regardless of setting (PPVs <50%). Overall, codes recorded in both the inpatient and outpatient settings yielded the highest PPVs but identified fewer patients compared with those recorded only in the inpatient or outpatient settings. Hemorrhagic stroke codes identified fewer patients than ischemic stroke codes (

). Hemorrhagic stroke codes recorded only in the inpatient setting had higher PPVs than outpatient-only codes. Overall, ischemic codes tended to have higher PPVs than hemorrhagic codes (

). Approximately 15% of code 436 patients had their code recorded on/after October 1, 2004, and the vast majority of these (99%) were in the outpatient setting. Nevertheless, there was no appreciable change in PPV before and after 436’s coding modification. Code 433.X1 with specific mention of infarction did increase the PPV appreciably but only approximately 8% of patients with a 433.XX code had a 433.X1 code. Code 434.X1 with specific mention of infarction did not alter the PPV appreciably over presence of the code 434.XX (Table 2).

In general, inclusion of health service indicators and exclusion of secondary inpatient diagnoses and TIA patients identified fewer patients, widened 95% CIs and did not change the PPVs substantially. Overall, inclusion of diagnostic imaging did not change PPV estimates, nor did inclusion of neurology visits. Inclusion of prescription antiplatelet exposure slightly improved PPVs (mean difference in PPVs across care settings = 1.5 [± 4.2], median difference in PPVs across care settings = 1.0). However, for codes 434 and V12.54, inclusion of prescription antiplatelet exposure resulted in more marked improvement in PPV (mean = 8.6 [± 9.7], median = 4). Removing secondary position inpatient codes did not affect the PPVs appreciably (mean = —0.5 [± 3.0] and median = 0), but removing patients with a confirmed TIA (code 435) slightly reduced PPVs overall (mean = –2.0 [± 2.5] and median = –1).

Disease registries provide opportunities for health systems to improve management of patients with chronic disease states. Initial patient identification using coded administrative data is an important part of developing a validated patient registry, particularly if the codes have high PPV. Using a standardized abstraction tool adapted from the Rochester Minnesota Stroke study form,

 we found only 60% of patients identified from administrative data using cerebrovascular ICD-9 codes had a confirmed cerebral event. We found that the settings where ICD-9 codes were recorded influenced both the accuracy of diagnosis and yield of identified cases. Codes recorded in both inpatient and outpatient settings had higher absolute PPVs, but identified fewer patients than codes recorded in only 1 of these settings. Attempts to improve the accuracy of 

 codes through various combinations with health services indicators produced, at best, only moderate improvements, with the exception of combining purchases of prescription antiplatelets with codes 434 and V12.54. The incorporation of the setting where the 

 codes were recorded and using combinations health service indicators are unique aspects of our study. Nevertheless, in most cases, our PPV estimates were associated with wide 95% CIs despite a relatively large sample size suggesting that administratively coded data elements may lack sufficient accuracy to be relied on without confirmation of cerebral events via medical record review.

Several inpatient studies have evaluated the accuracy of 

 codes for identifying ischemic stroke by assessing the sensitivity, specificity, and/or PPV of 

These studies also demonstrated less than optimal accuracy of 

codes in identifying confirmed stroke patients. One study reported a PPV of only 47% for 

 codes between 430 and 438 for correctly identifying incident stroke events.

 Additionally, several studies have revealed that registries derived from hospital discharge codes overestimate stroke.

codes recorded in inpatient and/or outpatient settings appears to modestly improve the accuracy of identifying confirmed cerebral events, the use of 

codes alone appears to lead to a high percentage of false-positive diagnoses, such that about 40% of events identified with these commonly used 

 codes are not confirmed cerebral events.

Benesch and colleagues found that limiting inpatient 

 codes to those listed in the primary discharge position increased stroke PPV.

 We hoped to increase event capture by using

 codes recorded in the primary or secondary discharge positions. Nevertheless, we also performed a subanalysis using only the primary position and found that removing secondary position codes did not decrease our inpatient PPVs. Our results may have differed from their study since we had relatively low rates of false positive strokes.

We reported PPV estimates for codes recorded only in the outpatient setting, capturing patients who may or may not have been hospitalized for treatment prior to enrolling in our health plan. We found that considerably more patients were identified in the outpatient setting. Similar to prior studies, we found that ICD-9 codes 434 and 436 had high PPVs for patients with confirmed ischemic stroke.

We were able to show that the PPVs for these codes slightly improved when the codes were recorded in both the inpatient and outpatient settings (97% and 93%, respectively). Since October 1, 2004, when 436 coding changes were implemented, the number of patients with this code decreased considerably, making the utility of this code to identify patients with stroke less robust.

code 435 (TIA) because treatment guidelines are similar for patients who experienced an ischemic stroke and TIA is an important risk factor for stroke, with 90-day risks reported as high as 17%.2 Interestingly, we found that code 435 was one of the most prevalent codes, capturing 1329 patients who had their cerebral event confirmed, and its PPVs were relatively high across settings (>84%) indicating potential utility for this code if identification of patients with probable TIA is desirable.

 codes for hemorrhagic stroke have reported higher PPVs than ischemic stroke codes.

 However, we found that, overall, hemorrhagic stroke codes had lower PPVs than ischemic stroke codes, perhaps because there were fewer patients with hemorrhagic stroke codes. We did find that inpatient hemorrhagic codes had higher PPVs than outpatient hemorrhagic codes suggesting that inpatient hemorrhagic events are coded more accurately.

 codes 432 “other and nonspecified intracranial hemorrhage” and 437 “other and illdefined cerebrovascular disease” performed poorly based on their definitions. For example 437 includes a variety of conditions under the umbrella of “ill defined,” including cerebral atherosclerosis, chronic cerebral ischemia, hypertensive encephalopathy, non-ruptured cerebral aneurysm, moyamoya disease, nonpyogenic thrombosis of intracranial venous sinus, and transient global amnesia.

Overall, the inclusion of health service indicators such as diagnostic imaging and neurology visits did not change appreciably the PPV estimates, nor did evaluating code 434.X1 with specific infarction terms. Code 433.X1 with specific mention of infarction did increase the PPV appreciably but identified only a small number of patients. Inclusion of prescription antiplatelet exposure slightly improved PPVs. We identified no other studies that employed health service indicators to assess their effects on PPVs.

code would capture a large number of patients and maintain a high PPV. We found that codes 434 and V12.54 came closest to ideal when recorded in the outpatient setting. Code 434.XX is likely highly predictive because the operating definition is specific to thrombosis or embolism of cerebral arteries.

 The high predictability of V12.54 is interesting because it is a patient self-reported code. To our knowledge, this is the first study to evaluate the accuracy of 

 status code V12.54, which proved to be highly effective capturing many patients and moderately predictive across the inpatient and outpatient settings. We recommend that health plans wishing to identify administratively patients with a history of stroke or TIA utilize 

 codes 433.X1, 434.XX, 436.XX, and V12.54. These codes demonstrated reasonable predictive abilities across care settings. Conversely, we caution plans from utilizing administratively codes with a low PPV as this may result in a large number of false positives.

Our study was not without limitations. We recognize that PPVs are subject to disease prevalence, however, we were unable to determine sensitivities and specificities as we did not identify or review patients who did not have the specified stroke codes. Given the large number of patients with stroke codes reviewed for this study, the number of non-cases we would have had to review to provide meaningful data would have been too resource intensive. We used a manual chart review process that relied on the accuracy of care and health services use of documentation in the EMR. There may also have been temporal changes in coding practices. The study was conducted at only 1 health plan and coding practices and accuracy may vary across institutions and health plans; thus, our findings may not be applicable at all institutions/ health plans. Similarly, coding and charting practices may have differed between individual physicians, whereby some physicians were more accurate and/or comprehensive.

While we have no evidence of differential coding and charting, we enhanced our chart review process by using a standardized chart abstraction tool to improve consistency and trained chart reviewers who had an inter-rater reliability consistent with a high level of agreement. Additionally, the duration of a member’s enrollment in KPCO may have impacted our ability to confirm strokes. Patients with longer enrollments would have had a greater opportunity to be diagnosed in multiple care settings and/or had medical information to confirm stroke.

This study determined that ICD-9 codes 433.X1, 434.XX, and V12.54 had positive predictive values >90%, and may be used to focus care on stroke patients.

Positive Predictive Values of ICD-9 Codes to Identify Patients With Stroke or TIA

: To evaluate whether patients with coronary artery disease (CAD) discharged from the Clinical Pharmacy Cardiac Risk Service (CPCRS) would maintain their lipid goals with use of an electronic laboratory reminder system.

: A 2-year, randomized study at Kaiser Permanente Colorado. 

: Patients with prior CAD (acute myocardial infarction, coronary artery bypass graft surgery, percutaneous coronary intervention) who had been enrolled in the CPCRS for at least 1 year and who had 2 consecutive low-density lipoprotein cholesterol (LDL-C), non—high-density lipoprotein cholesterol, and blood pressure readings at goal within 6 months before enrollment were randomized to remain in the CPCRS (CPCRS care) or to receive usual care from primary care physicians plus laboratory reminder letters (usual care). The primary outcome was maintenance of LDL-C goal at study end. The 

 test and χ2 test of association were used to assess differences in mean and categorical values, respectively. 

: A total of 421 patients (214 CPCRS care, 207 usual care) were randomized. Their mean age was 72 years; 74% were male. After 1.7 years of follow-up, the proportions of patients maintaining their LDL-C goal of <100 mg/dL were 91% and 93.1% in the CPCRS care and usual care groups, respectively (

 = .46). The proportions maintaining their LDL-C goal of <70 mg/dL were 68.6% and 56.8% in the CPCRS care and usual care groups, respectively (

: This study demonstrated that LDL-C measures can remain controlled in most patients discharged from a cardiac disease management program.

Patients with coronary artery disease can maintain their lipid levels after discharge from a disease management program.

Disease management programs for patients with heart disease improve processes of care and risk factor profiles for the patients enrolled; however, they are resource intensive.

Patients can maintain lipid levels after discharge from such programs through the use of electronic laboratory reminder letters.

Such a system of follow-up will allow disease management programs to provide care to other high-risk populations without the need for additional resources.

Treatment of elevated low-density lipoprotein cholesterol (LDL-C) is part of standard care for patients with coronary artery disease (CAD). National treatment guidelines recommend an LDL-C goal of <100 mg/dL for all patients with CAD and a goal of <70 mg/dL for those at particularly high risk.

 In addition, the National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (HEDIS), a tool used by the majority of health plans to measure performance on various dimensions of care and service, includes the proportion of patients with cardiovascular conditions who have LDL-C values of <100 mg/dL as part of the performance measures.

Statins are the mainstay treatment option for lowering LDL-C and have been shown to improve clinical outcomes and survival among patients with CAD.

 As a result of this and other treatment strategies, there has been a decrease in both the numbers of deaths and the rate of death from CAD.

 However, real-world data consistently demonstrate that a significant portion of patients with CAD are not screened, initiated on lipid-lowering therapy, or treated to recommended LDL-C goals.

 Although the reasons for undertreatment have not been fully elucidated, healthcare provider, patient, and healthcare system factors likely all contribute.

In an effort to improve treatment patterns, multidisciplinary cardiac disease management programs have been established.

 A meta-analysis of randomized controlled trials demonstrated that such programs improve processes of care, risk factor profiles including lipid and blood pressure levels, appropriate use of secondary prevention medications, and quality of life, and result in fewer hospitalizations for the patients enrolled.

 The majority of studies evaluating these programs have focused their evaluations on a prespecified enrollment time in a program, but have not investigated the sustainability of a program’s interventions over time once patients are discharged. Thus, few studies have explored the maintenance of treatment goals after discharge from such programs.

Given limited resources within healthcare systems, care delivery must be focused and efficient in order to provide care to the greatest number of high-risk patients and to be able to expand care to other patients who could derive benefit, such as those with diabetes, peripheral arterial disease, or cerebrovascular disease. However, methods to maintain the gains achieved within cardiac disease management programs need to be evaluated because such services are resource intensive. The purpose of this study was to evaluate whether patients with CAD who were at their LDL-C goal would maintain lipid control after discharge from a cardiac disease management program.

This was a 2-year, randomized controlled study conducted at Kaiser Permanente Colorado (KPCO). Kaiser Permanente Colorado is a group-model, closed-panel, nonprofit health maintenance organization that provides integrated healthcare services to more than 480,000 members at 18 medical offices in the Denver-Boulder metropolitan area. Kaiser Permanente Colorado utilizes electronic medical records that contain all office visit, vital, laboratory, and pharmacy data. Approval to conduct the study with a waiver of informed consent was obtained from the KPCO Institutional Review Board.

In 1998, KPCO established the Clinical Pharmacy Cardiac Risk Service (CPCRS), a centralized, clinical pharmacy specialist—managed, physician-directed disease management program that utilizes a systems-based approach to focus on the long-term management of more than 12,000 patients with CAD.

 All KPCO patients with a history of acute myocardial infarction, percutaneous coronary interventions, or coronary artery bypass graft surgery are enrolled into the program through direct referral by primary care physicians, referral from a registered nurse—managed cardiac rehabilitation program, or administrative queries of 

 codes. The CPCRS staff work closely with primary care providers, cardiologists, and nurses to ensure that lipids are controlled, evidence-based secondary prevention treatment strategies are initiated and adhered to, and follow-up laboratory tests are completed. The CPCRS focuses on patient follow-up to ensure that appropriate lipidlowering, antihypertensive, and smoking cessation medications are initiated and titrated, as necessary. Patient care is delivered primarily over the telephone.

All interventions by the CPCRS are made using collaborative drug therapy management protocols for lipid, hypertension, and laboratory management. With regard to fasting lipid profiles (FLPs), the CPCRS reorders laboratory tests approximately 8 to 12 weeks after any changes to lipid-lowering medications or therapeutic lifestyle changes and does so until the patients have achieved their LDL-C goal. Once patients are at their goal, FLPs are ordered every 6 months until at least 2 consecutive LDL-C values are at goal, at which time patients are monitored yearly. With regard to blood pressure, the specialists review blood pressure at the time FLP results are available and adjust medications accordingly. The patient’s primary care physician is notified of any medication changes via the electronic medical record.

All patients are entered into a Web-based electronic database (HealthTrac), which is integrated with the KPCO’s administrative, laboratory, pharmacy, diagnosis/procedure, vital sign, and demographic data. The HealthTrac database is used to identify patients due for laboratory tests in order to send reminder letters and minimize missed laboratory appointments. The CPCRS has achieved high participation rates (>90% of the entire CAD population at KPCO). The service also has achieved top-tier HEDIS scores, with 98% of the population screened and 82.8% with LDL-C of <100 mg/dL, and has demonstrated improvements in both surrogate and clinical outcomes of the patients enrolled.

 Prior to this investigation, patients were monitored by the CPCRS indefinitely or until they terminated their health plan membership with KPCO.

Patients identified for inclusion in the study were those who required only yearly follow-up per the CPCRS treatment protocol. Patients had to have at least 2 consecutive LDL-C and non—high-density lipoprotein cholesterol (non–HDLC) values at goal; the most recent value had to have been recorded within 6 months prior to study enrollment. There was no prespecified required time frame between the 2 FLPs for eligibility. The LDL-C and non–HDL-C goals were defined as <100 mg/dL and <130 mg/dL, respectively, for all patients except those with diabetes, multivessel coronary disease, at least 1 recurrent coronary event, or current smokers. For these patients, the LDL-C and non–HDL-C goals were <70 mg/dL and <100 mg/dL, respectively.

Patients also were required to have controlled blood pressure within 6 months prior to study enrollment. The blood pressure goal was defined as <140/90 mm Hg for all patients except for those patients with diabetes or chronic kidney disease, for which the goal was <130/80 mm Hg.

 Patients were excluded if they had a glycosylated hemoglobin measurement of ≥9% within the previous 6 months, had dementia, died within 30 days of randomization, or had a life expectancy of less than 3 years.

All patients eligible for study enrollment were identified administratively using the HealthTrac database and randomized on September 1, 2005. Eligible patients were randomly assigned in a 1:1 fashion using a computer-generated list of random numbers to either continue to receive CPCRS care or to be discharged from the CPCRS to receive usual care from the primary care team plus laboratory reminder letters (usual care).

Patients randomized to CPCRS care continued to receive the standard care delivered by the CPCRS for all CAD patients in the program. This care consisted of clinical pharmacy specialists telephoning patients to review the results of their annual FLP, blood pressure measurements, and medications and adherence; counseling on diet and exercise regimens; and making medication adjustments to maintain treatment goals. The specialists ordered follow-up laboratory tests through the electronic medical record. Additionally, all patients were mailed letters informing them of their results, were scheduled for a follow-up FLP, and received laboratory reminder letters. Each patient contact was documented in the electronic medical record for other healthcare providers to review and was estimated to take the clinical pharmacy specialist approximately 15 minutes.

Patients randomized to usual care were discharged from the CPCRS at the time of study enrollment (September 1, 2005), and an FLP was ordered for 1 year in the future with results to be returned electronically to each patient’s primary care physician. It was the responsibility of the physician to address the results as they became available and order follow-up laboratory tests, as needed, during the study period. Patients in the usual care group received laboratory reminder letters that were generated electronically using the HealthTrac database. However, unlike the CPCRS care group, patients in the usual care group received no contact or counseling by the CPCRS specialists or letters informing them of their FLP or blood pressure results. Other than the aforementioned reminder letters, physicians received no additional clinical assistance from CPCRS specialists with respect to the usual care patients. In rare instances, physicians were able to contact the CPCRS with specific questions regarding patients with significant medication side effects. Physicians did, however, have access to regional guidelines for the treatment of CAD risk factors.

Patients were followed for 2 years (until August 31, 2007) or until the date of termination of KPCO membership, a recurrent cardiovascular event, or death, whichever came first (study end). The primary outcome measure was the percentage of patients maintaining the LDL-C goal at study end. A number of secondary outcome measures were evaluated: the occurrence of fatal/nonfatal coronary events, all-cause mortality, and any-cause hospitalization over the 2-year study period between groups. Coronary events were defined as acute myocardial infarction, percutaneous coronary intervention, and/or coronary artery bypass graft. In addition, blood pressure control, maintenance of non—HDL-C goal, and persistence with lipid-lowering therapy were assessed at study end. The lipid and blood pressure values assessed for the outcomes were those reported most recently but no later than 1 year prior to the study end date. Persistence was confirmed if a patient had a day’s supply of lipid-lowering therapy, based on the fill date and supply, which overlapped the study end date.

Based on dichotomizing the LDL-C goal and assuming that 99% of the CPCRS care group and 90% of the usual care group would remain at LDL-C goal at follow-up, a total of 150 study patients per group (300 study subjects) were required to provide a power of 80% and a 2-sided alpha of .05. To account for study subjects who terminated KPCO membership over the 2-year study period (estimated at approximately 20%), the sample size was increased to 400 patients.

All statistical analyses were performed on an intention-totreat basis utilizing SAS statistical software (SAS, Cary, NC). Baseline characteristics were reported as means, medians, and standard deviations for interval-level and ratio-level variables (eg, age, length of time between events, mean lipid levels) and proportions for nominal-level and ordinal-level data (eg, sex, cardiovascular event history). Interval-level outcome variables were assessed for normality of their distributions, and appropriate tests (eg, 

 test, rank-sum test) were used to assess differences in mean values between groups. To assess differences in proportions between groups on categorical variables, the Pearson 

 test of association was utilized. Patients were excluded from an analysis if they had missing data in the follow-up period. The alpha level for statistical significance was set at .05.

A total of 421 patients were randomized; 207 to usual care and 214 to CPCRS care. The reasons for study exclusion are detailed in the Figure. There were no differences between the groups at baseline (

). The mean age of the population enrolled was 72 years and 74% were male. The majority of patients had a diagnosis of hypertension and a history of acute myocardial infarction. The majority of patients (76%) had an LDL-C goal of <100 mg/dL and 89% had a blood pressure goal of <140/90 mm Hg. The mean time of enrollment in the CPCRS prior to randomization was 4 years. The majority of patients (79%) were followed for the entire 2-year study duration; however, 8.4% terminated membership in KPCO (average of 9 months after randomization) and 12.6% died or had a recurrent cardiovascular event (average of 12 months after randomization) before the study end. The mean duration of follow-up was 1.8 years in the CPCRS care group and 1.6 years in the usual care group (

. The overall percentage of patients with at least 1 FLP performed during the study follow-up was 92.8% (94.3% CPCRS care vs 91.3% in usual care; 

 = .21). The percentage of patients who maintained their LDL-C goal at study end was 85%. The percentages of patients who maintained their LDL-C goal of <100 mg/dL at study end were 91% and 93.1% in the CPCRS care and usual care groups, respectively (

 = .46). The percentages of patients who maintained their LDL-C goal of <70 mg/dL at study end were 68.6% and 56.8% in the CPCRS care and usual care groups, respectively (

 = .23). There were no differences between groups in maintenance of the non—HDL-C goal at study end (88.7% CPCRS care and 88.2% usual care; 

Overall, 75.7% of patients maintained their blood pressure goal by study end. A higher proportion of patients in the usual care group maintained a blood pressure goal of <140/90 mm Hg compared with those in the CPCRS care group (75% CPCRS care vs 84.2% usual care; 

 = .03). Among those with a blood pressure goal of <130/80 mm Hg, there were no differences between groups in the proportion that maintained their goal (60% CPCRS care vs 54.5% usual care; 

There were few coronary events during the study period. A total of 3.3% of patients in CPCRS care compared with 5.8% in usual care experienced a fatal/nonfatal coronary event during the study period (

 = .21). Similarly, there were no differences in all-cause mortality (8.4% CPCRS care vs 7.7% usual care; 

 = .80) or any-cause hospitalization (33.6% CPCRS care vs 27.1% usual care; 

Lipid levels can remain controlled with electronic laboratory reminder letters for most patients with coronary artery disease discharged from a cardiovascular disease management program.

Thomas Delate, PhD

Articles

Out-of-Plan Pharmacy Use: Insights Into Patient Behavior

Positive Predictive Values of ICD-9 Codes to Identify Patients With Stroke or TIA

Outcomes of Patients Discharged From Pharmacy-Managed Cardiovascular Disease Management

Clinical and Financial Outcomes Associated With a Proton Pump Inhibitor Prior-Authorization Program in a Medicaid Population