The American Journal of Managed Care November 2008 - Special Issue
Systematic Analysis of Outcomes Evaluations Applied to Drug Management Programs
Objective: To evaluate prior analyses that quantify how drug management programs impact health plans and patients with respect to those analyses’ effectiveness in measuring program quality.
Methods: We examined 77 US studies from 1996 to 2007 that evaluated the respective drug management programs of managed care, indemnity insurance, Veterans Health Administration, and Medicaid health plans. Our review included those studies that assessed the ways in which specific drug management tools and the drug management program overall impacted economic, clinical, and/or humanistic outcomes.
Results: The 77 studies included in our review used 11 types of outcome end points. A total of 52 studies (68%) incorporated an economic end point; of these, 35 (68%) reported economic data only and did not address clinical or humanistic outcomes. Overall, 33 (43%) evaluated clinical or humanistic end points; of these, 17 (52%) also reported on an economic end point.
Conclusions: Although the number of studies evaluating drug management programs has trended upward, only a handful have integrated economic, clinical, and/or humanistic outcomes when assessing the effects these programs have on health plans and patients. The efforts of these researchers to assess the overall quality of drug management programs have fallen short. To ensure that drug management tools have a desired effect on outcomes and medical costs, measures used to evaluate drug management programs must be improved.
(Am J Manag Care. 2008;14(11 Spec No.):SP36-SP45)
Few studies designed to evaluate the impact of drug management programs have adequately assessed their quality, and more comprehensive assessments are needed to ensure the following:
- Drug management program tools improve or remain neutral to patient outcomes.
- Drug management tools do not reduce pharmacy costs at the expense of the patient’s well-being or with indifference to resultant medical costs.
- Health plans use drug management tools that deliver the most value (best health outcomes) per dollar spent.
Before quality can be measured, however, it must be defined. Genichi Taguchi was one of the earliest promoters for improving quality in manufacturing. He defined poor quality as “the loss a product imposes on society after it is shipped.”1 If we apply Taguchi’s definition to healthcare, poor quality might be described as the loss an intervention imposes on society after it is implemented. Although the manufacturing industry now has procedures for quality assessment firmly in place, measuring the quality of healthcare services—including prescription drug programs—and pinpointing a precise definition of “quality” appear much more elusive. The Institute of Medicine has defined quality in healthcare as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”2
The US healthcare industry has made progress in developing measures to evaluate medical care delivery and patient outcomes for individuals with private and public health insurance. National Committee for Quality Assurance accreditation and quality reporting (based on the Healthcare Effectiveness Data and Information Set) and the Joint Commission on Accreditation of Healthcare Organizations performance measurement initiatives are quality measurement programs3 that specify disease treatment and management measures, and incorporate appropriate indices for prescribing medication. Neither program outlines specific criteria for assessing drug management programs. The following organizations have made independent efforts to develop and implement quality measures for pharmaceutical care, broadly and in the payer/managed care setting:
• The Academy of Managed Care Pharmacy Formulary Submission Format designed an evidence-based, effectiveness-driven approach to formulary management.4Although these programs make a good beginning, none is comprehensive and each restricts itself to a particular stakeholder perspective. There is no consensus among the programs as to what should be measured, how it should be measured, and what outcomes should constitute thresholds for different quality measures. This lack of consensus has led to significant variations in data and results. Without standardized quality measures and public reports on drug management programs’ performance, it is difficult for stakeholders to assess or compare quality claims presented in the literature or by health plans.
• The National Quality Forum is developing voluntary standards for reporting the quality of therapeutic drug management.5
• The National Business Coalition on Health eValue program is used by business coalitions to compare health plans across several performance measures, including pharmacy programs.6
• The URAC Pharmacy Benefit Management Accreditation Format has established a benchmark for the quality of care in pharmacy benefit management.7
In addition to analyzing the methodologies used in the literature to quantify the impact of drug management programs, we wanted to offer insight as to whether these initiatives help ensure high quality overall (eg, the best possible outcomes for health plans and patients). This may help managed care organizations and other stakeholders in the United States improve the processes for measuring quality, better determine the effectiveness of drug management programs in the United States, and provide a framework for improving quality.
We searched the PubMed, International Pharmaceutical Abstracts, CINAHL, and Business Source Primer databases for relevant English-language articles published in 1996-2007. We also reviewed meeting abstracts from Value in Health, the official journal of the International Society for Pharmacoeconomics and Outcomes Research, and from the Academy of Managed Care Pharmacy. We used search terms relating to health insurance, managed care, pharmaceuticals, drug management tools (ie, formulary and step-therapy edit), and indicators of quality measurement (eg, outcomes). In our effort to identify relevant articles, the search terms listed in the eAppendix Table (available at www.ajmc.com) were applied singly and in combination. Once articles had been obtained, we reviewed the reference lists from key studies and related review articles to find any articles not captured during the automated search process.
A pharmacist or pharmacoeconomics researcher screened the titles and abstracts of all relevant articles—which included original research studies, meta-analyses, and systematic reviews—to identify those studies that appeared to satisfy criteria for inclusion. Researchers then evaluated each identified article in full to determine whether it met the parameters of our study.
We included only those studies conducted in the United States in a managed care, indemnity insurance, Veterans Health Administration, or Medicaid setting. For inclusion, a study was required to have evaluated the effects on outcomes of 1 or more drug management tools (eg, formulary restrictions, preferred drug list, prior authorization procedures). The study also had to have assessed the drug management program’s overall impact on 1 or more of the following end points: economic (monetary value or consumption), clinical (medical and/or clinical indicators), or humanistic (using patients’ assessments of quality of life [QOL], satisfaction, or outcomes). If a reviewer was uncertain whether a study should be included, a second reviewer assessed the article, and together they decided whether to include the study. We then systematically analyzed selected articles according to our main study objective, examining the study’s data sources, analyses of cost and outcome parameters, and drug management type.
Applying the search terms listed in the eAppendix Table, we identified 1099 potentially relevant articles. After careful screening to exclude studies that did not meet inclusion criteria, we identified 77 articles suitable for this review. The Table details each of the 77 articles included in our analysis.8-84 Research on the effects of drug management programs evolved over the study period. Only 5 (6.5%) of the 77 studies we reviewed that evaluated drug management programs were published before 2001; after 2001, the number of these types of studies increased dramatically, as did the breadth of end points considered. Although only 1 study published in 1996-1999 met our inclusion criteria, there were 4 qualifying studies published in 2000-2001, 15 in 2002-2003, 31 in 2004-2005, and 26 in 2006-2007.
The studies we reviewed relied on data from 1 or more of the following sources: pharmacy claims, medical service claims, electronic medical records (EMRs), modeling exercises, surveys, and published literature. Most studies used administrative data for their analyses. More than two-thirds of the studies (70.9%) used pharmacy claims, making these claims the most common source of data. Medical claims constituted the second most widely used data source, used by nearly half (41.8%) of the studies. In 17 studies (22.1%) survey data were used, and 4 studies (5.1%) used EMR data. A similar number of studies used data published in the literature to conduct meta-analysis (n = 6) or outcomes modeling (n = 7).
Primary Drug Management Tools Evaluated
The studies we examined measured the effects of 1 or more of 10 drug management tool types on various end points. Cumulatively, the studies encompassed the following 10 drug management tool types: formulary restrictions, preferred drug list, use of copayments or cost-sharing measures, prior authorization procedures, quantity level limits, step-therapy edits, therapeutic substitutions, generic drug programs, pharmacy cognitive services, and over-the-counter drug coverage (Figure 1). A majority of the studies (77%) evaluated only 1 drug management tool type, with cost sharing being the most common (n = 39; 50.6%), followed by formulary restrictions (n = 27; 35.1%).
Study End Points
Study end points used in the 77 articles we reviewed were assigned to 1 of 11 categories (Figure 2). Very few studies evaluated both patient-centered and economic outcomes. One study considered overall medical costs as an end point.23 A total of 52 studies (68%) focused on economic end points related to prescription drug costs or use. Of these 52 studies, 35 (67%) did not address clinical or humanistic outcomes. Overall, 33 studies (43%) evaluated patient-centered end points, of which approximately half (n = 17; 52%) also included an economic end point.
Of the 19 studies related to medication adherence or continuation, 11 (58%) also considered economic end points; the same was true for 5 of the 7 studies (71%) that examined clinical or psychological outcomes and 2 of the 3 studies (67%) that focused on guideline adherence. None of the studies that analyzed medication switching or provider workload considered both patient-centered and economic outcomes; only 1 of 12 studies (8%) that examined patient-reported satisfaction/outcomes also considered a cost or use end point.
On average, only 2 to 4 articles per 2-year period considered both economic and patient-centered end points (Figure 3). Despite the rapid growth in assessments of drug management programs after 2001, 26 of the articles published in 2003-2006 considered only patient-centered end points and 22 considered only economic ones; only 6 considered a combination of patient-centered and economic end points.
W. Edwards Deming, one of the founders of quality theories, surmised that “Better quality at lower price has a chance to capture a market. Cutting costs without improvement of quality is futile.”85 Operating on this tenet, researchers must strive to prove that healthcare interventions positively impact the quality of healthcare in addition to measuring how they affect cost and consumption. Otherwise, improved health outcomes might come at unacceptable costs or, conversely, economic savings might be achieved at the expense of a patient’s health.
Limitations to Previous Analyses
To determine how effective previous studies have been at assessing the quality of drug management programs, we evaluated the methodologies they used to quantify the effects of drug management programs or tools on health plans and patients. We found that many existing studies lacked a consistent methodologic approach and used a limited scope or perspective—they considered either economic outcomes or patient-centered outcomes but not both. As such, they fell short in providing a true assessment of quality.