To describe the implementation and impact of a centralized statin switch program at a large academic medical center.

Patients on atorvastatin were identified from electronic medical records and pharmacy claims data. Relevant information was sent to physicians for approval of the proposed switches. Approved patients were then contacted via phone and offered the opportunity to switch to simvastatin; those who switched received a new prescription for simvastatin. To assess the independent impact of the active switch process, conversion rates within a single insurance plan were compared for patients who participated in this program versus those who were contacted only by mail.

Physicians approved 3207 of the 3677 patients identified for this program. A total of 1710 approved patients accepted the switch, 704 declined, and 170 became ineligible. Information packets were mailed to 623 patients who could not be contacted. Within the single insurance plan, the generic dispensing rate for statins among the 1867 patients included in our program was significantly higher than that for the 2472 patients in the mail-only group (59.2% vs 35.8%, 

 <.001). Over 8 months, the direct cost of the program was $131,000 with projected annual cost savings of up to $1.14 million to payers and up to $250 for each patient who switched.

A proactive and voluntary statin switch program to promote the use of a lower cost generic alternative can be successfully implemented in a fee-for-service health system setting with benefits to patients, providers, and payers.

A statin switch program was successfully implemented in a fee-for-service patient population by a large physician group in an academic healthcare system setting.

This collaborative and voluntary program allowed physicians and patients to have complete control of the switch process.

A centralized switch program saved time and money, while promoting the use of a lower cost generic alternative.

Over an 8-month period, 1710 of 3677 patients identified for switching from a brand to a generic statin did so at a direct cost of $131,000 with projected annual cost savings of up to $1.14 million to payers because of lower drug costs. In addition, each patient who switched saved up to $250 because of lower copayments.

Prescription drug spending has significantly increased in the past decade, totaling $217 billion in 2006 and representing 10.3% of total healthcare expenditures in the United States.

 Of this amount, 22% was paid by patients out-of-pocket, 34% by public health insurance plans, and 44% by private insurance plans. In addition, prescription drug spending is predicted to increase by 8% to 9% annually through 2016.

Targeted medication conversions to more cost-effective alternatives represent a safe and effective way to reduce drug expenditures for patients, providers, and payers. Studies have shown that lower prescription drug copayments are correlated with higher levels of medication compliance for 3-hydroxy-3-methylglutaryl coenzyme-A reductase inhibitors (statins), which are commonly used cholesterol-reducing drugs.

 Gibson et al studied the impact of statin copayments on patient adherence and healthcare expenditures.

 They found that every $10 increase in copayment resulted in a 1.8% reduction in medication adherence among new users and a 3% reduction among continuing users. The study also found that among continuing users adherent to statins, there was a significantly lower incidence of coronary heart disease—related hospitalization.

 Similarly, Schultz et al found that a $15 increase in copayment was associated with a 10% decrease in statin compliance in a retrospective analysis of claims data.

 Results from these and other pharmacoeconomic studies suggest that interventions to reduce patient copayments will improve statin adherence and, in turn, improve outcomes and reduce overall healthcare costs, especially in high-risk populations.

All currently available statins have been shown to reduce the incidence of cardiovascular events, which supports the belief that these benefits are class effects.

 Charts comparing doses of statins that provide similar levels of low-density lipoprotein cholesterol (LDL-C) reduction have facilitated the substitution of one statin for another.

 In June 2006, simvastatin (Zocor) became the third statin available as a generic, and many insurers instituted programs such as tiered copayments and step therapy requirements to promote its use and to encourage their members to switch from brand-name statins, if clinically appropriate. For example, the University of Michigan Benefits Office moved all brand-name statins, except atorvastatin (Lipitor) 80 mg, to the highest copayment level and added a step-therapy edit requiring patients to try a generic statin before any other brand-name statin could be covered.

 When such a change occurs, patients are typically notified via mail and asked to contact their physicians to reassess therapy and adjust medications if necessary. When a patient calls to request a medication change, the new prescription has to be approved and then called in or faxed to the patient’s pharmacy. This process involves multiple message relays that require staff and clinician time and effort, which can disrupt clinic workflow and may result in patients being temporarily without their medication. All of this causes additional, avoidable costs to patients, providers, and payers.

To assist our patients and physicians, the University of Michigan Medical School Faculty Group Practice, which includes approximately 1400 physicians who are directly involved in patient care, initiated a voluntary program to switch eligible ambulatory patients from A to Z (atorvastatin [Lipitor] to simvastatin [Zocor]). Although this type of switch program has proven effective in organizations with fixed formularies, such as the Department of Veterans Affairs and Kaiser Permanente, our program is the first we know of that targeted a large multipayer population, including fee-for-service patients, through a physician group.

This program was implemented at the University of Michigan Health System (UMHS) in Ann Arbor. The UMHS is a nonprofit teaching institution that provides more than 1.6 million outpatient visits annually between 3 hospitals and 30 health centers, which include 120 primary care or specialty clinics. It serves a large multipayer population.

The switch program was scheduled to begin approximately 6 months after the introduction of generic simvastatin as the price was expected to drop at that time with multiple manufacturers entering the market. The program was approved by the UMHS Ambulatory Formulary Committee and the Faculty Group Practice. The analysis of the program, reported here, was approved by the University of Michigan Medical School Institutional Review Board.

Patients received letters notifying them about the forthcoming changes in prescription copayments and preferred medications from their respective insurers. Physicians received several e-mail messages informing them about the changes in their patients’ prescription copayments, reasons for the switch program, and how it would be conducted. In addition, presentations about the program were given to primary care physicians, cardiologists, and endocrinologists at their departmental meetings. A clinical pharmacist also visited several high-volume clinics to answer any additional physician or staff questions. Health center personnel were informed of the program by their medical and Administrative directors.

Eligible patients were identified from electronic medical records and pharmacy claims data provided by 2 managed care organizations, 1 fee-for-service insurer, Michigan Medicaid, and the University of Michigan Benefits Office. Patients were eligible if as of January 2007, they were (1) on atorvastatin 10, 20, or 40 mg/day, (2) enrolled in a collaborating prescription drug plan, and (3) receiving care from a UMHS primary care physician, cardiologist, or endocrinologist. 

All patients enrolled in the University of Michigan Benefits Office’s prescription drug plan received identical letters in November 2006 about the coverage changes for statins, but only patients with a UMHS provider were included in the active switch program. Those patients without a UMHS provider served as the control group to help evaluate the benefits of this active switch program over the passive approach of sending patients letters.

All patients on atorvastatin 80 mg/day were excluded from the switch program because we could not obtain similar levels of LDL-C reduction with simvastatin and because there were randomized controlled trials demonstrating benefits of atorvastatin 80 mg/day for certain patient populations. Patients on medications with possible differential response to simvastatin compared with atorvastatin (ie, carbamazepine, cyclosporine, danazol, delavirdine, efavirenz, imatinib, oxcarbazepine, risperidone, verapamil, warfarin) were eligible for the program, but their physicians were notified of these potential drug interactions.

An electronic database was created using Microsoft Access 2002 that included (1) pharmacy claims data from participating insurance plans; (2) patient demographics (eg, name, age, sex, address, phone), select medication history, and latest laboratory values (total cholesterol, high-density lipoprotein cholesterol, LDL-C, triglycerides, alanine aminotransferase test) from our health system’s electronic data warehouses; and (3) the physician identified for each patient using an algorithm based on the number of clinic visits the patient had in 2006. From the database, reports were produced for each physician with information on each of their eligible patients and, for patients using a mail-order pharmacy service, a preprinted prescription. Physicians reviewed the proposed medication switches, changed the statin dose as needed, and approved or declined their patients to be contacted regarding the switch program. To facilitate this process, clinical pharmacists per formed a preliminary review of the reports for 7 high-volume physicians (ie, more than 50 assigned patients, range 55-130), as well as several others who asked for help. A total of 972 (26%) patients were reviewed by 3 clinical pharmacists. Physicians returned their reports to the switch team and the information was entered into the program’s database.

It was recommended that patients switch to an equipotent or greater dose of simvastatin, with respect to LDL-C reduction: (1) from atorvastatin 10 or 20 mg to simvastatin 40 mg, based on the published results supporting moderate dosing of statins,20 and (2) from atorvastatin 40 mg to simvastatin 80 mg.

A computer-aided telephone interview program was developed to guide pharmacy students in their discussions with eligible patients. Using a standardized script, the students informed patients about the program, told them that their physicians had approved the switch, and asked whether patients wanted to switch from atorvastatin to simvastatin. A maximum of 6 attempts were made to contact each patient, including 2 during weekdays, 2 during weeknights, and 2 on Saturdays. An information packet about the program was mailed to patients who could not be contacted. 

Patients who elected to switch received a new prescription for simvastatin, which was either called in to their local pharmacy within 24 hours or sent to them if they used a mailorder pharmacy. Patients using mail-order pharmacies had the option of having a 30-day supply of simvastatin called in to a local pharmacy if they had fewer than 28 days of atorvastatin remaining. Each patient also received a letter confirming the medication change and a laboratory requisition form for a fasting lipid profile and serum alanine aminotransferase test, to be drawn approximately 8 weeks after initiating simvastatin. The patients’ electronic medical records were updated. At the program’s conclusion, physicians were given a report summarizing their patients’ switch status.

We took advantage of a natural experiment and compared the proportion of patients in 1 prescription drug plan who converted from atorvastatin to simvastatin in the active switch program with a similar group of patients in a passive switch program (ie, they received a letter notifying them of the copayment increase for brand-name statins). As in The active switch program, patients in the passive switch program who either were on a brand-name statin other than atorvastatin or were on atorvastatin 80 mg/day were excluded.

Total program expenditures include materials/supplies and the salary and benefits of personnel directly involved with the switch program. Personnel included an internist, a Clinical pharmacist, a data analyst, a programmer, an administrator, an administrative assistant, and 9 pharmacy and nursing students. The amount of time each individual spent on the program was determined to calculate the number of full-time equivalent staff needed for this program.

Total program savings were calculated from an insurer’s perspective and were based on the differences in estimated drug costs for atorvastatin and simvastatin for the patients who switched as part of this program. For a 30-day supply of atorvastatin 20 mg, we used the average wholesale price minus 10%, which was $110 in September 2007.

 For a 30-day supply of simvastatin 40 mg, we used the maximum allowable cost in Michigan, which was $10 in September 2007.

Descriptive statistics were utilized for the demographic data. The chi-square statistical test was utilized for comparative analyses and a 

 value of <.05 was considered to be significant. All analyses were performed with STATA version 8.1 (StataCorp, College Station, TX).

In January 2007, 3677 patients met all the inclusion criteria. Baseline characteristics were similar for patients who did and did not switch from atorvastatin to simvastatin (

). These patients were assigned to 182 physicians, 154 (84.6%) of whom responded with their decisions on 3336 (90.7%) patients (

). Of the nonresponding physicians, 24 (13.2%) were specialists (cardiologists or endocrinologists), accounting for 114 (3.1%) patients. Physicians approved the proposed switches for 3207 (87.2%) patients; of these patients, 1710 (53.3%) agreed to switch from atorvastatin to simvastatin. A total of 52 (1.4%) patients who initially declined the switch and 37 (1.0%) patients unreachable by phone were subsequently switched by their physicians during clinic visits.

Patients on lower doses of atorvastatin were more likely to switch to simvastatin (

 <.01). The average atorvastatin dose of patients who agreed to the switch was slightly lower than that of patients who did not switch (19.0 mg vs 20.4 mg, 

<.01). Almost all the switched patients (99.4%) were converted from atorvastatin to an equipotent or greater dose of simvastatin. Only 59 (3.5%) patients who switched were on 1 or more drugs considered to interact specifically with simvastatin; 39 (2.3%) of those patients were on warfarin. These patients were flagged and additional counseling regarding the potential effects of simvastatin on their anticoagulation tests were provided by a clinical pharmacist.

The University of Michigan Benefits Office had the most patients in this study, with a total of 4339 patients on atorvastatin 10, 20, or 40 mg/day. These patients all received a letter notifying them of the copayment increase for brand-name statins. Those University of Michigan employees or their dependents who had an UMHS provider were included in the active switch program (n = 1867) and the passive switch group consisted of the remaining 2472 patients who had a non-UMHS provider.

, patients in the active group were younger (58.1 vs 61.1 years old, 

 <.001) and were less likely to have drug interaction(s) with simvastatin (5.4% vs 8.3%, 

 <.001). By the third quarter of 2007, generic simvastatin was being used by 59.2% of patients in the active switch program compared with 35.8% of patients in the passive switch group (

). The 23.4% greater generic statin dispensing rate among switch program participants resulted in a $150,000 cost savings by the end of the third quarter of 2007 and an estimated $370,000 in cost savings annually, based on the University of Michigan Benefits Office’s prescription drug plan cost structure. Patients saved up to $168 annually due to lower copayment amounts ($7 for simvastatin vs $24 for atorvastatin), excluding any unmeasured indirect costs (eg, time spent to contact a physician regarding this switch program).

The total program cost was approximately $131,000, including $7500 for materials/supplies and $123,500 for the 1.9 full-time equivalents who worked on the program from November 2006 through June 2007. This total does not include the time that physicians and 2 ambulatory care clinical pharmacists spent reviewing patients for this program, although others might wish to include the opportunity costs of this time in such a calculation. This program was funded by the University of Michigan Medical School Faculty Group Practice, which received a nominal monetary amount for each patient converted from atorvastatin to simvastatin from 2 of the insurers to help cover the expenses of providing this service to our patients.

Total projected annual cost savings ranged from $150 to $250 for each patient because of lower copayments and from $670,000 to $1.14 million for the payers because of lower drug costs. The range in cost savings for the payers reflects the savings that might accrue if all payers engaged in a passive approach such as that by the University of Michigan Benefits Office or if none did. Allowing for some variation in drug pricing and assuming that patients get monthly refills, the cost difference between atorvastatin and simvastatin translates to an annual savings of about $1000 per person switched for the payer.

Our results demonstrate that physicians in a large academic medical center that serves a diverse patient population can successfully conduct a centralized statin switch program. In contrast to some of the other medication switch programs reported, the voluntary and flexible nature of our program allowed physicians and patients to be in control of the switch process, from approval to dosing to prescription processing.

Our security-enabled computer-aided telephone interview program facilitated the electronic management of our database and allowed us to track the status of our patients from initial data load through every step of the switch process. Dynamic call and print queues were set up based on prespecified algorithms and were automatically updated in real to facilitate administrative operations and coordinate staffing needs.

One problem we encountered was that intensive manual data lookup using our electronic medical record system was needed to fill in missing patient information, dosing regimens, and interacting drugs for some patients. Inaccurate patient contact information prevented us from reaching 143 (3.9%) patients. In addition, some patients were assigned to physicians who were no longer seeing them. Some of these problems would not occur with more robust electronic medical records systems (eg, those with coded medication lists) and accurate identification of each patient’s physician.

Articles

MANAGERIAL

This report shows that a successful, cost-effective statin switch program can be implemented by a large physician group via a centralized, collaborative process.

Moving From A to Z: Successful Implementation of a Statin Switch Program by a Large Physician Group

To assess the incremental cost-effectiveness of inhaled medication use in chronic obstructive pulmonary disease (COPD).

A Markov model was constructed to estimate the incremental quality-adjusted life-years (QALYs) gained of the alternative treatment arms used in the Towards a Revolution in COPD Health (TORCH) study (ie, salmeterol—fluticasone propionate combination [SFC], salmeterol, fluticasone, and placebo).

The cycle length for the model was set to 3 months, and the maximum time horizon was set to 3 years. The cost-effective analysis was conducted from a third-party payer’s perspective in the US healthcare system. Future costs and effects were discounted at 3%. Multiple 1-way sensitivity analyses and a probabilistic sensitivity analysis using Monte Carlo simulation were performed to handle uncertainty.

The most cost-effective strategies are placebo (as-needed short-acting bronchodilator use with no maintenance therapy) when willingness to pay (WTP) is less than $52,800/QALY gained and SFC when WTP exceeds that threshold. When no maintenance therapy is not an acceptable option, the most cost-effective strategies are treatment with salmeterol when WTP is less than $49,500/QALY gained and treatment with SFC when WTP exceeds that threshold. The base-case analysis showed that incremental cost-effectiveness ratios of salmeterol, fluticasone, and SFC relative to placebo were $56,519, $62,833, and $52,046/QALY gained, respectively.

The most cost-effective strategy in moderate-to-severe COPD depends on how much society is willing to pay to achieve health improvements. When treatment with as-needed short-acting bronchodilator use does not provide adequate control, salmeterol or SFC would be the drug of choice depending on WTP.

The most cost-effective strategy in moderate-to-severe chronic obstructive pulmonary disease depends on willingness to pay (WTP).

The most cost-effective strategy is no maintenance therapy (as-needed short-acting bronchodilator use only) when WTP is less than $52,800 per quality-adjusted life-year (QALY) gained.

When no maintenance therapy is not an acceptable option, the most cost-effective strategies are treatment with salmeterol when WTP is less than $49,500/QALY gained and combination treatment with salmeterol—fluticasone propionate when WTP exceeds that threshold.

In 2003, 10.7 million US adults were estimated to have chronic obstructive pulmonary disease (COPD).

 Healthcare costs for COPD have continued to increase with the increased prevalence of the disease. The total cost of COPD in 2004 was $37.2 billion, including $20.9 billion in direct healthcare expenditures, $7.4 billion in indirect morbidity costs, and $8.9 billion in indirect mortality costs.

The Towards a Revolution in COPD Health (TORCH) trial

 is a large multicenter, randomized, double-blind, parallel-group, placebocontrolled study. Approximately 6200 patients with moderate-to-severe COPD were randomly assigned to twice-daily treatment with salmeterol—fluticasone propionate combination (SFC) (50 µg and 500 µg, respectively), fluticasone propionate (500 µg), salmeterol (50 µg), or placebo for 3 years. In that study, treatment with SFC was associated with a 2.6% reduction in mortality, which failed to reach statistical significance. In several outcomes, salmeterol alone seemed superior to fluticasone alone. How to apply these results to clinical practice has been debated.

Given the comparable therapeutic efficacies and adverse event rates among the inhaled medications used in the TORCH study, cost-effectiveness may have an important role in the decision as to which is the agent of choice in moderate-to-severe COPD. The objective of this study was to assess the cost-effectiveness of these inhaled medications.

A Markov model was constructed with 4 mutually exclusive disease states for patients with moderate-to-severe COPD. The disease states include stable, exacerbation requiring a physician visit, severe exacerbation requiring hospitalization, and death. The cycle length for the model was set to 3 months to fully incorporate the effect of an exacerbation on quality of life,

 and the maximum time horizon was set to 3 years to match the duration of the TORCH study. The cost-effectiveness of 4 strategies used in the TORCH study (ie, salmeterol, fluticasone, SFC, and placebo) was compared. An incremental cost-effectiveness ratio (ICER) of each strategy was calculated. A discount rate of 3% was applied to costs and health benefits based on international recommendations.

 The effect of changes in the discount rate ranging from 0% to 7% was examined by 1-way sensitivity analysis. A half-cycle correction was applied in each period to allow for the fact that transitions between health states could take place at any time during the modeled 3-month intervals.

 Decision analyses were performed using TreeAge Pro (TreeAge Software, Inc, Williamstown, MA) and Excel 2000 (Microsoft, Redmond, WA).

Efficacy data were derived from the TORCH study. Efficacy data incorporated in the present model were frequency of exacerbation requiring a physician visit, hospitalization rate, all-cause mortality rate, and changes in health status of each treatment arm. It was assumed that moderate exacerbations requiring treatment with systemic corticosteroids required a visit to a physician’s office or an emergency department.

Utility weights were estimated by converting St George’s Respiratory Questionnaire (SGRQ) scores measured in the TORCH study to EuroQoL Five-dimension Questionnaire (EQ5D) scores. The EQ5D is a generic health-related quality-of-life measure with scores ranging from 0 to 1 (where 1 represents a perfect state) and allows measurement of utility weights. The following equation was formulated based on data from a study by Ståhl et al

× SGRQ Score (correlation coefficient, -0.9851)

Transition probabilities were calculated and applied as follows. The quarterly mortality rate associated with each arm was calculated using an exponential approximation described by Beck et al

 and incorporated in the Markov model. Study subjects who died during each 3-month period were excluded from further analysis. Physician visit and hospitalization rates observed in the TORCH study were adjusted for the cycle length of 3 months and incorporated in the model as transition probabilities. The survivors of each 3-month cycle continued through another cycle, and a similar set of probabilities for death, hospitalization, and exacerbation was applied. These nonfatal event rates were also used between exacerbation states because such information was not provided in the TORCH study. Twelve 3-month cycles translated into 36 months of follow-up.

The cost-effective analysis was conducted from a thirdparty payer’s perspective in the US healthcare system; hence, only direct costs were included in this analysis. Costs for medications related to respiratory disease were determined on the basis of the mean wholesale price for 200612 discounted by 15% to adjust for typical retail acquisition costs, with a $2.50 dispensing fee added for each 30-day period.

 The following assumptions were made to estimate the costs of exacerbations: (1) Ten percent of unscheduled physician visits occurred in emergency departments.

 (2) A 7-day course of prednisone (40 mg/d) and antibiotics was used for outpatient treatment of COPD exacerbations. (3) The mean length of hospital stay for COPD exacerbation was 4.9 days.

 The mean cost of hospitalization because of COPD exacerbation was quoted from Solucient’s Medicare database,16 which was based on the Medicare Provider Analysis and Review File. Actual costs rather than reimbursement were used because the reimbursement per discharge was lower than the mean actual costs. The cost of a physician visit was based on the mean reimbursement of the Current Procedural Terminology-4 code 99214 for a visit made by an established patient to a general practitioner.

 Costs for inpatient physician visit and emergency department visit for COPD exacerbations were based on data by Wilson et al.

 The cost of antibiotic treatment was based on data by Sin et al.

 All costs are reported in 2006 US dollars (adjusted when necessary) using the Consumer Price Index inflation calculator provided by the US Department of Labor.20 The baseline values used in the present model for all variables are listed in 

Variability was assessed by multiple 1-way sensitivity analyses. The value of each variable was placed in the decision model with its upper and lower limits, while holding all other values constant (eAppendix Table 1). For efficacy variables, these limits were derived from the 95% confidence intervals of relative risks for each variable, comparing each active arm with placebo results reported in the TORCH study. For cost variables, these limits were decided as follows. The limits of medication costs were derived from the highest and lowest costs listed in the Drug Topics 2006 Red Book.

The limits of hospitalization cost were derived from the 25th percentile and 75th percentile of the mean cost listed in the Medicare database.

 The limits of minor exacerbation costs were set at -20% and 20% of the base value

 because variances were not found in the literature. Threshold analyses identified the value of each variable across its range (if any) at which the ICER of another strategy exceeds that of the most cost-effective strategy. Monte Carlo simulation was used in the model to handle the uncertainty probabilistically, with distributions assigned to multiple parameters to reflect the second-order uncertainty.

 Probability distributions of exacerbation rates, hospitalization rates, all-cause mortality rates, quality-of-life utility weights, and costs of the medications were incorporated in the model. Efficacy parameters were assigned beta distributions, and unit cost parameters were assigned gamma distributions.

 The present analysis was based on 10,000 simulations. The uncertainty in costs and effects was illustrated as a cost—benefit acceptability curve.

At baseline, the patient population analyzed in the present model was 76% male and comprised 43% current smokers and had the following mean (SD) values: age of 65 (8.3) years, prebronchodilator forced expiratory volume in the first second of expiration (FEV

 percentage of 44.0% (12.4%), and baseline SGRQ score of 49.3 (17.1). 

 summarizes the results of the base-case analysis. The ICERs of salmeterol, fluticasone, and SFC relative to placebo were $56,519, $62,833, and $52,046/QALY gained, respectively. No treatment arms were clearly dominated by any other arm (ie, no strategies have higher cost and lower effectiveness than some other options). The salmeterol and fluticasone strategies were each dominated by the principle of extended dominance by a blend of the SFC and placebo strategies. Multiple 1-way sensitivity analyses revealed that the incremental cost-effectiveness among the alternative treatment arms was sensitive to several variables, including costs of SFC and salmeterol, all-cause mortality rate with each treatment arm, hospitalization rate with salmeterol, and utility weight with each treatment arm during a stable phase of the disease. The results of comparative cost-effectiveness were robust over reasonable parameter uncertainty with other variables (eAppendix Table 1).

 shows a selection of the 1-way sensitivity analyses (tornado diagram) for the salmeterol arm. The dotted vertical line represents the base-case incremental cost-effectiveness estimate ($56,519/QALY gained). In the graph, a horizontal bar was generated for each selected variable being analyzed. Expected value is displayed on the horizontal axis, so each bar represents the range of expected values generated by varying the related variable. A wide bar indicates that the associated variable has a large potential effect on the expected value of the present model. Most notable among these variables were the utility weight and the mortality rate. As the utility rate was varied from 0.569 to 0.591, the ICER ranged from $37,522 to $117,396/QALY gained, whereas the ICER ranged from $42,204 to $93,120/QALY gained when the quarterly mortality rate was varied from 0.97% to 1.46%. The variables that significantly affected the cost-effectiveness and the magnitudes of effects of their ranges on the cost-effectiveness in the SFC and fluticasone arms were similar to those in the salmeterol arm.

A sensitivity analysis using different data sets to estimate utility weights was also performed. Rutten-van Mölken et al26 examined the correlation between EQ5D utility scores and SGRQ scores. Assuming a linear correlation between EQ5D and SGRQ scores and using the EQ5D scores calculated with the United Kingdom value set (also known as the MVH A1 value set), the following equation was formulated to estimate utility weights. EQ5D Index = (1.173 - 0.009160) × SGRQ Score. Then, ICERs for salmeterol, fluticasone, and SFC were $53,873, $68,175, and $52,785/QALY gained, respectively. These results were similar to those of the present base-case analysis. However, when the EQ5D scores calculated with the US value set

 were used, ICERs for salmeterol, fluticasone, and SFC relative to placebo were $58,149, $85,806, and $60,159/QALY gained, respectively. The fluticasone strategy was dominated by the salmeterol strategy. The SFC strategy was still the most effective but also the most costly. The ICER for SFC relative to salmeterol was $62,604/QALY gained.

 (available at www.ajmc.com) shows the uncertainty around the costs and health benefits on a cost-effectiveness plane. Each dot represents 1 of 10,000 model simulations. Data points from the Monte Carlo simulation illustrate that SFC is generally associated with a gain in QALYs at the expense of higher costs compared with other inhaled medications. 

shows a net benefits acceptability curve that represents, at each level of willingness to pay (WTP), the percentage of the simulations for which each strategy is cost-effective relative to the others. The placebo arm (as-needed short-acting bronchodilator use with no maintenance therapy) was the most cost-effective strategy when WTP was less than $52,800/QALY gained. The SFC arm became the most cost-effective strategy when WTP exceeded that threshold. Among the active arms, salmeterol had the highest likelihood of being the most costeffective when WTP was less than $49,500/QALY gained, and the SFC strategy became the most costeffective when WTP exceeded that threshold.

To my knowledge, this is the first published study that compares the cost-effectiveness of an inhaled corticosteroid (ICS), a long-acting ß

-agonist (LABA), and a combination of both from a US healthcare payer’s perspective. The cost-effectiveness ratios of the inhaled medications analyzed in this study were comparable to those of other therapies commonly used in clinical practice.28 Although the reduction in all-cause mortality with either strategy did not reach statistical significance in the TORCH study, the increase in QALYs gained was significantly better with all active treatments compared with placebo, especially with the combination therapy (eAppendix Figure). The effect of fluticasone on QALYs gained was also positive because of significant improvement in quality of life, despite the fact that it had a trend toward increased allcause mortality and increased risk of pneumonia.

The SFC therapy seemed to be the most cost-effective in the base-case analysis. However, the results were sensitive to WTP and to changes in several variables (eAppendix Table 1).

The present study differs in some respects from a recent Canadian study.

 In that study, the cost of the combination therapy per QALY gained was consistently higher than that of LABA (ie, LABA was consistently more cost-effective than the combination therapy), while the present study showed comparable cost-effectiveness between SFC and salmeterol. A possible explanation for this discrepancy is that the greater improvement in quality of life among the combination group during a stable phase of the disease was not considered in the Canadian study, which could have led to underestimation of the incremental effectiveness in the combination group. When the incremental QALYs gained of SFC during the stable phase of the disease, relative to salmeterol, were eliminated from the present model, the costeffectiveness ratio of SFC increased from $52,046 to $95,136/ QALY gained, and SFC became less cost-effective than salmeterol ($56,519/QALY gained), as the Canadian study noted. After the adjustment described, the difference in cost-effectiveness ratios between the combination therapy and LABA became strikingly similar.

The ICERs were also calculated using key variables from a recently published systematic review

 to assure the robustness of the baseline analysis. The ICERs for salmeterol, fluticasone, and SFC relative to placebo were $53,911, $81,900, and $56,466/QALY gained, respectively. The ICERs relative to the next less costly or less effective strategy are given in 

 (available at www.ajmc.com). Fluticasone was dominated by salmeterol (ie, fluticasone was more expensive but less effective than salmeterol). A net benefits acceptability curve is shown in 

. The most cost-effective strategies were placebo (asneeded short-acting bronchodilator use only) when WTP was less than $55,200/QALY gained, salmeterol when WTP was between $55,200 and $62,400/QALY gained, and SFC when WTP was more than $62,400/QALY gained. The variables incorporated in the model were relative risks for death (0.91, 1.00, and 0.82 for LABA, ICS, and combined LABA-ICS vs placebo, respectively); differences in COPD-related hospitalizations

 (-0.01, -0.02, and -0.03/patient-year for LABA, ICS, and combined LABA-ICS vs placebo, respectively); changes in SGRQ scores (-1.59, -1.54, and -3.10 for salmeterol, ICS, and combined LABA-ICS, respectively), which were converted to EQ5D scores using the same method; and relative risks for COPD exacerbations (0.87, 0.85, and 0.77 for LABA, ICS, and combined LABA-ICS, respectively).

There are several limitations in this analysis. First, the results should be interpreted cautiously given the magnitude in the uncertainty of incremental costs and effectiveness. The efficacy data may have been affected by the differences in healthcare system and hospitalization criteria in different countries, as the TORCH study was an international trial. Other confounding factors include high and inconsistent dropout rates (34%-44%) in each arm of the TORCH study, as well as the effect of cointerventions in a real-world setting and their interactions with the study drugs.

Second, the utility values were estimated by converting SGRQ scores to EQ5D scores. In general, utility values derived from choice-based methods such as the standard gamble and the time trade-off are thought to be the most robust and to have the greatest theoretical validity and acceptable levels of reliability. 32 However, to my knowledge, there are no established standards for how to estimate or measure utility weights in a cost-effectiveness analysis for patients with COPD, as summarized in Table 2. Although there is a paucity of data on the precision of the utility estimates used in this study, the uncertainty of those estimates was assessed in the sensitivity analyses. The cost-effectiveness ratio of fluticasone was always inferior to that of the other arms, and the costeffectiveness ratio between salmeterol and SFC was comparable regardless of the method used. A change in the utility weights associated with each arm resulted in large changes in the cost-effectiveness outcomes in the present study. This may mean that the parameter has to be determined very accurately or that an alternative has to be redesigned for low sensitivity. The method used to estimate the utility values needs further validation in future studies.

Third, the present model was limited to a 3-year time horizon and to patients with moderate-to-severe COPD. The costeffectiveness of the medications beyond 3 years was not studied because it is unclear that the clinical efficacies and potencies of the inhaled medications remain the same beyond that point. Furthermore, there have been many controversies about the long-term adverse effects of ICS (such as cataract and osteonporosis) as to whether they cause such adverse effects or if it is merely an  association,

 as well as about the safety profile of LABAs as to whether they increase deaths in patients with asthma and COPD.

 The costs of these potential adverse events were not included in this study because it was thought to be unlikely that these events would occur in the 3-year time frame. The present study was also compared with another Canadian study by Sin et al.

 They reported that the cost per QALY gained with the use of ICS in patients with stage 2 or 3 disease when no mortality benefits were assumed was $34,100 at a discount rate of 3% under a 3-year time horizon. When adjusted for inflation and for the differences in the costs of the medication and  hospitalization between Canada and the United States, the cost per QALY gained with the use of ICS using the present model was $41,028 in 2004 US dollars at a discount rate of 3% under a 3-year time horizon, which is comparable to the estimate in the study by Sin et al. Despite all these limitations, the results of the present study were comparable to the Canadian studies

In conclusion, this analysis showed that the most cost-effective strategy in  moderate-to-severe COPD depends on how much society is willing to pay to achieve health improvements. The most cost-effective strategies are placebo (as-needed shortacting bronchodilator use with no maintenance therapy) when WTP is less than $52,800/QALY gained and SFC when WTP exceeds that threshold. When treatment with as-needed shortacting bronchodilator use does not provide adequate control or is not an acceptable option, the most cost-effective strategies are salmeterol when WTP is less than $49,500/QALY gained and SFC when WTP exceeds that threshold. These results may assist clinicians and healthcare policy makers to make informed decisions as to which therapy for moderate-to-severe COPD would be optimal in the US healthcare system.

From the Department of Pulmonary, Critical Care and Environmental Medicine, University of Missouri—Columbia.

The author reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis; and provision of study materials or patients.

Yuji Oba, MD, Department of Pulmonary, Critical Care and Environmental Medicine, University of Missouri—Columbia, One Hospital Dr, Health Science Center MA419A, Columbia, MO 65212. E-mail: 

CLINICAL

This cost-utility analysis compares the cost-effectiveness of salmeterol, fluticasone propionate, combination salmeterolfluticasone, and no maintenance therapy in chronic obstructive pulmonary disease.

Cost-Effectiveness of Salmeterol, Fluticasone, and Combination Therapy for COPD

 To report on the first year of a program using standardized rating scales within a large, multisite mental health system of care for children and to assess the validity, reliability, and feasibility of these scales.

Naturalistic follow-up of clinicians’ ratings.

Clinicians filled out the Brief Psychiatric Rating Scale for Children (BPRS-C) and the Children’s Global Assessment Scale (CGAS) at intake and discharge/90-day follow-up for all new patients.

Data were collected on 2396 patients from all 20 sites serving children in the Partners HealthCare network. Mean scores for both BPRS-C and CGAS showed worst functioning at inpatient sites, followed by Acute Residential Treatment, then partial hospital, then outpatient sites. All patients re-rated at discharge or 90-day follow-up showed a significant improvement in scores. Inter-item reliability on the BPRS-C was acceptable, with Cronbach alphas of .78 and .81. Feasibility at intake was demonstrated in that 66% of all patients had a completed form at intake. Reassessment at discharge also appeared to be feasible in more restrictive levels of care, but less feasible in outpatient sites, where fewer than 25% of all patients had a follow-up form.

This evaluation suggested that the 2 standardized measures appeared to be valid and reliable as part of routine intake and discharge/follow-up in a large child psychiatry system of care. Whether these measures are truly clinically useful remains to be demonstrated because there is at present no gold standard for assessing the quality of treatment or change caused by it.

Two standardized psychiatric rating scales used within a large, diverse, multisite mental health system of care for children were assessed after their first year of use.

The scales were determined to be valid, reliable, and feasible for assessing admissions as required by a managed care insurer.

Collecting data at follow-up/discharge was possible in about two-thirds of inpatient, partial hospital, and acute residential treatment cases, but in fewer than 25% of cases in outpatient settings.

Patient scores on both measures showed statistically significant improvements from admission/

Increasingly, individual insurers as well as governmental and regulatory agencies are requiring mental health clinicians to use standardized instruments to assess patients as a part of clinical care.

 supports the measurement of outcomes in psychiatry, but stresses the importance of using measures that are feasible in clinical settings as well as valid and reliable. We describe the experiences of a large and diverse child mental health system of care, Partners Psychiatry and Mental Health (PPMH), in the first year of implementing a uniform set of outcome measurements across all sites at all levels of care. We focused on (1) examining the extent to which outcomes measures were implemented successfully within the PPMH system and (2) the validity and reliability of the measures themselves as they pertain to this setting.

To comply with contractual terms from the Commonwealth of Massachusetts, the Massachusetts Behavior Health Partnership (a Medicaid managed care carve-out company) required providers to use standardized, validated instruments to assess and track outcomes for all new patients. Beginning July 1, 2005, these patients had to be rated at intake/admission and at regular 90-day follow-up intervals and/or discharge. The mandate also required clinicians to use the information gathered from these forms in patient feedback and treatment planning.

In order to gather systemwide data, all PPMH sites agreed to utilize the same instruments and procedures. In choosing measures, a steering committee of staff from the sites considered ease of use, low cost (public domain preferred), and broad applicability across a wide range of ages, diagnoses, and levels of care. After assessing the available options, the committee chose 2 measures: the Brief Psychiatric Rating Scale for Children (BPRS-C)

 as a measure of symptom severity and the Children’s Global Assessment Scale (CGAS)

The BPRS-C has been used in a number of studies of research and clinical populations including patients with bipolar disorder, schizophrenia, and psychotic disorder and other studies requiring a multidimensional assessment of symptoms.

 The measure consists of 21 discreet symptom areas, each rated for severity

 along a 7-point Likert scale. A total score ranging from 0 to 126 is computed by summing all 21 items, with higher scores indicating greater symptom severity. Changes in this total score are used when evaluating pretreatment to posttreatment change.

 The reliability and consistency of the BPRS-C have been tested over time, and it has become an integral measure in diverse psychiatric venues.

 The CGAS is a practical complement to the BPRS-C because it is used for the assessment of “social and cognitive competence rather than symptoms.”

 PPMH added the CGAS as a second measure in order to obtain an easily understood rating of overall functioning. The CGAS has been used in research for more than 2 decades and is very similar to Axis V of the 

Diagnostic and Statistical Manual of Mental Disorders

 Several studies have used the CGAS to track patients over time.

We embedded the BPRS-C and CGAS in an Outcomes Rating Form that also included several other fields that we needed for this project. The Outcomes Rating Form is a single 2-sided measure that contains the BPRS-C, CGAS, and 10 other variables about the treatment facility, patient, and clinician. Individual clinicians are responsible for filling out all fields; they select their answers by filling in a blank circle on the scanable form. Other information collected on each form includes medical record number, date of administration, patient’s age and sex, patient’s insurance provider, hospital location, and indication whether the form is for an initial visit/ admission, a quarterly follow-up, or a discharge visit. Clinical fields include CGAS score, primary and secondary Axes I and II diagnoses, and the 21 BPRS-C items. To ensure reliability, clinicians were provided with the descriptive anchors for the BPRS-C developed by Hughes and colleagues

 and level descriptions for the CGAS as developed by Shaffer and colleagues

 on separate pieces of paper, and received on-site training in how to apply them.

During spring 2005, before the commencement of the outcomes initiative, members of the Child Outcomes committee traveled to all of the PPMH sites to provide clinicians with background on the initiative and instruction in the procedures and the use of the 2 measures. An off-site professional vendor was selected to scan and process the standardized forms. Beginning on July 1, 2005, completed forms were mailed on a monthly basis from each site to the vendor, who then provided data files to the Institute for Health Policy, a healthcare research organization within 1 of the PPMH hospitals. The Institute for Health Policy maintains the database and is primarily responsible for data analyses and reporting on issues of compliance and data accuracy. Administrative personnel generally collect forms, and at many sites those clinicians with outstanding forms are sent reminders.

Pediatric patients are seen at 20 PPMH sites. These sites are part of academic medical centers (Brigham and Women’s, McLean, and Massachusetts General hospitals), community hospitals (North Shore Medical Center), and community health centers affiliated with these institutions (Massachusetts General Hospital Charlestown, Chelsea, Revere, and North End; Brigham and Women’s South Jamaica Plain and Brookside; North Shore Boston Street and Lafayette Street clinics). The level-of-care array includes inpatient (McLean Franciscan Hospital and North Shore Medical Center Hunt units), combined partial hospital and Acute Residential Treatment (McLean units in Belmont, Brockton, and at the Franciscan Hospital, and the Klarman Eating Disorders Center); freestanding partial hospital (North Shore); and outpatient clinics at 11 of these locations. All sites are in eastern Massachusetts and most are affiliated in some way with the Harvard Medical School. The sites range in size from 2 to 20 full-time-equivalent clinicians.

All patients who were admitted to inpatient, partial hospital, or acute residential treatment, or who had initial visits to outpatient sites between July 1, 2005, and June 30, 2006, were included in the study sample. Patient discharges and follow-up visits after June 30, 2006, that could be matched to admission/initial visits occurring during our study period also were included. The study was reviewed by the Partners Institutional Review Board and classified as exempt.

Because we wanted to assess the degree of compliance with the requirement of obtaining an Outcomes Rating Form for each admission/discharge, we had to calculate the denominator of all new cases for the entire system. We combined data from several PPMH billing databases to determine the total number of admissions and intakes for all sites between July 2005 and the end of June 2006.

The characteristics of patients participating in our initiative were assessed using information from intake/admission forms. Intake and follow-up completion rates across levels of care were used to assess feasibility. As noted above, completion rates for intakes/admissions were calculated by comparing the number of completed forms with the expected number based on billing data. Follow-up completion rates were calculated by comparing the number of patients with both intake/admission forms and follow-up/discharge forms with the number of patients with intake/admission forms.

We hypothesized that overall validity of the 2 measures score over the course of treatment within each level of care; (2) a significant difference in impairment score on each measure according to the level of care (inpatient, partial hospital, acute residential treatment, outpatient); and (3) cross-measure agreement between the 2 instruments. For the BPRS-C, reliability was explored via the Cronbach alpha statistic. Because the CGAS is a single-item rating, inter-item reliability could not be calculated. Changes in scores over the course of treatment were assessed at the patient level by determining the difference between follow-up/discharge ratings and intake/admission ratings for both the BPRS-C and CGAS, with significance testing using paired 

 tests. Differences in intake scores across levels of care were assessed by using analysis-of-variance and contrast tests. We assessed cross-measure agreement between the 2 tools at both intake/admission visit and 90-day follow-up/discharge using Pearson product moment correlations.

During the first year of this project, throughout the entire 20 sites of Partners Child Psychiatry and Mental Health System, 3636 children and adolescents were seen for intake/admission. Of these children, 2396 were rated using the standardized  Outcomes Rating Form for CGAS/BPRS-C. As would be demonstrated if there were (1) an improvement in shown in Table 1, 508 of these forms (21%) were from inpatient sites, 311 (13%) were from partial hospital sites, 555 (23%) were from acute residential treatment sites, and 1022 (43%) were from outpatient sites. Duplicate record numbers were flagged and removed from the dataset if they appeared by date to represent duplicate ratings of the same patient for the same period of time.

Across all levels of care the mean age of patients was 13.4 years. Systemwide there were 1308 (55%) females and 1075 (45%) males. Table 1 shows that about one-third of all patients (33.6%) were covered by some form of commercial Blue Cross Blue Shield product, another 22% by other commercial insurers, and nearly one-third (29%) were covered by Medicaid. In about 16% of all cases, the insurance could not be categorized or was missing.

Table 2 shows the number of admission/intake forms expected according to Management Information System billing data and the number and percentage of forms actually received for each level of care. As shown, intake form return rates ranged from 63% to 70% across the 4 levels of care, with overall intake/admission returns at 66% systemwide.

Table 2 also shows the number of follow-up/discharge forms received at each level of care for patients who had an Outcomes Rating Form completed at admission/intake. For inpatient,  partial, and acute residential treatment hospitals the discharge completion rate ranged from 65% to 79%. At outpatient sites the follow-up completion rate was much lower (22%). Systemwide, the overall rate of follow-up/discharge Outcomes Rating Forms was 49%. It was not possible to calculate the treatment interval between intake and follow-up, because the field on the form asked for date of form completion rather than date of follow-up/discharge. In the outpatient sites the forms were to be given at approximately 90 days into treatment, whereas at inpatient, partial hospital, and acute residential treatment sites the forms were to be completed on the day of discharge, a pre/post interval that could vary considerably.

Given the lower rates of completion for follow-up ratings, within each level of care the full sample at intake was compared with the smaller sample of patients who completed forms at follow-up as well as intake for each of the 3 major demographic variables. Although statistically significant differences were observed for more than half (7/12) of the comparisons, all the differences were small (±10%) and did not appear to show a clear pattern when compared across levels of care. For inpatients there were no significant demographic differences for cases seen for admission and discharge versus those seen for admission alone; for outpatients, only insurance type revealed a significant difference (patients with Medicaid were significantly less likely to have completed follow-up forms). In acute residential treatment and partial hospital cases, however, patients with Medicaid were more likely to complete discharge forms. For acute residential treatment patients, females were less likely to have discharge forms; for partial hospital patients, females were more likely to have discharge forms. For both of these levels of care, patients over age 18 years were less likely to have discharge forms.

Table 3 shows the mean scores, the changes in mean scores, and significance levels of the 

 tests for pairs comparing pretest and posttest data for subjects with valid BPRS-C and CGAS data at both intake/admission and follow-up/discharge. Time 1 means are for only those 1185 subjects who also have time 2 scores (paired data within subjects), not the full sample of 2396 who completed forms at time 1. The number of subjects at time 2 is much smaller due primarily to the large number of outpatients who did not return for follow-up after intake and/ or missing data. The means are given for each level of care separately.

As the table shows, mean scores for both BPRS-C and CGAS showed significant improvements over time (

 <.001 for all) within all 4 levels of care. Paired 

 tests demonstrate that patients in inpatient, acute residential treatment, partial hospital, and outpatient treatment showed statistically significant improvement in functioning (as measured by mean increases in time 2 CGAS scores), as well as decrease in symptoms (measured by mean decreases in time 2 BPRS-C scores) from intake/admission to 90-day follow-up/discharge.

Table 3 also can be viewed from another perspective, illustrating the way in which BPRS-C and CGAS mean scores differed across levels of care, with poorer functioning at more restrictive levels of care. Contrast analyses using generalized linear models show that mean initial CGAS scores were significantly different among all levels of care and that mean initial BPRS-C scores were significantly different between any pair of settings except between the acute residential treatment and partial hospital settings. Specific contrast analysis significance levels are listed in footnote b of Table 3.

Table 4 shows the correlations between the BPRS-C and CGAS scores within each level of care and in the overall sample at intake/admission (time 1) and at follow-up/discharge (time 2). As in Table 3, data shown are only for subjects who had scores at both time 1 and time 2. At intake/admission these 2 measures showed significant correlations with each other ranging from r = -.34 to r = -.47 for the 4 different levels of care. At follow-up/discharge the correlations between CGAS time 2 and BPRS-C time 2 also were significant and in fact were somewhat higher, ranging from r = -.59 to r = -.69 at the 4 levels of care.

For the BPRS-C, the measure clinicians used to rate symptoms of their patients, inter-item reliability was assessed using the Cronbach alpha statistic. Ratings of .78 for initial assessments and .81 for follow-up assessments demonstrate the internal consistency of clinician ratings, suggesting that the BPRS-C was a reliable instrument in this population.

Data from the first 12 months of a systemwide outcomes measurement initiative confirm that it is possible to implement procedures using standardized instruments to rate symptom severity and level of functioning at intake/admission and to collect follow-up/discharge data across the full range of facilities for mental health care for children from inpatient to outpatient. All 20 sites in the current sample were able to incorporate standardized rating forms into their intake procedures, with 66% of all eligible intake cases having assessment forms completed during the first year of the initiative.

Follow-up assessment also appeared to be feasible, at least in the more restrictive levels of care. For inpatient, acute residential treatment, and partial hospital patients, the rates of discharge form completion ranged from about two-thirds to about three-quarters of all admissions, whereas in the outpatient settings follow-up forms were completed for fewer than one-quarter of all intakes. Determining the true feasibility of follow-up form completion in outpatient settings proved to be quite difficult. Calculating a valid follow-up form completion rate for outpatient cases was impossible with the data we had because the denominator of all patients who actually entered treatment was not known. Because data on the number of visits for each case were not collected, it was not possible to tell which patients dropped out after just 1 or 2 intake visits.

An unpublished report titled “Measuring outcomes in outpatient child psychiatry: the contribution of electronic technologies and parent report” by J. M. Murphy and colleagues (Massachusetts General Hospital, 2008) provides additional information on completion of follow-up forms. In 1 large clinic where the actual number of visits could be tracked for each patient, an examination of cases that did not have follow-up forms showed that about half of all children seen for intake did not return for more than 1 or 2 additional visits. Because clinicians were not required to fill out follow-up forms for cases unless there had been at least 3 visits, the other cases would have to be excluded from a valid denominator. Using this figure to estimate the true denominator for outpatient follow-up forms in this 1 outpatient clinic would have led to a completion rate of about 40%. It also is important to note that when the above outpatient setting switched to an electronic data collection  system, it was possible to obtain follow-up forms for more than 80% of cases. Both of these findings suggest that it is possible to make follow-up assessments more feasible in outpatient settings.

Validity of the measures was supported by several findings. First, scores obtained here were comparable to those reported for patients receiving similar levels of care in other studies. Second, in the current study, the 2 measures used were significantly correlated with each other at both intake and followup. Third, scores on both measures improved significantly from intake to follow-up in all 4 levels of care. Fourth, scores on both measures showed significantly more impairment in more restrictive levels of care compared with less restrictive levels of care. A reasonably high level of inter-item reliability was shown for the BPRS-C at both administrations.

Strengths and Limitations of the Study and the Program

Strengths of this study include a relatively large and representative sample drawn from a relatively large and diverse child psychiatry system of care, high rates of form completion at intake under real-world conditions, and the validity and reliability of the measures used. Limitations include lower levels of form completion at time 2, especially in outpatient settings, and lack of information on compliance with the insurers’ other mandates to incorporate score data into treatment planning and patient feedback. The fact that both measures relied only on clinician report to assess improvement leaves open the question as to the degree to which such reports could be biased.

Strengths of the program were the fact that it was able to incorporate outcome measures as a standard of care in a large and diverse system and that it has been sustainable over more than 3 years. Limitations included the time burden the initiative placed on clinicians and administrative staff. Moreover, many clinicians reported that because they themselves provided all of the information on the rating forms, the scores did not give them any new information that would make a significant contribution to treatment planning. As noted by Busch and Sederer,1 the lack of benefit to individual clinicians and patients was probably a negative incentive for completing the forms. Nevertheless, the Child Outcomes committee concluded that the clinician-completed BPRS-C and CGAS should still be used as a systemwide threshold measure with the possibility that parent-completed instruments and/or diagnosisspecific measures might need to be added in the future. Such additions would have to be weighed carefully, considering the additional burden for clinicians and administrative staff.

In the project’s first year, some unexpected benefits as well as burdens were identified. Data from the project gave the system a yardstick to assess functioning and symptom severity at different levels of care (inpatient, acute residential treatment, partial hospital, and outpatient), which has enabled the psychiatry department to provide quality assurance data for a number of administrative reports.

Although it is not possible from the data in this study to determine the extent to which these measures may have been used to provide data for treatment planning or patient feedback (the insurer’s other 2 mandates), it was clear that the original procedure of using paper and pencil forms mailed out for off-site scoring did not lend itself to providing the kind of rapid feedback that would be most useful for these purposes. However, use of many of the newer types of electronic technology (eg, input of the measures through kiosks, fax backs, tablets, or digital pens) could speed up the process considerably. Several of these methods already have been pilot-tested at 1 of the PPMH sites and have demonstrated an ability to substantially increase outpatient form completion rates at both intake and follow-up (Murphy et al, unpublished report, Massachusetts General Hospital, 2008). With or without these methods, because other major insurers now are mandating their own outcomes measures, it seems likely that the use of such measures is here to stay.

 From the McLean Hospital (JG, RJB, MP, RLB), Belmont, MA; the Department of Psychiatry (JG, RJB, MP, CV, MJ, JMM), Harvard Medical School, Boston, MA; the Department of Psychiatry (RAC), North Shore Medical Center, Salem, MA; the MGH Institute for Health Policy (CV, ZL), Boston, MA; the Department of Psychiatry (NTK, RLB, MJ, JMM), Massachusetts General Hospital, Boston, MA; Partners Psychiatry and Mental Health (SJD, KJS), Boston, MA; and the Department of Psychiatry (MJ), Newton Wellesley Hospital, Newton, MA.

The authors (JG, RJB, RAC, MP, CV, NTK, RLB, ZL, MJ, SJD, KJS, JMM) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Concept and design (JG, RJB, RAC, MP, NTK, ZL, MJ, SJD, KJS, JMM); acquisition of data (RJB, MP, NTK, RLB, JMM); analysis and interpretation of data (JG, RJB, RAC, MP, CV, NTK, RLB, ZL, SJD, KJS, JMM); drafting of the manuscript (RAC, MP, NTK, RLB, MJ, JMM); critical revision of the manuscript for important intellectual content (JG, RJB, MP, CV, RLB, ZL, MJ, SJD, KJS, JMM); statistical analysis (CV, ZL, JMM); provision of study materials or patients (RJB, RLB, JMM); administrative, technical, or logistic support (RJB, RLB, MJ, JMM); and supervision (CV, MJ).

J. Michael Murphy, Department of Psychiatry, Massachusetts General Hospital, 55 Fruit St, YAW 6A, Boston, MA 02114. E-mail: mmurphy6@partners.org.

Assessing outcomes in psychiatric practice: guidelines, challenges, and solutions. 

2. Hughes CW, Rintelmann J, Emslie GJ, Lopez M, MacCabe N.

 A revised anchored version of the BPRS-C for childhood psychiatric disorders. 

3. Massachusetts Behavioral Health Partnership (MBHP). 

Clinical outcomes management protocol: performance specifications and phase-in timelines.

http://www.masspartnership.com/provider/outcomesmanagement/Outcomesfiles/Quality%20Alert%2010_%20Final.pdf

4. Shaffer D, Gould MS, Brasic J, et al. 

A Children’s Global Assessment Scale (CGAS). 

5. DelBello MP, Findling RL, Kushner S, et al. 

A pilot controlled trial of topiramate for mania in children and adolescents with bipolar disorder. 

6. Lachar D, Randle SL, Harper R, et al. 

The Brief Psychiatric Rating Scale for Children (BPRS-C): validity and reliability of an anchored version. 

7. Ross RG, Novins D, Farley GK, Adler LE. 

A 1-year open-label trial of olanzapine in school-age children with schizophrenia. 

Multidimensionally impaired: the good news. 

9. Zalsman G, Carmon E, Martin A, Bensason D, Weizman A, Tyano S. 

Effectiveness, safety, and tolerability of risperidone in adolescents with schizophrenia: an open-label study. 

The Brief Psychiatric Rating Scale for Children [Brief Reports and Reviews]. 

12. Gabbard GO, Coyne L, Kennedy LL, et al. 

Two standardized rating scales appeared to be valid and reliable for use at admission and possibly follow-up in a child psychiatry system of care.


April 2009

Moving From A to Z: Successful Implementation of a Statin Switch Program by a Large Physician Group

Cost-Effectiveness of Salmeterol, Fluticasone, and Combination Therapy for COPD

Assessing Outcomes in Child Psychiatry

Does Opioid Therapy Affect Quality of Care for Diabetes Mellitus?

The Transfer of Uninsured Patients From Academic to Community Primary Care Settings

Primary Care Strength Linked to Prevention Programs for Cardiovascular Disease

Pharmaceutical Company Influence on Nonsteroidal Anti-Inflammatory Drug Prescribing Behaviors