Currently Viewing:
The American Journal of Managed Care October 2016
Cost-Effectiveness of a Statewide Falls Prevention Program in Pennsylvania: Healthy Steps for Older Adults
Steven M. Albert, PhD; Jonathan Raviotta, MPH; Chyongchiou J. Lin, PhD; Offer Edelstein, PhD; and Kenneth J. Smith, MD
Economic Value of Pharmacist-Led Medication Reconciliation for Reducing Medication Errors After Hospital Discharge
Mehdi Najafzadeh, PhD; Jeffrey L. Schnipper, MD, MPH; William H. Shrank, MD, MSHS; Steven Kymes, PhD; Troyen A. Brennan, MD, JD, MPH; and Niteesh K. Choudhry, MD, PhD
Currently Reading
Benchmarking Health-Related Quality-of-Life Data From a Clinical Setting
Janel Hanmer, MD, PhD; Rachel Hess, MD, MS; Sarah Sullivan, BS; Lan Yu, PhD; Winifred Teuteberg, MD; Jeffrey Teuteberg, MD; and Dio Kavalieratos, PhD
Connected Care: Improving Outcomes for Adults With Serious Mental Illness
James M. Schuster, MD, MBA; Suzanne M. Kinsky, MPH, PhD; Jung Y. Kim, MPH; Jane N. Kogan, PhD; Allison Hamblin, MSPH; Cara Nikolajski, MPH; and John Lovelace, MS
A Call for a Statewide Medication Reconciliation Program
Elisabeth Askin, MD, and David Margolius, MD
Postdischarge Telephone Calls by Hospitalists as a Transitional Care Strategy
Sarah A. Stella, MD; Angela Keniston, MSPH; Maria G. Frank, MD; Dan Heppe, MD; Katarzyna Mastalerz, MD; Jason Lones, BA; David Brody, MD; Richard K. Albert, MD; and Marisha Burden, MD
Mortality Following Hip Fracture in Chinese, Japanese, and Filipina Women
Minal C. Patel, MD; Malini Chandra, MS, MBA; and Joan C. Lo, MD
Estimating the Social Value of G-CSF Therapies in the United States
Jacqueline Vanderpuye-Orgle, PhD; Alison Sexton Ward, PhD; Caroline Huber, MPH; Chelsey Kamson, BS; and Anupam B. Jena, MD, PhD
Periodic Health Examinations and Missed Opportunities Among Patients Likely Needing Mental Health Care
Ming Tai-Seale, PhD; Laura A. Hatfield, PhD; Caroline J. Wilson, MSc; Cheryl D. Stults, PhD; Thomas G. McGuire, PhD; Lisa C. Diamond, MD; Richard M. Frankel, PhD; Lisa MacLean, MD; Ashley Stone, MPH; and Jennifer Elston Lafata, PhD
Does Medicare Managed Care Reduce Racial/Ethnic Disparities in Diabetes Preventive Care and Healthcare Expenditures?
Elham Mahmoudi, PhD; Wassim Tarraf, PhD; Brianna L. Maroukis, BS; and Helen G. Levy, PhD

Benchmarking Health-Related Quality-of-Life Data From a Clinical Setting

Janel Hanmer, MD, PhD; Rachel Hess, MD, MS; Sarah Sullivan, BS; Lan Yu, PhD; Winifred Teuteberg, MD; Jeffrey Teuteberg, MD; and Dio Kavalieratos, PhD
Health-related quality-of-life data are often collected during routine clinical care. We present a method to create nationally representative benchmarks for clinical subspecialties.
Demographics from the samples are presented in Table 1. The demographics of the Cardiology Local and the All MEPS samples are quite different; the Cardiology Local sample has a substantially lower proportion of females (42% vs 52%), lower proportion of racial minorities (8% minorities vs 18% minorities), and substantially higher mean CCI score (1.74 vs 0.62). The Cardiology MEPS subsample (43% female, 13% minorities; mean CCI score of 1.57) is a closer fit to the Cardiology Local sample with these demographic parameters than the All MEPS subsample. However, the distribution of specific comorbidities is still different between the Cardiology Local and Cardiology MEPS samples, with the Cardiology Local sample having more than twice the reported rate of congestive heart failure and cerebrovascular disease and half the rate of acute myocardial infarction.

Primary Outcome: SF-6D

When the Cardiology Local sample is compared with the All MEPS sample, there are statistically significant differences between mean SF-6D scores in the 65-to-74-years and 75-years-or-older age groups; however, these do not reach a clinically important difference of 0.0427 (Table 2). In contrast, when the Cardiology Local sample is compared with the Cardiology MEPS sample, there are statistically significant differences between mean SF-6D scores in 4 of the 5 age groups (45-54, 55-64, 65-74, ≥75). In 3 of these age groups (45-64, 55-64, and 65-75), the Cardiology Local sample has a clinically important improvement over Cardiology MEPS.

In the regression-based sensitivity analyses combining Cardiology Local and Cardiology MEPS data, SF-6D scores were statistically important and had a clinically important improvement in the Cardiology Local sample for the same 4 age groups (Table 3). For example, in those aged 55 to 64, Cardiology Local SF-6D scores were 0.059 higher than the Cardiology MEPS scores after adjusting for sex and number of comorbid conditions. As expected, as the number of comorbid conditions increased, the SF-6D scores decreased. SF-6D scores were also lower for females than males.

Secondary Outcomes: MCS and PCS

The MCS mean scores in the Cardiology Local sample are statistically, although not clinically, higher than the All MEPS sample in 4 age groups (18-44, 55-64, 65-74, ≥75) (Table 4). There are no statistically significant differences in the MCS values of the Cardiology Local sample compared with the Cardiology MEPS sample. In the sensitivity analyses combining Cardiology Local and Cardiology MEPS data, MCS scores are statistically different in 3 age strata (55-64, 65-74, ≥75), with the Cardiology Local sample reaching clinically important differences (of at least 5) in 2 of the age groups (55-64 and 65-74) (Table 5).

For PCS mean scores stratified by age, the Cardiology Local sample is statistically different from the All MEPS sample in 2 age groups (18-44, 45-54), and the Cardiology Local sample has clinically importantly lower scores than All MEPS in these groups (Table 4). There are 3 statistically different values when comparing the Cardiology Local sample with the Cardiology MEPS sample (age groups 45-54, 65-74, ≥75), and the differences reach clinical significance (improvement) in the 45-to-54 age group. In the sensitivity analyses, combining Cardiology Local and Cardiology MEPS data, PCS scores are statistically different in 4 age strata (45-54, 55-64, 65-74, ≥75), with the Cardiology Local sample reaching improved clinically important differences in these 4 age groups (Table 5).


This report illustrates the use of MEPS to generate nationally representative HRQoL benchmark scores for patients receiving care from a cardiologist. We used these benchmark scores to add an important dimension of understanding to the routine HRQoL data collected in cardiology clinics of a large healthcare system. Compared with the national benchmarks, the local cardiology clinics had both statistically significant and clinically important improvements in health preference scores for those aged 45 to 74. Our system did not have any scores that were statistically or clinically lower than the national benchmarks.

To better understand why overall health preference scores for local cardiology clinics were different than the national benchmarks, we also calculated MCS and PCS scores from the PROs. These analyses showed fewer differences than the overall health preference score. Compared with the national benchmarks, the improvements in the local cardiology clinics were generally larger for PCS than MCS scores. These findings suggest that there is greater room for improvement in mental health outcomes than physical health outcomes; focusing quality improvement on mental health outcomes within our cardiology clinics may have a great impact on patient HRQoL. Findings such as these illustrate the power of using generic health measurement, even in subspecialty care.

This study also shows the limitations of comparing normative values in clinic populations to the general population. There are several generic HRQoL measures that have been included in nationally representative datasets, such as MEPS, the Joint Canada/United States Survey of Health, the National Health Measurement Study, and the US Valuation of the EuroQol 5-Dimension (EQ-5D) Health States Survey28; however, these datasets are primarily filled with healthy respondents. When the Cardiology Local sample was compared with the All MEPS sample in this study, there were no clinically important differences in health preference scores or MCS scores. The Cardiology Local sample had clinically importantly worse PCS scores in 2 of the age groups (18-44 and 45-54). Compared with the individuals in MEPS who reported seeing a cardiologist, the Cardiology Local sample was the same as or better than, the Cardiology MEPS benchmarks.

MEPS is a rich dataset with information about many medical expenditures, including subspecialty visits, and it also includes generic health preference measures, such as the EQ-5D from 2000 to 2003, the SF-12v1 from 2000 to 2002, and the SF-12v2 from 2003 to present. The analysis shown in this report could be used to create similar benchmarks for other clinic or patient populations for which the MEPS sampling would reflect the local sample of interest. For example, it would be possible to select respondents in MEPS who have been to an endocrinology clinic or who report expenditures for thyroid disorders. Selecting rare conditions would reduce the sample size, although analysts could consider combining multiple years of MEPS data for more respondents. MEPS is a representative sample of the US civilian noninstitutionalized population, explicitly excluding institutionalized individuals in prisons or nursing homes and US citizens in the military. We expected these exclusions to bias our results in the opposite direction that was found, as the clinics in our healthcare system serve individuals in nursing homes.

Comparing local cardiology clinic samples to MEPS is limited by differing methods of eliciting the comorbid conditions appearing in each dataset. Comorbid conditions in the local dataset were included in the problem list or medical history in the EHR; conditions are physician-coded and do not have to be active at the time. Comorbid conditions in MEPS are reported by the respondent for the Household Component field and then recorded by the interviewer as verbatim text. This text was then coded by professional coders to fully specified ICD-9-CM codes, including medical condition and V codes. Although codes are verified, and error-rate percentages for any coder are low, the Agency for Healthcare Research and Quality recommends against assuming that household respondents have high accuracy in reporting condition data.29 More work is necessary to determine the comparability of these methods to elicit comorbid conditions.

Usual techniques for case-mix adjustment include age, sex, and comorbidity information.30 Our primary analyses compared age strata between the Cardiology Local sample and the Cardiology MEPS subsample because the number of comorbid conditions and the proportion of females were similar in each data source. Given the small strata sample sizes, we did not further stratify by sex. Given the small number of nonwhite respondents, we were unable to look for differences by race, although there are well-known differences in HRQoL reports by race in the general population.31

The stratified analysis allowed us to incorporate MEPS weights so the estimates are representative of the US noninstitutionalized civilian population. For a sensitivity analysis, we combined the Cardiology Local and Cardiology MEPS samples, which adjusted for sex and number of comorbid conditions within the age strata. These analyses gave similar results to the stratified analyses but did not allow the use of MEPS weights. Techniques should be developed to include case adjustment and weighting when comparing clinic samples to national normative values.


Our work demonstrates that collection and interpretation of PROs in a clinical setting is feasible. Furthermore, incorporating PROs provides information that complements traditional clinical measures by emphasizing patient-centered care. We illustrate a technique to create benchmarks to allow interpretation of patient-reported outcomes in a clinic setting. Our findings suggest the clinical practices in our sample are achieving or exceeding benchmarks created from nationally representative data. They also provide clues about areas of improvement that could be high yield within these practices. If used more broadly, PROs could add an important dimension to our understanding of high-value care. 

Author Affiliations: Division of General Internal Medicine (JH, SS, LY, WT, DK) and Division of Cardiology (JT), University of Pittsburgh Medical Center, Pittsburgh, PA; Health System Innovation and Research, University of Utah Schools of the Health Sciences (RH), Salt Lake City, UT.

Source of Funding: PRIcis is funded by the Chief Medical and Scientific Officer of the University of Pittsburgh Medical Center. Dr Hanmer is supported by the National Institutes of Health through Grant Number KL2 TR000146. Dr Kavalieratos is supported by K12HS022989 from AHRQ, and a Junior Faculty Career Development Award from the National Palliative Care Research Center. 

Author Disclosures: Dr J. Teuteberg is a consultant for HeartWare, CareDx, Abiomed, Acorda Therapeutics; he has also received lecture fees for speaking at the invitation of a commercial sponsor (HeartWare, CareDx). The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JH, RH, DK); acquisition of data (RH, SS, WT); analysis and interpretation of data (JH, DK, SS, JT, WT, LY); drafting of the manuscript (JH, JT, LY); critical revision of the manuscript for important intellectual content (JH, RH, DK, JT, WT); statistical analysis (JH, SS, LY); obtaining funding (RH).

Address Correspondence to: Janel Hanmer, MD, PhD, Department of General Internal Medicine, University of Pittsburgh Medical Center, 230 McKee Pl, Ste 600, Pittsburgh, PA 15213. E-mail:  

1. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life: a conceptual model of patient outcomes. JAMA. 1995;273(1):59-65.

2. Lohr KN. Quality measures: patient-reported outcomes for quality improvement of clinical Practice. Agency for Healthcare Research and Quality website. Published May 14, 2012. Accessed January 14, 2015.

3. Osoba D. Translating the science of patient-reported outcomes assessment into clinical practice. J Natl Cancer Inst Monogr. 2007;(37):5-11.

4. Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res. 2012;21(8):1305-1314. doi: 10.1007/s11136-011-0054-x.

5. Spertus JA. Evolving applications for patient-centered health status measures. Circulation. 2008;118(20):2103-2110. doi: 10.1161/CIRCULATIONAHA.107.747568.

6. Fryback DG, Dunham NC, Palta M, et al. US norms for six generic health-related quality-of-life indexes from the National Health Measurement study. Med Care. 2007;45(12):1162-1170.

7. McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. 3rd ed. New York, NY: Oxford University Press; 2006.

8. Torrance GW. Measurement of health state utilities for economic appraisal. J Health Econ. 1986;5(1):1-30.

9. Hanmer J, Lawrence WF, Anderson JP, Kaplan RM, Fryback DG. Report of nationally representative values for the noninstitutionalized US adult population for 7 health-related quality-of-life scores. Med Decis Making. 2006;26(4):391-400.

10. Luo N, Johnson JA, Shaw JW, Feeny D, Coons SJ. Self-reported health status of the general adult US population as assessed by the EQ-5D and Health Utilities Index. Med Care. 2005;43(11):1078-1086.

11. Sullivan PW, Ghushchyan V. Preference-based EQ-5D index scores for chronic conditions in the United States. Med Decis Making. 2006;26(4):410-420.

12. Iezzoni LI. The risks of risk adjustment. JAMA. 1997;278(19):1600-1607.

13. Ettorchi-Tardy A, Levif M, Michel P. Benchmarking: a method for continuous quality improvement in health. Healthc Policy. 2012;7(4):e101-e119.

14. Module 7. measuring and benchmarking clinical performance. Agency for Healthcare Research and Quality website. Accessed January 14, 2015.

15. Chan PS, Oetgen WJ, Buchanan D, et al. Cardiac performance measure compliance in outpatients: the American College of Cardiology and National Cardiovascular Data Registry’s PINNACLE (Practice Innovation And Clinical Excellence) program. J Am Coll Cardiol. 2010;56(1):8-14. doi: 10.1016/j.jacc.2010.03.043.

16. Medical Expenditure Plan Survey (MEPS). Agency for Healthcare Research and Quality website. Accessed January 14, 2014.

17. Ware JE, Kosinski M, Keller SD. How to Score the SF-12 Physical and Mental Health Summary Scales. 3rd ed. Boston, MA: The Health Institute, New England Medical Center, 1998.

18. Hays RD, Sherbourne CD, Mazel RM. The RAND 36-item health survey 1.0. Health Econ. 1993;2(3):217-227.

19. Brazier JE, Roberts J. The estimation of a preference-based measure of health from the SF-12. Med Care. 2004;42(9):851-859.

20. Torrance GW. Measurement of health state utilities for economic appraisal: a review. J Health Econ. 1986;5(1):1-30.

21. Gandek B, Ware JE, Aaronson NK, et al. Cross-validation of item selection and scoring for the SF-12 Health Survey in nine countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51(11):1171–1178.

22. Ware JE Jr, Kosinski M, Keller SD. A 12-item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233.

23. 36-Item Short Form Survey scoring instructions. RAND Health website. Accessed April 2016.

24. 12-Item Short Form Survey. RAND Health website. Accessed April 2016.

25. D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49(12):1429-1433.

26. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10(4):407-415.

27. Walters SJ, Brazier JE. Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Qual Life Res. 2005;14(6):1523-1532.

28. Hanmer J, Hays RD, Fryback DG. Mode of administration is important in US national estimates of health-related quality of life. Med Care. 2007;45(12):1171-1179.

29. Medical Expenditure Panel Survey—MEPS HC-146: 2011 medical conditions. Agency for Healthcare Research and Quality website. Published September 2013. Accessed January 14, 2015.

30. Pope GC, Kautter J, Ingber MJ, Freeman S, Sekar R, Newhart C. Evaluation of the CMS-HCC Risk Adjustment Model [final report]. CMS website. Accessed January 14, 2015.

31. Robert SA, Cherepanov D, Palta M, Dunham NC, Feeny D, Fryback DG. Socioeconomic status and age variations in health-related quality of life: results from the National Health Measurement Study. J Gerontol B Psychol Sci Soc Sci. 2009;64(3):378-389. doi: 10.1093/geronb/gbp012. 
Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up