Currently Viewing:
The American Journal of Managed Care October 2017
Low-Value Antibiotic Prescribing and Clinical Factors Influencing Patient Satisfaction
Adam L. Sharp, MD, MS; Ernest Shen, PhD; Michael H. Kanter, MD; Laura J. Berman, MPH; and Michael K. Gould, MD, MS
Countywide Physician Organization Learning Collaborative and Changes in Hospitalization Rates
Brent D. Fulton, PhD, MBA; Susan L. Ivey, MD, MHSA; Hector P. Rodriguez, PhD, MPH; and Stephen M. Shortell, PhD, MPH, MBA
Boosting Workplace Wellness Programs With Financial Incentives
Alison Cuellar, PhD; Amelia M. Haviland, PhD; Seth Richards-Shubik, PhD; Anthony T. LoSasso, PhD; Alicia Atwood, MPH; Hilary Wolfendale, MA; Mona Shah, MS; and Kevin G. Volpp, MD, PhD
Currently Reading
Use of Patient-Reported Outcomes and Satisfaction for Quality Assessments
Anne P. Ehlers, MD, MPH; Sara Khor, MS; Amy M. Cizik, PhD, MPH; Jean-Christophe A. Leveque, MD; Neal S. Shonnard, MD; Rod J. Oskouian, Jr, MD; David R. Flum, MD, MPH; and Danielle C. Lavallee, PharmD,
Improving Care Transitions: Complex High-Utilizing Patient Experiences Guide Reform
Nancy Ambrose Gallagher, PhD, APRN-BC; Donna Fox, RN; Carrie Dawson, MS, RN; and Brent C. Williams, MD, MPH
The Option Value of Innovative Treatments for Non–Small Cell Lung Cancer and Renal Cell Carcinoma
Julia Thornton Snider, PhD; Katharine Batt, MD, MSc; Yanyu Wu, PhD; Mahlet Gizaw Tebeka, MS; and Seth Seabury, PhD
Statewide Data Infrastructure Supports Population Health Management: Diabetes Case Study
Craig Jones, MD; Mary Kate Mohlman, PhD; David Jorgenson, MS; Karl Finison, MA; Katie McGee, MS; and Hans Kastensmith
Is Higher Patient Satisfaction Associated With Better Stroke Outcomes?
Xiao Xiang, PhD; Wendy Yi Xu, PhD, MS; and Randi E. Foraker, PhD, MA
A Health Plan's Investigation of Healthy Days and Chronic Conditions
Tristan Cordier, MPH; S. Lane Slabaugh, PharmD; Eric Havens, MA; Jonathan Pena, MS; Gil Haugh, MS; Vipin Gopal, PhD; Andrew Renda, MD; Mona Shah, PhD; and Matthew Zack, MD

Use of Patient-Reported Outcomes and Satisfaction for Quality Assessments

Anne P. Ehlers, MD, MPH; Sara Khor, MS; Amy M. Cizik, PhD, MPH; Jean-Christophe A. Leveque, MD; Neal S. Shonnard, MD; Rod J. Oskouian, Jr, MD; David R. Flum, MD, MPH; and Danielle C. Lavallee, PharmD,
Understanding the relationships among patient-reported outcomes, satisfaction, and quality is the first step in drawing meaningful conclusions that can then be translated into policy.
Previous studies have explored the relationship between patient satisfaction and other outcome measures (eg, clinical outcomes, patient-reported functional outcomes), but conclusions have varied among studies. In one prospective study, Godil et al reported that patient satisfaction measures at 90 days post operation were not correlated with clinical outcomes, such as complications and readmissions, and as such, they should not be used as a proxy for overall quality.24 These findings were confirmed in a subsequent review article.8 The time frame of measurement is important, as well: PROs 3 months post operation are not necessarily predictive of PROs at 1 year.21,22 This latter finding may be clinically relevant as satisfaction metrics are frequently measured at time points close to the date of care delivery, rather than at later time points, which might better reflect long-term improvements or outcomes. These studies highlight the unresolved issues related to the use of PROs and satisfaction data in quality assessments.

More generally, there is evidence that satisfied patients are more likely to utilize healthcare resources and thereby incur costs, but it is not clear that the increased healthcare utilization is related to better outcomes.25 Among surgical patients, one study found no association between satisfaction and measures of quality as defined by the Surgical Care Improvement Program, such as adequate antibiotic prophylaxis before surgery,26 again pointing to the concern that patients base satisfaction ratings on nonclinical factors that are not reflected in clinical or administrative data. Our analysis found a correlation between patient-reported improvement in pain and function and satisfaction in the patient-level analysis, but there was a relatively large proportion of individuals who reported being somewhat or very satisfied who also had a negative outcome in either pain or function. Potential reasons for this include appreciation of the nursing care that the patient received, the organization of their care, or attentiveness of their care team as a whole. Future work should focus on understanding the domains of care that are most correlated with satisfaction.


The survey utilized in this study was created under the auspices of quality improvement to help surgeons and hospitals understand more about their patients. The evidence is still emerging as to the complete set of questions that are relevant to support patient- and hospital-level decision making, as well as to provide quality assessments. One limitation of this analysis is the potential inability to capture all domains that are important to patients, and thus it may be biased to those areas deemed clinically relevant to the survey creators. A second limitation is the restriction of the analysis to lumbar surgery patients alone, meaning the results may not be reflective of all clinical conditions. We encourage future research endeavors that focus on how measures of the patient experience correlate in other health conditions. Because this analysis included only those with complete data, these results may not be representative of outcomes from nonresponders. We do not have information regarding nonresponders, and it may be that patients who respond to surveys are systematically different from those who do not. It may also be that improvement in pain or function is representative of better patient selection. Although this is outside the scope of this research, future work should focus on understanding which patients will benefit most from spine surgery. Finally, the survey supported through CERTAIN is currently only available in English, which may further bias our results. 


In Washington state, current policy and payment recommendations require health systems to capture PRO data for patients undergoing lumbar spine surgery. It is imperative that we understand how to interpret this PRO information to ensure that patients get the right care at the right time. With the ongoing concerns about PRO implementation, collection, interpretation, and application,27,28 it is necessary to focus on how these important data can inform decisions about patient care. Successful implementation depends not only on the available infrastructure for collection and analysis, but also on the commitment of healthcare providers to the concept that PROs add value to the ongoing quality discussion. Without this focus, we risk more measurement and more data without the ability to transform patient care. There is considerable work to be done before we fully understand the appropriate role of PROs in healthcare. 

The implications of this work are far-reaching, especially with the recent passage of MIPS, which emphasizes quality and value over volume.6 Results such as this may serve to inform policy makers who are deciding on the most appropriate measures to use in quality assessments. Understanding the relationships among PROs, satisfaction, and general quality is the first step in drawing meaningful conclusions that can then be translated to policy. From a patient’s perspective, our study is important because it may help patients better interpret the significance of a particular physician or hospital having high satisfaction ratings. Furthermore, this work requires the inclusion of patients, providers, payers, and policy makers to ensure that we are selecting the appropriate domains on which to judge quality.

Author Affiliations: Department of Surgery (APE, DRF, DCL), and Surgical Outcomes Research Center (SK), and Department of Orthopaedics and Sports Medicine (AMC), University of Washington, Seattle, WA; Neuroscience Institute, Virginia Mason Medical Center (J-CAL), Seattle, WA; Rainer Orthopedic Institute, Proliance Surgeons Inc (NSS), Puyallup, WA; Swedish Neuroscience Institute, Swedish Medical Center (RJO), Seattle, WA.

Source of Funding: Dr Ehlers was supported by a training grant from the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number T32DK070555. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Data used for this project were collected with grant number R01HS020025 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The Surgical Care and Outcomes Assessment Program (SCOAP) is a Coordinated Quality Improvement Program of the Foundation for Health Care Quality. CERTAIN is a program of the University of Washington, the academic research and development partner of SCOAP.

Author Disclosures: Dr Cizik is currently supported by a grant from the NIH (grant number R21AR068009) investigating nonresponse in spine surgery. The data included in this manuscript submission are not reflective of any data collected as part of the disclosed study. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. Portions of this work were presented at Concordium (September 2015; Washington, DC) and the International Society for Quality of Life Annual Conference (October 2015; Vancouver, BC, Canada).

Authorship Information: Concept and design (APE, SK, NSS, DRF, DCL); acquisition of data (SK, NSS, RJO, DRF, DCL); analysis and interpretation of data (APE, SK, AMC, J-CAL, NSS, RJO, DRF, DCL); drafting of the manuscript (APE, J-CAL, DRF, DCL); critical revision of the manuscript for important intellectual content (APE, SK, AMC, J-CAL, DRF, DCL); statistical analysis (SK, DCL); provision of patients or study materials (NSS, RJO); obtaining funding (DRF, DCL); administrative, technical, or logistic support (AMC, NSS, RJO, DCL); and supervision (AMC, NSS, DCL). 

Address Correspondence to: Anne P. Ehlers, MD, MPH, University of Washington, 1107 NE 45th St, Ste 502, Box 354808, Seattle, WA 98195. E-mail: 

1. 2012 Annual Progress Report to Congress: National Strategy for Quality Improvement in Health Care. Agency for Healthcare Research and Quality website. Published April 2012. Accessed May 2016.

2. Patient Protection and Affordable Care Act, 2010. Government Publishing Office website. Accessed May 2016.

3. Institute of Medicine; Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. doi:

4. NQF-endorsed measures for person- and family-centered care: phase I technical report. National Quality Forum website. Published March 4, 2015. Accessed May 2016.

5. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. FDA website. Published December 2009. Accessed May 2016.

6. H.R.2.—Medicare Access and CHIP Reauthorization Act of 2015. website. Accessed May 2016.

7. HCAHPS fact sheet. Hospital Consumer Assessment of Healthcare Providers and Systems website. Published August 2013. Accessed February 2016.

8. Tevis SE, Schmocker RK, Kennedy GD. Can patients reliably identify safe, high quality care? J Hosp Adm. 2014;3(5):150-160.

9. Casarett D. Lessons in end-of-life care from the V.A. The New York Times website. Published November 11, 2015. Accessed May 2016.

10. Robbins A. The problem with satisfied patients. The Atlantic website. Published April 17, 2015. Accessed May 2016.

11. Lee MJ, Shonnard N, Farrokhi F, et al; Spine SCOAP-CERTAIN Collaborative. The Spine Surgical Care and Outcomes Assessment Program (Spine SCOAP): a surgeon-led approach to quality and safety. Spine (Phila Pa 1976). 2015;40(5):332-341. doi: 10.1097/BRS.0000000000000750.

12. Weinstein JN, Lurie JD, Tosteson TD, et al. Surgical vs nonoperative treatment for lumbar disk herniation: the Spine Patient Outcomes Research Trial (SPORT) observational cohort. JAMA. 2006;296(20):2451-2459.

13. Spine/low back pain topic: report & recommendations. The Bree Collaborative website. Published November 2013. Accessed May 2016.

14. Devine EB, Alfonso-Cristancho R, Devlin A, et al; CERTAIN Collaborative. A model for incorporating patient and stakeholder voices in a learning health care network: Washington State’s Comparative Effectiveness Research Translation Network. J Clin Epidemiol. 2013;66(suppl 8):S122-S129. doi: 10.1016/j.jclinepi.2013.04.007.

15. Flum DR, Fisher N, Thompson J, Marcus-Smith M, Florence M, Pellegrini CA. Washington state’s approach to variability in surgical processes/outcomes: Surgical Clinical Outcomes Assessment Program (SCOAP). Surgery. 2005;138(5):821-828. doi: 10.1016/j.surg.2005.07.026.

16. Fairbank JC, Pynsent PB. The Oswestry Disability Index. Spine (Phila Pa 1976). 2000;25(22):2940-2952; discussion 2952.

17. Parker SL, Adogwa O, Paul AR, et al. Utility of minimum clinically important difference in assessing pain, disability, and health state after transforaminal lumbar interbody fusion for degenerative lumbar spondylolisthesis. J Neurosurg Spine. 2011;14(5):598-604. doi: 10.3171/2010.12.SPINE10472.

18. Cizik AM, Lee MJ, Martin BI, et al. Using the spine surgical invasiveness index to identify risk of surgical site infection: a multivariate analysis. J Bone Joint Surg Am. 2012;94(4):335-342. doi: 10.2106/JBJS.J.01084.

19. Mirza SK, Deyo RA, Heagerty PJ, Turner JA, Lee LA, Goodkin R. Towards standardized measurement of adverse events in spine surgery: conceptual model and pilot evaluation. BMC Musculoskelet Disord. 2006;7:53. doi: 10.1186/1471-2474-7-53.

20. ASA Physical Classification System. American Society of Anesthesiologists website. Published October 15, 2014. Accessed May 2016.

21. Parker SL, Asher AL, Godil SS, Devin CJ, McGirt MJ. Patient-reported outcomes 3 months after spine surgery: is it an accurate predictor of 12-month outcome in real-world registry platforms? Neurosurg Focus. 2015;39(6):E17. doi: 10.3171/2015.9.FOCUS15356.

22. McGregor AH, Doré CJ, Morris TP. An exploration of patients’ expectation of and satisfaction with surgical outcome. Eur Spine J. 2013;22(12):2836-2844. doi: 10.1007/s00586-013-2971-6.

23. Bible JE, Kay HF, Shau DN, O’Neill KR, Segebarth PB, Devin CJ. What patient characteristics could potentially affect patient satisfaction scores during spine clinic? Spine (Phila Pa 1976). 2015;40(13):1039-1044. doi: 10.1097/BRS.0000000000000912.

24. Godil SS, Parker S, Zuckerman SL, et al. Determining the quality and effectiveness of surgical spine care: patient satisfaction is not a valid proxy. Spine J. 2013;13(9):1006-1012. doi: 10.1016/j.spinee.2013.04.008.

25. Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Arch Intern Med. 2012;172(5):405-411. doi: 10.1001/archinternmed.2011.1662.

26. Lyu H, Wick EC, Housman M, Freischlag JA, Makary MA. Patient satisfaction as a possible indicator of quality surgical care. JAMA Surg. 2013;148(4):362-367. doi: 10.1001/2013.jamasurg.270.

27. Bilimoria KY, Cella D, Butt Z. Current challenges in using patient-reported outcomes for surgical care and performance measurement: everybody wants to hear from the patient, but are we ready to listen? JAMA Surg. 2014;149(6):505-506. doi: 10.1001/jamasurg.2013.5285.

28. Lavallee DC, Chenok KE, Love RM, et al. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood). 2016;35(4):575-582. doi: 10.1377/hlthaff.2015.1362.
Copyright AJMC 2006-2018 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!