Currently Viewing:
The American Journal of Managed Care October 2017
Low-Value Antibiotic Prescribing and Clinical Factors Influencing Patient Satisfaction
Adam L. Sharp, MD, MS; Ernest Shen, PhD; Michael H. Kanter, MD; Laura J. Berman, MPH; and Michael K. Gould, MD, MS
Countywide Physician Organization Learning Collaborative and Changes in Hospitalization Rates
Brent D. Fulton, PhD, MBA; Susan L. Ivey, MD, MHSA; Hector P. Rodriguez, PhD, MPH; and Stephen M. Shortell, PhD, MPH, MBA
Boosting Workplace Wellness Programs With Financial Incentives
Alison Cuellar, PhD; Amelia M. Haviland, PhD; Seth Richards-Shubik, PhD; Anthony T. LoSasso, PhD; Alicia Atwood, MPH; Hilary Wolfendale, MA; Mona Shah, MS; and Kevin G. Volpp, MD, PhD
Currently Reading
Use of Patient-Reported Outcomes and Satisfaction for Quality Assessments
Anne P. Ehlers, MD, MPH; Sara Khor, MS; Amy M. Cizik, PhD, MPH; Jean-Christophe A. Leveque, MD; Neal S. Shonnard, MD; Rod J. Oskouian, Jr, MD; David R. Flum, MD, MPH; and Danielle C. Lavallee, PharmD, PhD
Improving Care Transitions: Complex High-Utilizing Patient Experiences Guide Reform
Nancy Ambrose Gallagher, PhD, APRN-BC; Donna Fox, RN; Carrie Dawson, MS, RN; and Brent C. Williams, MD, MPH
The Option Value of Innovative Treatments for Non–Small Cell Lung Cancer and Renal Cell Carcinoma
Julia Thornton Snider, PhD; Katharine Batt, MD, MSc; Yanyu Wu, PhD; Mahlet Gizaw Tebeka, MS; and Seth Seabury, PhD
Statewide Data Infrastructure Supports Population Health Management: Diabetes Case Study
Craig Jones, MD; Mary Kate Mohlman, PhD; David Jorgenson, MS; Karl Finison, MA; Katie McGee, MS; and Hans Kastensmith
Is Higher Patient Satisfaction Associated With Better Stroke Outcomes?
Xiao Xiang, PhD; Wendy Yi Xu, PhD, MS; and Randi E. Foraker, PhD, MA
A Health Plan's Investigation of Healthy Days and Chronic Conditions
Tristan Cordier, MPH; S. Lane Slabaugh, PharmD; Eric Havens, MA; Jonathan Pena, MS; Gil Haugh, MS; Vipin Gopal, PhD; Andrew Renda, MD; Mona Shah, PhD; and Matthew Zack, MD

Use of Patient-Reported Outcomes and Satisfaction for Quality Assessments

Anne P. Ehlers, MD, MPH; Sara Khor, MS; Amy M. Cizik, PhD, MPH; Jean-Christophe A. Leveque, MD; Neal S. Shonnard, MD; Rod J. Oskouian, Jr, MD; David R. Flum, MD, MPH; and Danielle C. Lavallee, PharmD, PhD
Understanding the relationships among patient-reported outcomes, satisfaction, and quality is the first step in drawing meaningful conclusions that can then be translated into policy.

Objectives: Recent focus on patient-reported outcomes (PROs) has created a new challenge as we learn how to integrate these outcomes into practice along with other quality metrics. We investigated the relationship between PROs and satisfaction among spine surgery patients. We hypothesized that there would be significant disparities between patient satisfaction and PROs at the 1-year postoperative time point.

Study Design: Retrospective cohort study of adults undergoing elective lumbar spine surgery at 12 hospitals participating in the Spine Surgical Care and Outcomes Assessment Program.

Methods: Satisfaction, pain, and function scores were collected at 1 year post operation, along with clinical information, to determine the relationship between PROs and satisfaction at the patient level.

Results: Among 520 patients (mean age = 63 ± 13 years; 47% male), the majority of patients (82%) reported being satisfied with surgery. Satisfaction was associated with both improvement in pain (odds ratio [OR], 1.33; 95% CI, 1.17-1.51) and function (OR, 1.06; 95% CI, 1.04-1.08). However, even among patients who did not improve in pain or function, more than half (59%) reported being satisfied. 

Conclusions: Overall, patients undergoing elective lumbar spine surgery reported being satisfied with outcomes, but the reported responses in PROs were much more variable. As the expectations increase to include PRO measures as valid quality indicators, it is necessary to dedicate time and consideration to understanding the relationships among these measures to support meaningful translations into healthcare policy.

Am J Manag Care. 2017;23(10):618-622
Takeaway Points

The findings from this study indicate that the quality profile of a hospital is highly dependent on the domain of measurement. 
  • Recent focus on patient-reported outcomes (PROs) has created a new challenge as we learn how to integrate these outcomes into practice along with other quality metrics. 
  • Understanding the relationships among PROs, satisfaction, and quality is the first step in drawing meaningful conclusions that can then be translated into policy.
Traditionally, clinical outcomes, such as the rates of adverse events or mortality, served as the metrics by which we measured treatment success or failure. Although these objective measures of quality allow for comparisons across hospitals and physicians, the sole use of clinical outcomes may provide an incomplete appraisal of healthcare quality. Within the last few years, payers, policy makers, and healthcare quality organizations focused on patient-reported outcomes (PROs) as important markers of quality distinct from assessments of patient satisfaction and clinical outcomes.1-4 PROs are defined as measures of health that are directly reported by the patient, rather than interpreted by a clinician or reported in the medical record.5 This newer metric is becoming financially relevant as a result of the Merit-Based Incentive Payment System (MIPS), passed as part of the Medicare Access and CHIP Reauthorization Act of 2015.6 PROs have been proposed by policy makers and payers as a required component of quality assessments and reimbursement schedules.7,8 As clinicians and hospitals are charged with adding PRO measures to established clinical and satisfaction metrics, a challenge exists to integrate PROs in a manner that both improves patient care and provides a meaningful assessment of quality.

Many PRO scores assess a patient’s perception of improvement after an intervention, but others aim to quantify a patient’s level of satisfaction with their care. However, it is not clear that these patient satisfaction measures correlate with the delivery of safe and effective care as defined by clinicians and caregivers. Satisfaction surveys are often criticized for focusing on the “wrong” outcomes, such as the hospital noise level or the variety of food choices offered to patients, rather than on the clinical result of a treatment or intervention.9 Patient satisfaction also may not reflect the total patient experience, as one patient might report high levels of satisfaction with their overall care despite a poor clinical outcome10 and another patient might report dissatisfaction with their care while simultaneously benefiting from a positive clinical outcome. 

Spine surgery represents 1 relevant scenario in which integration of both PRO and clinical outcome measures improves the holistic view of an individual’s outcome following surgery. Functional impairment and pain are often the primary drivers for patients considering spine surgery; therefore, understanding which patients are most likely to see improvement in these symptoms can inform the decision to be made between a surgical and a nonsurgical option. Using PROs to inform direct patient care versus measures for assessing changes in a patient population is increasingly called for in today’s healthcare environment.11,12 In Washington state, healthcare providers are experiencing demands to implement and interpret PROs in spine surgery due to recent policy recommendations from the Bree Collaborative, proposed in 2015. The Bree Collaborative calls for the use of the Oswestry Disability Index (ODI) and the Patient-Reported Outcomes Measurement Information System (PROMIS) 10, measures of functional and general quality of life, respectively, for all patients undergoing single-level lumbar spine fusion procedures. This policy is intended to allow appropriate selection of surgical candidates from the vast population of adults with low back pain and to limit expensive surgical interventions to those patients most likely to benefit from the procedure.13 

Given the differences between PROs, which typically focus on functional ability and pain, and patient satisfaction measures, it is not clear that one or the other necessarily provides a true measure of outcome or that these measures are robust enough to allow for policy and reimbursement decisions. To address this, we investigated the relationship between PRO measures and patient satisfaction at 1 year following spine surgery. We hypothesized that there would be significant disparities between patient satisfaction and PRO measures, suggesting that the use of satisfaction scores as a primary measure of quality is flawed. 


The Comparative Effectiveness Research Translation Network (CERTAIN) is a research and analytic platform aligned with the Spine Surgical Care and Outcomes Assessment Program (Spine SCOAP).11,14,15 Spine SCOAP is a collaborative of hospitals within Washington state whose mission is to aggregate data from hospitals, healthcare providers, and patients to generate evidence-based best-practice measures that can be dispersed among the collaborative to improve the care of patients in the state and beyond. We conducted a retrospective cohort study of adults undergoing elective lumbar spine surgery from 2012 to 2014 at 12 Spine SCOAP hospitals who also participated in CERTAIN PRO data collection. 

Eligible participants completed a survey that included measures of pain, function, and satisfaction both before surgery and at 1 year after. Pain was measured using a 10-point numeric rating scale with “0” representing “no pain” and “10” representing “worst pain possible.” Function was measured using the ODI, a composite measure that grades function on a scale of 0 to 100, with higher scores indicating worse disability.16 Improvement in pain or function was established by a minimally clinically important difference between baseline and follow-up (for pain, a change of at least 2 points from baseline; for function, a change of at least 15 points from baseline).17 We measured satisfaction using a single question: “How satisfied are you with the overall result of your spine operation so far?” The satisfaction question was scored using a 4-point modified Likert scale (Table 1).

The University of Washington Human Subjects Division approved the retrospective review of these data. All analyses were carried out using Stata version 11 (STATA Corp; College Station, Texas).

We analyzed patient-reported satisfaction, pain, and function at 1 year post operation to determine the association between patient characteristics and a positive outcome in any or all domains. Patient characteristics were obtained through Spine SCOAP and included information about sociodemographics, underlying diagnosis (eg, spinal stenosis, herniated disc), and operative characteristics (eg, fusion). An invasiveness index based on the type of intervention at each vertebral level as well as the number of operated levels was included, as this has been shown to be correlated with clinical outcomes in spine surgery patients.18,19 The American Society of Anesthesiologists Physical Classification System score was included as an additional measure of physical status at the time of the operation. This is a commonly used score applied to every patient undergoing surgery and is intended to provide an additional description of patient risk.20 We selected the 1-year postoperative time point because PROs and satisfaction in the early postoperative period are not necessarily reflective of findings at 1 year,21,22 which are felt to be a more reliable indicator of long-term satisfaction. 

To determine the relationship among the 3 domains, we used a Poisson regression model stratified by pain and function improvement that was predictive of satisfaction (somewhat or very satisfied) at 12 months following spine surgery. A subset of patients (n = 69; 13%) did not have clinical information available through Spine SCOAP and was excluded from the regression analysis. To account for the fact that other patient characteristics are known to correlate with overall satisfaction,23 we included sociodemographic factors, diagnostic factors, and operative characteristics as measured by SCOAP in the regression model. The resulting model was clustered by hospital to account for correlated data. In cases such as this, where multiple observations are drawn from the same hospital, there is a concern that these results from within each individual hospital are correlated due to extrinsic factors that are not measured and that this effect varies between hospitals. Failure to account for correlation can result in a biased estimate and an incorrect inference. 


At the 1-year follow-up point, 520 patients (mean age = 63 ± 13 years; 47% male) had complete baseline and follow-up survey data and 451 (87%) had clinical information available from Spine SCOAP. More than half of patients reported that they were very satisfied (n = 301; 58%) while one-fourth of patients reported that they were somewhat satisfied (n = 124; 24%) with their spine surgery result. Table 2 demonstrates the interaction among patient-reported pain and function improvement and patient satisfaction. Patients with a higher magnitude of pain improvement more often reported that they were somewhat or very satisfied: 62% patients with no pain improvement or worse pain were satisfied, whereas 96% of those with 5 or more points of pain improvement were satisfied. We found a similar trend in satisfaction and function: 61% of those with no functional improvement or worse function reported being satisfied and 80% of those whose function improved more than 30 points reported being satisfied. Interestingly, even among patients who had no change in their symptoms or reported that they were worse, 59% of patients were still somewhat or very satisfied.

In our stratified analysis, adjusting for patient characteristics, patients who had no improvement in back pain were 1.7 times (95% CI, 1.27-2.16) and 2.3 times (95% CI, 1.71-3.11) more likely to be satisfied if they experienced a minor improvement in function (ODI score reduced 15-29 points) or major improvement in function (ODI score reduced ≥30 points), respectively, compared with patients who had no improvement or worse function. Of the patients with improved back pain at 1 year, a mild improvement in function was associated with 1.3 times (95% CI, 1.00-1.71) the likelihood of being satisfied, and a major improvement in function was associated with 1.4 times (95% CI, 1.08-1.81) the likelihood of being satisfied compared with patients who had no improvement or worse function outcomes (Table 3).


In this analysis, we sought to determine how PRO measures compare with other quality measures, such as patient-reported satisfaction. We analyzed data from a statewide clinical registry encompassing a collaborative of hospitals in Washington state that allowed for an assessment of both hospital-level and patient-level outcomes. The majority of patients reported being satisfied with their spine surgery result, and we found a strong association between patient-reported improvements in pain and function and satisfaction. Despite this understandable finding, more than half of the patients with no improvement or worse outcomes in pain or function were satisfied with their surgery. This incongruence suggests that satisfaction ratings may be based on nonclinical aspects of care not captured by this survey. Reasons for this incongruence may be related to the patient’s perception that their care was especially respectful or well coordinated. Conversely, there was a group of patients who did improve but were not satisfied, which may indicate that they had other expectations beyond pain or functional improvement. We can conclude that the “quality” profile of a hospital is highly dependent on the domain of measurement and that understanding the items measured by a particular instrument is critical for interpretation of results.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up