Currently Viewing:
The American Journal of Managed Care May 2019
Evaluation of Value-Based Insurance Design for Primary Care
Qinli Ma, PhD; Gosia Sylwestrzak, MA; Manish Oza, MD; Lorraine Garneau; and Andrea R. DeVries, PhD
The Presurgical Episode: An Untapped Opportunity to Improve Value
Erika D. Sears, MD, MS; Rodney A. Hayward, MD; and Eve A. Kerr, MD, MPH
Clarification of References to Medication Adherence Scale
Open Doors to Primary Care Should Add a “Screen” to Reduce Low-Value Care
Betsy Q. Cliff, MS; and A. Mark Fendrick, MD
From the Editorial Board: Daniel B. Wolfson, MHSA
Daniel B. Wolfson, MHSA
Cost-Effectiveness of DPP-4 Inhibitor and SGLT2 Inhibitor Combination Therapy for Type 2 Diabetes
Manjiri Pawaskar, PhD; S. Pinar Bilir, MS; Stacey Kowal, MS; Claudio Gonzalez, MD; Swapnil Rajpathak, MD; and Glenn Davies, DrPH
Improving Provider Directory Accuracy: Can Machine-Readable Directories Help?
Michael Adelberg, MA, MPP; Austin Frakt, PhD; Daniel Polsky, PhD; and Michelle Kitchman Strollo, DrPH, MHS
Electronic Consults for Improving Specialty Care Access for Veterans
David E. Winchester, MD, MS; Anita Wokhlu, MD; Juan Vilaro, MD; Anthony A. Bavry, MD, MPH; Ki Park, MD; Calvin Choi, MD; Mark Panna, MD; Michael Kaufmann, MD; Matthew McKillop, MD; and Carsten Schmalfuss, MD
Potential Impact of Pharmaceutical Industry Rebates on Medication Adherence
Leah L. Zullig, PhD; Bradi B. Granger, PhD; Helene Vilme, DrPH; Megan M. Oakes, MPA; and Hayden B. Bosworth, PhD
Producing Comparable Cost and Quality Results From All-Payer Claims Databases
Maria de Jesus Diaz-Perez, PhD; Rita Hanover, PhD; Emilie Sites, MPH; Doug Rupp, BS; Jim Courtemanche, MS; and Emily Levi, MPH
Currently Reading
Beyond Satisfaction Scores: Exploring Emotionally Adverse Patient Experiences
Laura M. Holdsworth, PhD; Dani L. Zionts, MScPH; Karen Marie De Sola-Smith, PhD; Melissa Valentine, PhD; Marcy D. Winget, PhD; and Steven M. Asch, MD
Pilot of Urgent Care Center Evaluation for Acute Coronary Syndrome
Ryan P. Radecki, MD, MS; Kevin F. Foley, PhD; Timothy S. Elzinga, MD; Cynthia P. Horak, MD; Thomas E. Gant, MS; Heather M. Papp, BA; Adam J. Morris, BS; Natalie R. Hauser, BA; and Briar L. Ertz-Berger, MD, MPH

Beyond Satisfaction Scores: Exploring Emotionally Adverse Patient Experiences

Laura M. Holdsworth, PhD; Dani L. Zionts, MScPH; Karen Marie De Sola-Smith, PhD; Melissa Valentine, PhD; Marcy D. Winget, PhD; and Steven M. Asch, MD
This study explores the causes of emotionally adverse patient experiences in cancer care and presents a taxonomy for analyzing free-text patient data.

Objectives: Although improving the average patient experience is at the center of recent efforts to make cancer care more patient centered, extreme experiences may be more informative for quality improvement. Little is known about the most deeply dissatisfying experiences that predispose disengagement and negatively influence patient outcomes. We sought to establish a framework for emotionally adverse patient experiences and identify the range of common causes.

Study Design: Qualitative study including in-depth interviews and free-text survey comments.

Methods: Thematic analysis of 20 open-ended patient interviews and 2389 free-text survey comments collected in a medical center’s cancer clinics.

Results: Emotionally adverse experiences were rarely reported in survey comments (96; 4.0%) but more frequently discussed in interviews (12 interview participants). Such experiences were identified through explicit statements of negative emotion, language, syntax, and tone. Among these rare comments, hostility as an indicator was easiest to identify, whereas passive expressions of fear or hopelessness were less reliably identified. We identified 3 mutually inclusive high-level domains of triggers of negative emotion—system issues, technical processes, and interpersonal processes—and 10 themes within those domains. There was wide variation in the causes of emotionally adverse experiences and evidence of a complex interplay of patient expectations and preconditions that influenced the perception of negative experiences.

Conclusions: This study presents a taxonomy for classifying emotionally adverse patient experiences expressed in free-text format. Further research should test how perceptions of adverse experiences correspond to recorded ratings of patient satisfaction and subsequent enrollment or utilization.

Am J Manag Care. 2019;25(5):e145-e152
Takeaway Points

Extreme dissatisfaction with care can have negative consequences for patients with cancer, such as nonadherence to treatment and disengagement. Understanding and identifying the causes of negative experiences could help focus quality improvement efforts.
  • Although emotionally adverse experiences were extremely rare, their causes were diverse, including coordination, technical skills, communication, bad provider and staff behavior, wait times, scheduling, finance and insurance, physical symptoms, travel, and education and information.
  • Perception of adverse experiences was influenced by patient priorities, past experiences, clinical needs, and expectations.
  • We present a taxonomy that could be used to meaningfully analyze free-text patient data.
Improving patients’ experiences as they face serious illness is a worthy goal, and it correlates strongly with retention in care.1 Evidence has accumulated that better patient experience has important ancillary benefits, including better treatment adherence and self-reported quality of life.2-5 Although characteristics of care that lead to positive ratings of patient experience are becoming better understood,6-9 less is known about the correlates of extreme ratings. Deeply dissatisfying experiences may not have the same correlates as positive ones, and their consequences may be more severe.

In consumer behavior research, events eliciting the strongest negative emotional responses that drive consumers away are sometimes known as disgusters.10-12 Disgusters are issues that are both very important and very negative for consumers, as opposed to annoyances (negative but less important).12 Patients are not simply consumers, so we must be cautious in applying marketing theory to healthcare. Nonetheless, the concept of discretely classifying the severity of adverse experiences may help increase understanding of the relationship of patient experience to subsequent adherence, utilization, and outcomes (Podtschaske et al, unpublished data, 2015). Negative experiences, although themselves consequential, also correlate to additional negative consequences, such as avoidance or withdrawal from care,13,14 lack of participation in decision making,15,16 nondisclosure of concerns to doctors,17,18 nonadherence to treatment,19,20 increased use of emergency services,21 and seeking care elsewhere.22,23 Such actions have negative health consequences for patients and implications for retention in a patient-centered healthcare system.

Understanding patient experience in cancer care is particularly important. Beyond its inherent value, changing providers or poor participation in shared decision making may have worse consequences in cancer than in many other chronic diseases. Previous qualitative investigations of negative experiences have identified some causes, including perception of disparity and exclusion from resources,23 wait times resulting in delayed treatment,24 unmet information needs,24 having the severity of symptoms dismissed or minimized by oncologists,25 and excessive self-coordination of care.23,24 Although these studies provide useful clues, there is limited evidence of the roots of extreme adverse experiences in cancer care. As part of an ongoing effort to transform cancer care quality, we aimed to develop a better understanding of emotionally adverse experiences that are egregious to patients and harmful to engagement among oncology patients, which we see as a parallel to the well-known concept of serious adverse events in healthcare.


We conducted a secondary analysis of patient interviews and free-text survey comments collected as part of an evaluation of a transformation effort in the cancer clinics of an academic medical institution in the United States using the concept of “emotionally adverse experiences” as a lens. We drew on Fortini-Campbell’s marketing framework12 to define our concept of emotionally adverse experiences. Her framework proposes that consumers make decisions along 2 axes: importance and good-positive/bad-negative valence. She argues that understanding how consumers experience a product or brand on different issues can help target areas for brand improvement. Issues perceived as important and negative are termed disgusters, which drive consumers away and thus should be prioritized. Although this framework is derived from marketing theory, we see it as applicable to quality improvement in healthcare. However, we broadened our definition to reflect that, unlike consumers in other markets, patients might be unable to switch providers. Drawing on Fortini-Campbell’s framework and directly from the data, we defined such experiences as being indicated by changing providers, filing a complaint, nonadherence to treatment, disengagement from care, or consideration of the aforementioned options; bad word of mouth; and expressions of affront.

Twenty patients were recruited for interviews by flyers at clinic visits. Participants had to be adult (≥18 years) patients with cancer treated at the center. Interviews were semistructured, asking open-ended questions about patients’ experiences with cancer care; they took place in person (in cancer center conference rooms or patients’ homes) or by telephone. Additionally, we analyzed 2389 free-text comments written by patients or their caregivers on surveys that asked structured questions regarding 1 of 4 topic areas: access, communication, coordination, or information and shared decision making.26 Front-desk staff distributed the surveys to patients at check-in for clinic appointments. We included both in-depth interviews and free-text survey comments to minimize methodological bias.

Handwritten comments were transcribed into a database and imported along with interview transcripts into QSR International’s NVivo 11 Pro for analysis. The analytic process followed guidance by Miles and Huberman.27 Two coders independently analyzed data inductively looking for identifiers of affect in language, syntax, and tone and for content that described adverse experiences. Transcripts were first read and then reread while listening to audio (if it was available) to see whether additional negative emotions could be detected in verbal data. We created 3 coding structures for which data were coded at all 3 levels: (1) affective identifiers, (2) triggers (content of issues) that related to an adverse experience, and (3) a 3-tiered subjective rating for the level of adversity (extreme, annoyance, would have been nice to have). Data had to contain 1 or more of the affective identifiers and be rated by the coder in the extreme to constitute an extreme negative experience.

We used the Pleasure–Arousal–Dominance (PAD) framework28 to facilitate interpretation of the data and refine our coding structure of affective identifiers. The PAD framework characterizes emotional states along 3 dimensions: pleasure (+P)/displeasure (–P), aroused (+A)/not aroused (–A), and dominance (+D)/submissiveness (–D). In the data, we found that patients expressed negative emotion in a range of ways, which was difficult to interpret. We therefore used the model to define the emotions expressed by patients and focus analysis on the displeasure axis (ie, bored, disdainful, anxious, hostile). Through discussion, we refined our coding structure to agree on the identifiers of negative emotion and extreme negativity. After coding all data, we looked for patterns across codes to create higher-level, explanatory categories, and we examined whether there were differences in the content of emotionally adverse experiences between interviews and survey comments.

As an additional check of our understanding, we presented sample data, identifiers, and triggers to volunteers from the cancer patient and family advisory council for their opinion as to the completeness of the categories. This process confirmed our list and yielded the additional criterion of description specificity of the event as an identifier (ie, greater detail meant an adverse experience was more impactful).

This study received a nonresearch determination from the Stanford Institutional Review Board in July 2014 because the primary purpose was to evaluate quality improvement efforts.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up