Currently Viewing:
The American Journal of Accountable Care December 2015
The Need to Level the Playing Field Between Accountable Care Organizations and Medicare Advantage
David Introcaso, PhD
The Debate Over Drug Costs: Instead of "How Much" We Spend, Let's Focus on What We Get in Terms of Health
A. Mark Fendrick, MD Co-Editor-in-Chief, The American Journal of Managed Care Professor of Medicine, School of Medicine Professor of Health Management and Policy, School of Public Health Director, Cen
The Pediatric ACO: A New Frontier in Accountable Care
Anthony D. Slonim, MD, DrPH
Revisiting the Role of Academic Medical Centers in Medicare Shared Savings Program ACOs
Benjamin M. Gerber, JD
The Emerging Business Models and Value Proposition of Mobile Health Clinics
Khin-Kyemon Aung AB; Caterina Hill, MSC, MA; Jennifer Bennet, BS; Zirui Song, MD, PhD; and Nancy E. Oriol, MD
Now That ACOs Are Engaged, What About Those Patients?
Mary K. Caffrey
Currently Reading
It Is Time to Ask Patients What Outcomes Are Important to Them
Leif I. Solberg, MD; Stephen E. Asche, MA; John C. Butler, MD; David Carrell, PhD; Christine K. Norton, MA; Jeffrey G. Jarvik, MD, MPH; Rebecca Smith-Bindman, MD; Juliana O. Tillema, MPA; Robin R. Whi
Bundled Payments for Diabetes Care and Healthcare Costs Growth: A 2-Year Follow-up Study
Sigrid Mohnen, PhD; Caroline Baan, PhD; and Jeroen Struijs, PhD
Using Guidelines: When Is It Appropriate?
Maude St-Onge, MD, PHD, FRCPC
The Impact of Direct Oral Anticoagulants in the Managed Care Environment
Charles E. Mahan, PharmD, RPh, PhC; and Robert Lavender, MD, FACP

It Is Time to Ask Patients What Outcomes Are Important to Them

Leif I. Solberg, MD; Stephen E. Asche, MA; John C. Butler, MD; David Carrell, PhD; Christine K. Norton, MA; Jeffrey G. Jarvik, MD, MPH; Rebecca Smith-Bindman, MD; Juliana O. Tillema, MPA; Robin R. Whi
Patients with abdominal or back pain identified 21 outcomes important to them, but the reported outcomes are quite different from the symptom and function outcomes studied by researchers.

Objectives: To identify the outcomes desired by patients (and their family members) with abdominal or back pain and to compare patient and physician opinions regarding the importance of each outcome.

Study Design: Mixed methods.

Methods: After identifying 21 potentially important outcomes from the lit­erature and telephone interviews with patients and family members, we asked 40 patients, 11 family members, and 11 primary care physicians in telephone interviews to rate the importance of each outcome to patients on a scale of 1 to 5 scale (5 = most important), stratified by pain location.

Results: Mean patient ratings of the 21 outcomes ranged from 3.3 to 5, with the average rating across all items higher for patients with back pain than those with abdominal pain (4.50 vs 4.09; P = .049). Physicians rated the importance of these outcomes to patients significantly lower than the patients did for both abdominal pain (4.1 vs 3.5; P = .04) and back pain (4.5 vs 3.6; P = .0003). Family member ratings were similar to those of the patients (4.3 vs 4.2; P = .8), where­as physicians rated the importance to patients to be an average of 0.6 points lower than the ratings of patients for abdominal pain and 0.8 points lower for back pain.

Conclusions: Many outcomes are important to patients and their family mem­bers, but they mostly represent quality-of-life events rather than the symptom and function measures heretofore focused on by researchers. Physicians appear to rate most of these outcomes somewhat lower in importance.
Patient outcomes and their measurement for comparative effectiveness research, performance measurement, and patient care are increasingly important. Early measures of patient outcomes were usually selected from the perspective of a clinician, but there was a shift to a more patient-centric perspec­tive 15 years ago as measures of health status and quality of life.1 By 2010, patient outcomes had attained such priority nationwide that the Patient-Centered Outcomes Research Institute (PCORI) was established to emphasize incorporating patient viewpoints and outcomes in every aspect of healthcare research.2

This large shift has required the development of patient-re­ported outcomes measures (PROMs). Subsequently, the National Institutes of Health established the Patient-Reported Outcomes Measurement Information System (PROMIS) in 2004 with the goal of “providing clinicians and researchers access to efficient, precise, valid, and responsive…measures of health and well-be­ing.”3 Although this system was primarily created for clinical re­search, it is rapidly becoming the standard source of PROMs for all purposes. The PROMIS Assessment Center currently contains measure sets that are publicly available in 19 domains for assessing the physical, mental, and social health of both adults and children.

However, the PROMIS tools largely measure specific patient functions and symptoms and seem to assume that such specific measures are the only outcomes that matter to patients and their families. The PROMIS website does not describe any involve­ment of patients in the selection of its outcome measures, and no publications describe attempts to learn from patients what outcomes they care most about. Although patients probably do care about relief from symptoms and improvement in function, there may also be other outcomes that are equal or more import­ant to them.

Therefore, we developed this study to discover what outcomes patients and their family members cared about among those who had experienced back or abdominal pain problems that were se­rious enough to require advanced diagnostic imaging studies. We focused on patients with actual experiences, thinking they should be most aware of outcomes important to them. After identifying potentially important outcomes through preliminary open-ended interviews, we surveyed patients, their family members, and pri­mary care physicians to learn their rating of the importance of each outcome and then compared these ratings among patients, family members, and physicians.



We conducted this study among the patients receiving care from an 800-physician multi-specialty medical group in the Minneap­olis-St. Paul metropolitan area. In order to facilitate access to health plan claims data for these patients for a later phase of the study, patient recruitment was limited to the 60% with insurance from the affiliated health plan; this population includes patients on prepaid medical assistance and a racial and ethnic profile sim­ilar to that of the Twin Cities metropolitan area.

Pilot Interviews

In order to create a list of potentially important patient outcomes, we randomly identified adults who had undergone either a mag­netic resonance imaging (MRI) or computed tomography (CT) scan for abdominal or back pain about 1 year prior. We first sent these individuals a letter explaining the study and providing an opportunity to opt out of further recruitment contacts. If they did not opt out, trained interviewers called them to assess their willingness to participate in a telephone interview, during which they were asked about all of the outcomes they had wanted from the care of their pain problem. Interviewers next read them a list of other possible outcomes that had been generated by the re­search team (which included a patient co-investigator) and asked them to rate the importance of each outcome on a scale of 1 to 5 (5 = extremely important, 1 = not at all important). Finally, they were asked again about any other outcomes they might have thought of. We gave completed interviewees a $40 gift card—this incentive was mentioned in both the pre-notification letter and introductory script. Out of 7 patients contacted within 3 tele­phone attempts, 6 agreed to participate and completed the inter­view. Two of those 6 said there was a family member who was very familiar with their pain problem and would be willing to be contacted, so we also recruited them and completed interviews. The responses from these 8 people were so consistent that no further pilot interviews seemed needed.

Patient/Family Interviews

Following the pilot, in hopes of completing 40 patient telephone interviews, we identified an additional random pool of 83 adult patients who had experienced a first CT or MRI scan of the back or abdomen for pain about 1 year prior. We followed the same protocol used in the pilot; the interview script—revised after the pilot calls—confirmed eligibility, obtained demographic infor­mation, and asked about the most important outcomes desired from the medical care of their problem. Then we asked subjects to rate a revised list of 21 outcomes (see Table 1 for the list in the order in which items were asked) on an importance scale of 1 to 5 (again, 5 = extremely important, 1 = not at all important). Finally, we asked them whether there was a family member who was familiar with their problem who might be willing to complete a similar interview. If so, we made contact arrangements and the family member interview followed the same process. Interviews lasted an average of 15 minutes and all respondents were sent a $40 gift card.

Physician Interviews

We identified and recruited practicing adult primary care physi­cians through the help of a co-investigator (CV) at the Institute for Clinical Systems Improvement for interviews by our practic­ing primary care physician co-investigator (JB). Because the phy­sicians had not had a specific personal experience to reference, we provided them with common case scenarios for patients with back or abdominal pain as context for their answers. We asked what outcomes they thought would be most important to the scenario patients; then they were asked to rate each of the 21 out­comes (see Table 1) identified from patients on the same 1-to-5 scale from the perspective of their patients. They were also asked about the most important outcomes from their perspective as a clinician.


Associations between type of pain and patient attributes were tested using contingency tables and Pearson’s χ2 and Fisher’s ex­act tests. Independent samples t tests were used to test differenc­es in outcome importance ratings between abdominal and back pain patients, as well as differences in outcome importance rat­ings between patients and physicians. The study has 80% power (α = 0.05, 2-sided test) to detect a difference of 0.9 standard deviations (SDs) in the mean outcome rating from abdominal and back pain patients. An α of 0.05 was considered statistically significant, 2-tailed tests were used, and there was no correction for multiple testing. Analyses were conducted using SAS version 9.1 (SAS Institute Inc, Cary, North Carolina).


Of 83 patients with call attempts, we could not reach 33, and 1 of those contacted was ineligible as they reported not having the scan (see Figure 1). Of the 49 eligible patients who could be contacted, 41 completed interviews—but the data from 1 was inadvertently lost—for a participation rate of 84% of eligible pa­tients contacted, or 49% of all those attempted. Twelve patients nominated a family member who had been actively involved in the care of their pain problem for interviews, and 11 of those 12 completed them. Responding patients were similar to nonre­sponders by age, race, Medicare insurance, pain type, and scan type. The major difference was that responders were much more likely to be females (92.5% vs 46.2%; P <.001) and to be on Med­icaid (7.5% vs 0%; P = .045).

The characteristics of participants are listed by pain type in Table 2. The majority were female and white. Aside from the type of scan, none of the other characteristics differed signifi­cantly by pain type. Ten of the family members were spouses and 1 was a daughter. Family member characteristics were similar to those of the patients except that family members were more likely to be male (7 of 11 [63%]) and married (10 of 11 [91%]) than patients as a whole.

Seventeen primary care physicians were identified as potential participants in the second-stage interview, but 4 of them referred us to colleagues who agreed to participate and completed inter­views along with 7 others. Each physician was from a different clinic: 4 were from the Twin Cities metro area and 7 were from the rest of the state; 2 were internists and 9 were family physi­cians; 3 were female and 8 male; and 6 had been in practice for 26 to 43 years, 2 for 12 to 15 years, and 3 for 4 to 6 years.

In the initial open-ended phase of the pilot patient inter­views, the main outcome identified by most patients and family members was to learn the cause of the pain. Seven of the other outcomes on the list were also mentioned spontaneously before respondents were given the list to rate. Only 1 (“minimize dis­comfort from the tests used to assess the pain”) was raised that was not on the list.

Table 3 contains the mean importance ratings by patients for the 21 outcomes and the difference between those ratings by pa­tients with abdominal pain versus those with back pain. There was considerable overlap between the 2 groups, although patients with abdominal pain gave lower importance ratings for nearly all outcomes than did patients with back pain; the overall mean rat­ing was 0.4 points lower than those with back pain (4.09 vs 4.50; P = .049). Higher ratings were given by patients with abdomi­nal pain for only 2 outcomes: to find the cause and to get rapid and complete relief. The overall mean rating of the 21 outcomes from family and patients was similar (4.32 for family, 4.28 for pa­tients; P = .86). Most (8 of 11) family members were connected with patients who had experienced abdominal pain. Importance ratings by family members of abdominal pain patients were the same or directionally higher than those of abdominal pain pa­tients for 16 of 21 outcomes, and there were no statistically sig­nificant differences.

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up