Currently Viewing:
The American Journal of Managed Care May 2017
Drivers of Excess Costs of Opioid Abuse Among a Commercially Insured Population
Lauren M. Scarpati, PhD; Noam Y. Kirson, PhD; Miriam L. Zichlin, MPH; Zitong B. Jia, BA; Howard G. Birnbaum, PhD; and Jaren C. Howard, PharmD
Critical Incident Stress Debriefing After Adverse Patient Safety Events
Reema Harrison, PhD, MSc, BSc, and Albert Wu, MD, MPH
Assessing the Effect of the VHA PCMH Model on Utilization Patterns Among Veterans With PTSD
Ian Randall, PhD; Charles Maynard, PhD; Gary Chan, PhD; Beth Devine, PhD; and Chris Johnson, PhD
State Prescription Drug Monitoring Programs and Fatal Drug Overdoses
Young Hee Nam, PhD; Dennis G. Shea, PhD; Yunfeng Shi, PhD; and John R. Moran, PhD
Disparities in Diabetes and Hypertension Care for Individuals With Serious Mental Illness
Junqing Liu, PhD; Jonathan Brown, PhD; Suzanne Morton, MPH; D.E.B. Potter, MS; Lisa Patton, PhD; Milesh Patel, MS; Rita Lewis, MPH; and Sarah Hudson Scholle, DrPH
The Cost of Adherence Mismeasurement in Serious Mental Illness: A Claims-Based Analysis
Jason Shafrin, PhD; Felicia Forma, BSc; Ethan Scherer, PhD; Ainslie Hatch, PhD; Edward Vytlacil, PhD; and Darius Lakdawalla, PhD
Prescription Opioid Registry Protocol in an Integrated Health System
G. Thomas Ray, MBA; Amber L. Bahorik, PhD; Paul C. VanVeldhuisen, PhD; Constance M. Weisner, DrPH, MSW; Andrea L. Rubinstein, MD; and Cynthia I. Campbell, PhD, MPH
Opioid Prescribing for Chronic Pain in a Community-Based Healthcare System
Robert J. Romanelli, PhD; Laurence I. Ikeda, MD; Braden Lynch, PharmD; Terri Craig, PharmD; Joseph C. Cappelleri, PhD; Trevor Jukes, MS; and Denis Y. Ishisaka, PharmD
Currently Reading
The Association of Mental Health Program Characteristics and Patient Satisfaction
Austin B. Frakt, PhD; Jodie Trafton, PhD; and Steven D. Pizer, PhD

The Association of Mental Health Program Characteristics and Patient Satisfaction

Austin B. Frakt, PhD; Jodie Trafton, PhD; and Steven D. Pizer, PhD
Across many measures of Veterans Health Administration mental health care program characteristics, treatment continuity is most strongly and positively associated with patient satisfaction.  

Objectives: Satisfaction with care is an important patient-centered domain of health system quality. However, satisfaction measures are costly to collect and not directly modifiable. Therefore, we assessed the relationships between veterans’ satisfaction and measures of modifiable aspects of Veterans Health Administration (VHA) mental health care programs.

Study Design: For a sample of 6990 patients who received mental health care from the VHA in 2013, we used survey and administrative data to investigate the association of a suite of access and encounter satisfaction measures with a large collection of measures of program characteristics.

Methods: We estimated risk-adjusted correlations between 6 satisfaction measures (across 2 domains: access and encounter satisfaction) and 28 mental health care program characteristics (across 4 domains: program reach, psychosocial service access, program intensity, and treatment continuity).

Results: We found that satisfaction with access to care was higher than experiences with care encounters, but that broad measures of mental health care program reach and intensity were positively associated with both kinds of satisfaction. No measures of psychosocial service access were positively associated with access and encounter satisfaction. Most measures of treatment continuity were consistently and positively associated with both kinds of satisfaction.

Conclusions: As the VHA strives to increase access to, and provision of, mental health care, policy makers and program managers should be aware that satisfaction with care, as it is currently measured, may not rise as more patients initiate treatment, unless continuity of care is maintained or enhanced.

Am J Manag Care. 2017;23(5):e129-e137
Takeaway Points
  • Broad measures of mental health care program reach and intensity and most measures of treatment continuity were consistently and positively associated with patient satisfaction. 
  • Because psychosocial services are clinically valuable, policy makers and managers should not interpret a lack of association with satisfaction to justify reducing their availability. 
  • Policy makers and managers should be aware that satisfaction with care, as currently measured, may not rise as more patients initiate treatment, unless continuity of care is maintained or enhanced. 
  • Policy makers and managers should continue to track patient satisfaction and to specifically target satisfaction with mental health care.
Satisfaction with care is an important, patient-centered measure of health system performance because it can identify gaps in quality that could be missed by other measures,1 help detect cross-population disparities,2 and serve as a catalyst for quality improvement.3,4 Yet, the relationship between patient satisfaction and quality of care, although generally positively correlated,5 is not fully understood.6 Studies have found patient satisfaction to be associated with hospital process quality7; lower rates of readmissions, heart attack mortality,8,9 and surgical quality10; and better long-term outcomes.11 Fenton et al12 found it also positively correlated with higher healthcare utilization, costs, and all-cause mortality.

Assessment of what objective health system measures drive patient satisfaction is important for 2 reasons. First, because it relies on patient surveys, satisfaction remains expensive and challenging to measure at a high frequency.13 However, many other measures of health system performance are easily obtained at a high frequency from administrative data (eg, process quality measures or readmissions). If satisfaction is highly correlated with these other administrative measures, they offer supplements to satisfaction surveys—ways to monitor and improve aspects of care related to satisfaction during longer intervals between measurement. Second, satisfaction is not directly modifiable; improvements must come from changes in the processes of care or investments in services that patients value. 

For these reasons, we studied the relationships between a set of patient satisfaction measures and a large collection of mental health program characteristics for patients with a recent mental health encounter in the Veterans Health Administration (VHA), the largest provider of mental health care in the United States.14 Prior work has documented variation in satisfaction across VHA patients with psychiatric diagnoses. Rosenheck et al15 found that VHA patients who were discharged from the hospital with a primary psychiatric or substance use diagnosis were more likely to be satisfied with their care if they were older, in better health, or had a long length of stay. 

Burnett-Zeigler et al16 reported that VHA patients with psychiatric diagnoses who were younger, nonwhite, or lower-income; had a service-connected disability; or had received a posttraumatic stress disorder (PTSD) or a substance use disorder diagnosis were less likely to be satisfied with their care. Hepner et al17 examined perceptions of behavioral health care among VHA patients who received mental health care. Seventy-four percent said they were helped by treatment, but only 32% reported an improvement in symptoms. Holcomb et al18 found that the satisfaction of midwestern VHA patients with psychiatric diagnoses positively correlated with better self-reported outcomes. Patients with co-occurring substance use and psychotic disorders who were treated in VHA residential substance use disorder treatment programs that had more positive perceptions and satisfaction exhibited greater engagement in care and experienced better outcomes.19 Finally, Hoff et al20 reported lower levels of satisfaction among VHA patients with psychiatric diagnoses than those with medical diagnoses.


Since 2002, the VHA Office of Quality and Performance has fielded the Survey of Healthcare Experiences of Patients (SHEP), an ongoing monthly mail survey of patients’ experiences during their most recent VHA encounter. Modeled on the Consumer Assessment of Healthcare Providers and Systems survey and based on a stratified design that selects from the specialty care domains as well as new and established primary care patients within each facility,21 SHEP samples about 30,000 ambulatory care patients each month who visit the VHA and who were not surveyed in the prior year. The 2013 version of SHEP is our source of satisfaction measures, with an overall response rate of about 44% and slightly higher response rates for males and substantially higher response rates for older patients (eAppendix Table A1 [eAppendices available at]).

In 2010, the Department of Veterans Affairs (VA)’s Office of Mental Health Operations (OMHO) implemented the Mental Health Information System (MHIS) Dashboard,22 which includes facility-level quality metrics consistent with the goals of the VA’s Uniform Mental Health Services Handbook.23,24 In addition, the mental health domain of the VHA Strategic Analytics for Improvement and Learning (SAIL) includes 25 administrative data–based performance measures related to access, continuity of care, patient safety, and quality of care at a facility level.25 We used the 2013 MHIS Dashboard and precursors to SAIL mental health domain report metrics (MHIS and SAIL are refined on an ongoing basis), shared with us by OMHO, to predict patient-level satisfaction responses to the 2013 SHEP. 

SHEP surveys patients with a recent VHA encounter (the “index” encounter). To merge facility-level MHIS/SAIL-based mental health program characteristics, we associated each SHEP respondent with the VHA facility where they had the index encounter. For risk adjustment, we also merged, at the patient level, demographic and Elixhauser26 comorbidity data from VHA administrative files. 

Our interest was in the relationships between mental health care program characteristics and patient satisfaction, so we used data from a subset of SHEP respondents—those with a recent mental health encounter. To accomplish this, we restricted the SHEP sample to respondents with index encounters in the same quarter and year as encounters for mental health. Because most SHEP respondents complete and return surveys 2 or more months after the index visit, this approach guaranteed that the majority would have had a recent mental health encounter prior to providing satisfaction feedback. Therefore, although some of the survey questions ask patients to report satisfaction based on the prior 12 months of care, it would be likely that patients’ impressions were more heavily influenced by their most recent mental health encounter. Nevertheless, unlike prior analyses of satisfaction among VHA mental health patients,1,17 we were not directly assessing satisfaction with mental health care services. Our final sample included 6990 patients across 165 VHA facilities, although not all patients responded to all survey items due to SHEP question skip patterns (eAppendix Table A2). All analyses were conducted at the patient level.

Patient Satisfaction Variables

Satisfaction with timeliness of care, which we termed “access satisfaction,” is measured by SHEP asking respondents how often they were able to obtain needed care right away and were able to get VHA appointments as soon as they thought they needed care, excluding the times they needed urgent care. Access to VHA tests or treatments is measured by SHEP asking how easy it was to access that care in the last 12 months. Response options for the above 3 measures included “always,” “usually,” “sometimes,” or “never.” There was no cardinal meaning to these categorical responses. Therefore, we dichotomized them to eliminate fine gradations in the ordinal scale Specifically, following Prentice et al,27 we dichotomized these to 1 for responses of “always” or “usually” and 0 otherwise (Table 1). 

Encounter satisfaction, which measures satisfaction with the care received or provider seen, is measured by SHEP asking respondents to rate VHA healthcare in the last 12 months on a scale of 0 to 10, where 0 indicates the “worst healthcare possible” and 10 the “best healthcare possible.” Satisfaction with the respondents’ personal doctor/nurse is also assessed on a 0-to-10 scale. For the same reasons given above, we dichotomized these to 1 for responses of 9 or 10 and 0 otherwise.27 Satisfaction with the most recent VHA visit is assessed on SHEP with a scale ranging from 1 to 7, where 1 indicates “completely dissatisfied” and 7 “completely satisfied.” We dichotomized this to 1 for responses of 6 or 7 and 0 otherwise.27 

Program Characteristics Variables

The mental health program measures we considered are listed and defined in Table 2 and are organized into 4 areas of focus: 1) program reach (eg, the proportion of patients receiving mental health care), 2) psychosocial service access (eg, the proportion of patients initiating psychosocial treatment or psychotherapy), 3) program intensity (eg, the number of encounters per year), and 4) treatment continuity (eg, the proportion of discharged patients with follow-up within 7 days). Within each area, we examined 5 or more performance metrics. Transitional work visits and supportive employment visits mentioned in Table 2 are occupational therapy treatment modalities.28,29


Descriptive Statistics

Tables 1 and 2 report means of demographic control variables, dependent variables (patient-level satisfaction), and key independent variables (facility-level program characteristics). eAppendix Table A3 reports the means for diagnostic risk-adjustment variables. Table 1 shows that the average age of patients in our sample was 62 years, 55% were married, 8% were female, and 12% were black. In addition, 17% of our sample had an alcohol use disorder; 9%, a drug use disorder, 37%, psychosis; and 48%, a depression diagnosis (all as defined by Elixhauser25 and listed in eAppendix Table A3). These figures were higher than the general population because we deliberately selected a sample of patients with a VHA mental health visit. Table 1 also shows that, across our sample, most patients reported high levels of satisfaction for all but 1 measure. Table 2 shows facility-level program characteristics organized by the 4 domains. 

The facility-level program characteristics in Table 2 were each computed by OMHO on the full sample of patients implied by each characteristic, not just the patients in our study sample. For example, the program reach characteristic of “PTSD” is defined as “% of patients with PTSD who receive specialty outpatient care for PTSD.” This means that this measure captures, for each facility and year, the percentage of patients with PTSD seen by the facility in that year who received specialty outpatient care for PTSD.

Multivariate Analysis

Separately, for each satisfaction measure (the dependent variable) and each program characteristic (the key independent variable), we estimated an ordinary least squares (OLS) model, controlling for age (in years), marital status (1 = married, 0 = not married), sex (1 = female, 0 = male), race (1 = black, 0 = nonblack), and comorbidities.26 We accounted for heteroscedasticity with robust standard errors. In sensitivity analyses, we also ran models with clustering, facility random effects, and logistic regression. These produced similar results, which are not shown.

Copyright AJMC 2006-2017 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up

Sign In

Not a member? Sign up now!