Currently Viewing:
The American Journal of Managed Care January 2015
Disease-Modifying Therapy and Hospitalization Risk in Heart Failure Patients
Fadia T. Shaya, PhD, MPH; Ian M. Breunig, PhD; and Mandeep R. Mehra, MD, FACC, FACP, FRCP
Frequency and Costs of Hospital Transfers for Ambulatory Care-Sensitive Conditions
R. Neal Axon, MD, MSCR; Mulugeta Gebregziabher, PhD; Janet Craig, PhD, RN; Jingwen Zhang, MS; Patrick Mauldin, PhD; and William P. Moran, MD, MS
Celebrating Our 20th Anniversary
A. Mark Fendrick, MD, and Michael E. Chernew, PhD Co-Editors-in-Chief, The American Journal of Managed Care
Value-Based Insurance Design: Benefits Beyond Cost and Utilization
Teresa B. Gibson, PhD; J. Ross Maclean, MD; Michael E. Chernew, PhD; A. Mark Fendrick, MD; and Colin Baigel, MBChB
Changing Physician Behavior: What Works?
Fargol Mostofian, BHSc; Cynthiya Ruban, BSc; Nicole Simunovic, MSc; and Mohit Bhandari, MD, PhD, FRCSC
State of Emergency Preparedness for US Health Insurance Plans
Raina M. Merchant, MD, MSHP; Kristen Finne, BA; Barbara Lardy, MPH; German Veselovskiy, MPP; Casey Korba, MS; Gregg S. Margolis, NREMT-P, PhD; and Nicole Lurie, MD, MSPH
Relationship of Diabetes Complications Severity to Healthcare Utilization and Costs Among Medicare Advantage Beneficiaries
Leslie Hazel-Fernandez, PhD, MPH; Yong Li, PhD; Damion Nero, PhD; Chad Moretz, ScD; S. Lane Slabaugh, PharmD, MBA; Yunus Meah, PharmD; Jean Baltz, MMSc, MSW; Nick C. Patel, PharmD, PhD; and Jonathan R. Bouchard, MS, RPh
Revisiting Hospital Length of Stay: What Matters?
Mollie Shulan, MD; and Kelly Gao
Medical Homes: Cost Effects of Utilization by Chronically Ill Patients
Jason Neal, MA; Ravi Chawla, MBA; Christine M. Colombo, MBA; Richard L. Snyder, MD; and Somesh Nigam, PhD
Value-Based Insurance Design and Medication Adherence: Opportunities and Challenges
Kevin A. Look, PharmD, PhD
New Start Versus Continuing Users on Aripiprazole: Implications for Policy
Rashid Kazerooni, PharmD, BCPS; Joseph B. Nguyen, PharmD, BCPS; Mark Bounthavong, PharmD, MPH; Michael H. Tran, PharmD, BCPS; and Nermeen Madkour, PharmD, CSP
Multiple Chronic Conditions in Type 2 Diabetes Mellitus: Prevalence and Consequences
Pei-Jung Lin, PhD; David M. Kent, MD, MSc; Aaron Winn, MPP; Joshua T. Cohen, PhD; and Peter J. Neumann, ScD
Prognostic Factors of Mortality Among Patients With Severe Hyperglycemia
Ya-Wun Guo, MD; Tzu-En Wu, MD, MS; and Harn-Shen Chen, MD, PhD
Currently Reading
Survey Nonresponders Incurred Higher Medical Utilization and Lower Medication Adherence
Seppo T. Rinne, MD, PhD; Edwin S. Wong, PhD; Jaclyn M. Lemon, BS; Mark Perkins, PharmD; Christopher L. Bryson, MD; and Chuan-Fen Liu, PhD

Survey Nonresponders Incurred Higher Medical Utilization and Lower Medication Adherence

Seppo T. Rinne, MD, PhD; Edwin S. Wong, PhD; Jaclyn M. Lemon, BS; Mark Perkins, PharmD; Christopher L. Bryson, MD; and Chuan-Fen Liu, PhD
In a retrospective cohort analysis, diabetic nonresponders to a patient satisfaction survey had higher healthcare costs, clinic visits, and hospitalizations, but lower medication adherence.
To compare healthcare costs, utilization, and medication adherence between diabetic responders and nonresponders to a patient satisfaction survey.

Study Design
We performed a retrospective cohort study of 40,766 patients with diabetes who had been randomly selected to receive the 2006 Veterans Affairs' Survey of Healthcare Experiences of Patients. Outcomes were measured during the following year.

We used multivariable models to compare healthcare costs (generalized linear models), utilization (negative binomial regression), and adherence to oral hypoglycemic medications (logistic regression) between survey responders and nonresponders.

There were 26,051 patients (64%) who responded to the survey. Survey nonresponders incurred significantly higher healthcare costs (incremental effect, $792; 95% CI, $599-$986; P <.01). Nonresponders had a modest increase in primary care (incidence rate ratio [IRR], 1.06; 95% CI, 1.05-1.08; P <.01) and specialty care visits (IRR, 1.17; 95% CI, 1.12-1.22; P <.01), but more substantial increases in mental health visits (IRR, 1.74; 95% CI, 1.62-1.87; P <.01) and hospitalizations (IRR, 1.60; 95% CI, 1.46-1.75; P <.01). Medication adherence was significantly lower among survey nonresponders (odds ratio, 0.68; 95% CI, 0.65-0.74; P <.01).

Nonresponders to a patient satisfaction survey incurred higher healthcare costs and utilization, but had lower medication adherence. Understanding these characteristics helps to assess the impact of nonresponse bias on patient satisfaction surveys and identifies clinical practices to improve care delivery.

Am J Manag Care. 2015;21(1):e1-e8
Diabetic nonresponders to a patient satisfaction survey incurred higher healthcare utilization and costs, but lower medication adherence.
  • Our findings describe the potential effect of nonresponse bias on patient satisfaction surveys.
  • Nonresponders may represent a challenging patient population that relies more heavily on the healthcare system, but practices less self-care.
  • Knowing the characteristics of nonresponders could help clinicians target interventions to a particularly vulnerable patient population.
  • Potential biases related to survey nonresponse should be considered as policy makers begin to tie reimbursements to patient satisfaction surveys.
Patient satisfaction, which is routinely evaluated through patient surveys, has become increasingly important in assessing quality improvement, particularly as health systems—such as that of the the Veterans Health Administration (VA)—move toward a patient-centered focus to deliver care.1 As policy makers continue to tie healthcare reimbursement to patient satisfaction, these satisfaction surveys are becoming integrated into clinical care.2 Despite the widespread use of patient surveys, however, there is limited research examining the validity of patient satisfaction measures. In particular, it is not known how systematic biases, including nonresponse bias, impact the results of patient satisfaction surveys and how nonresponders interact with the healthcare system.3-5

The majority of research characterizing survey nonresponders has focused on population-based health surveys.6-14 Population-based surveys target a broad population, including individuals who have limited contact with healthcare.12 In contrast, patient satisfaction surveys focus specifically on individuals who are connected to the health system and elicit responses that are pertinent to patients’ health and satisfaction with care. As a result, determinants and outcomes associated with nonresponse to clinic-based patient satisfaction surveys and population-based health surveys may differ.

The VA has conducted regular assessments of patient satisfaction and offers a unique opportunity to study nonresponders in these surveys. Recognizing the characteristics of survey nonresponders may also help clinicians detect specific patient needs. Survey nonresponders are thought to have unique personality characteristics and may respond differently to medical care when compared with responders.15 For example, a study of patients with chronic illness found that individuals who did not respond to a medication-beliefs survey exhibited less persistence to pharmacological therapy.16 Knowing this difference could provide an opportunity for more in-depth counseling for survey nonresponders. Understanding characteristics of nonresponders is particularly important with patients who have a chronic illness such as diabetes, as they may represent a particularly vulnerable medical population.

The objective of our study was to identify factors associated with nonresponse to a patient satisfaction survey and examine whether the nonresponse was associated with greater utilization of healthcare. To do this, we studied a large population of diabetic patients who were seen in VA primary care clinics and mailed a questionnaire related to their healthcare experience. We compared healthcare costs, number of outpatient and inpatient encounters, and adherence to oral hypoglycemic medications between survey responders and nonresponders.



We performed an observational study examining diabetic veterans who received healthcare at VA outpatient clinics during the 2005 and 2006 fiscal years (FYs) (October 1, 2004, to September 30, 2006). Data were collected from the VA electronic medical record and administrative databases. The Survey of Healthcare Experiences of Patients (SHEP) was administered to a random subset of VA patients during FY2006, and we compared characteristics between survey responders and nonresponders. The VA Puget Sound Health Care System Institutional Review Board approved the study.

Patient Survey

SHEP is an annual survey conducted by the VA Office of Informatics and Analytics to assess patients’ perceptions of the healthcare received during a particular clinic visit.17 Every month, a stratified random design is used to survey patients from each VA facility that provided ambulatory care. To ensure adequate representation of both specialty care and primary care, approximately 15 patients were randomly selected from each of the following visit categories: specialty care, established primary care, and new primary care. Patients were eligible for the survey if they visited a VA clinic during the previous month and had not participated in a SHEP during the prior 12 months. Across the VA, 431,921 patients received the 2006 SHEP, and the response rate was 57.6% (N = 248,850).

In 2006, SHEP included 107 questions and was estimated to take 30 minutes to complete. It included questions on obtaining an appointment, arrival and registration, interaction with the provider, a self-evaluation of health, health behaviors, and individual demographics. Selected patients were initially sent a letter that explained the purpose of the survey, and encouraged them to participate. One week later, the SHEP questionnaire was mailed. Reminder postcards were sent the following week, and survey collection remained open for 3 weeks after the postcards were sent. We collected administrative data on all patients who were mailed a SHEP questionnaire, including responders and nonresponders.

Study Population

Our study sample was derived from a cohort of 444,414 veterans with diabetes who were on oral diabetes medications and were previously seen in VA primary care clinics (see Wong et al for more details).18 To address medication use, patients were included if they had at least 2 outpatient prescriptions for an oral diabetes medication (metformin, sulfonylurea, or thiazolidinedione) during FY2006. Of these patients, 40,766 were mailed a patient satisfaction survey. The response rate among patients in our diabetes cohort was 64%. All analyses were weighted back to the patient cohort of 444,414 using sample weights constructed to account for patients seeking VA care more frequently being more likely to receive SHEP.

Information on patient characteristics, comorbidities, and medication use was collected for all patients in the study. Patient characteristics consisted of age, gender, race, marital status, and having received free VA care for disability or low income. Access to VA care was assessed by distance to the most frequently visited VA facility. The severity of diabetes was measured by the Diabetes Complications Severity Index score, which is based on A1C and end organ damage.19 The overall patient healthcare burden was evaluated using Diagnostic Cost Group (DCG) scores, which are based on the predicted cost of patients’ underlying comorbidities.20 In addition to measuring DCG scores, specific comorbidities were identified by having 2 outpatient encounters with an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis, or 1 inpatient encounter with an ICD-9-CM diagnosis, and included heart failure, atrial fibrillation, chronic obstructive pulmonary disease, renal failure, stroke, and depression.


Our primary outcome was costs for all healthcare paid for by the VA during FY2007. We examined 3 categories of costs: outpatient, inpatient, and total costs. Healthcare costs were obtained from the Decision Support System (DSS) National Extracts for services provided in the VA, and the VA Fee Basis files (for services provided by non-VA providers and paid for by the VA). The Consumer Price Index was used to convert costs in calendar year 2006 to 2007 constant dollars.

Secondary outcomes included the number of outpatient face-to-face encounters, including primary care, specialty care, and mental health visits, as well as the number of hospitalizations in FY2007. We also assessed medication adherence to oral hypoglycemic agents derived from DSS National Outpatient Pharmacy Extracts that included drug names, prescription dispense dates, and days supplied. We used prescription refill data to calculate medication possession ratios (MPRs) for oral hypoglycemic agents.21 We classified a patient as adherent if the MPR was greater than or equal to 80% during the first quarter of 2007.22

Statistical Analysis

For bivariate analyses, we used χ2 and t tests to compare characteristics of survey responders and nonresponders. For multivariate analyses, we used a logistic regression to assess patient factors associated with survey nonresponse.

We analyzed cost measures using generalized linear models (GLMs). Due to skewness in the distribution of cost, each cost variable was transformed prior to multivariable analysis. We used the Modified Hosmer-Lemeshow test to determine the appropriate link function and the Modified Park Test to assess the shape of the variance distribution.23,24 For each cost model, the appropriate link function was the cubic root function and the appropriate GLM family was the Gamma distribution. Because the majority of patients were not hospitalized in the followup year (FY2007), we analyzed inpatient costs using 2-part models.25 In the first part, we estimated a probit model across all patients to model the probability of hospitalization. In the second part, we estimated a GLM across the subsample of patients with nonzero inpatient costs. We then calculated expected costs for each patient by multiplying predicted hospitalization probability (first part) and inpatient costs (second part). For all models, we calculated incremental effects (IEs), or the change in respective cost measures associated with survey nonresponse.

We analyzed utilization measures using negative binomial regressions. Results were reported as incidence rate ratios (IRRs), which were derived by dividing the expected number of encounters or hospitalizations among the survey nonresponders by the expected number of encounters or hospitalizations among the survey responders. Finally, we used a logistic regression to assess the relationship between medication adherence and survey nonresponse.

We also performed a sensitivity analysis, restricting our analyses to patients who were less than 65 years old, to isolate those patients who were most likely to rely solely on the VA system for healthcare. All multivariate analyses were controlled for patient characteristics and comorbidities and adjusted for sampling weights. All statistical analyses were performed using STATA 11.2 (StataCorp LP, College Station, Texas).26


Descriptive Statistics

Of the 40,766 patients in our cohort who were mailed the SHEP, 26,051 patients (64%) responded to the survey and 14,715 patients (36%) did not respond. Patient characteristics differed between the 2 groups (Table 1). Survey nonresponders were more likely to be younger (P <.01), female (P <.01), unmarried (P <.01), and nonwhite (P <.01). A higher proportion of survey nonresponders received free care from the VA compared with survey responders (P <.01). DCG scores were similar between the 2 groups, though specific comorbidities differed. Survey nonresponders had a lower diabetes severity index (P <.01) and a lower prevalence of hypertension (P = .01), coronary artery disease (P <.01), and atrial fibrillation (P <.01). Nonresponders had significantly higher prevalence of depression (P <.01), posttraumatic stress disorder (P <.01), and schizophrenia (P <.01).

Copyright AJMC 2006-2019 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up