Patient Experience Midway Through a Large Primary Care Practice Transformation Initiative

Practice transformation toward comprehensive primary care slightly improved patient experience in 3 of 6 domains of care: access, provider support, and shared decision making.
Published Online: March 15, 2017
Kaylyn E. Swankoski, MA; Deborah N. Peikes, PhD, MPA; Stacy B. Dale, MPA; Nancy A. Clusen, MS; Nikkilyn Morrison, MPPA; John J. Holland, BS; Timothy J. Day, MSPH; and Randall S. Brown, PhD
ABSTRACT

Objectives:
To determine how the multipayer Comprehensive Primary Care (CPC) initiative that transforms primary care delivery affects the patient experience of Medicare fee-for-service beneficiaries. The study examines how experience changed between the first and second years of CPC, how ratings of CPC practices have changed relative to ratings of comparison practices, and areas in which practices still have opportunities to improve patient experience.

Study Design: Prospective study using 2 serial cross-sectional samples of more than 25,000 Medicare fee-for-service beneficiaries attributed to 496 CPC practices and nearly 9000 beneficiaries attributed to 792 comparison practices. 

Methods: We analyzed patient experience 8 to 12 months and 21 to 24 months after CPC began, measured using 6 domains of the Consumer Assessment of Healthcare Providers and Systems Clinician and Group 12-Month Survey with Patient-Centered Medical Home supplemental items. We compared changes over time in patients giving the best responses between CPC and comparison practices using a regression-adjusted difference-in-differences analysis.

Results: Patient ratings of care over time were generally comparable for CPC and comparison practices, with slightly more favorable differences—generally of small magnitude—for CPC practices than expected by chance. There were small, statistically significant, favorable effects for 2 of 6 composite measures measured using both the proportion giving the best responses and mean responses: getting timely appointments, care, and information; providers support patients in taking care of their own health; and providers discuss medication decisions. There was an additional small favorable effect on the proportion of patients giving the best response in getting timely appointments, care, and information; there was no effect on the mean.

Conclusions: During the first 2 years of CPC, CPC practices showed slightly better year-to-year patient experience ratings for selected items, indicating that transformation did not negatively affect patient experience and improved some aspects slightly. Patient ratings for the 2 groups were generally comparable, and both faced substantial room for improvement. 

Am J Manag Care. 2017;23(3):178-184
Takeaway Points

The 4-year Comprehensive Primary Care (CPC) initiative aimed to transform primary care delivery. 
  • Two years into CPC, Medicare patient ratings of care over time were generally comparable for CPC and comparison practices. 
  • There were statistically significant favorable effects in the proportion of patients giving the best responses for 3 of 6 composite measures of the Consumer Assessment of Healthcare Providers and Systems Clinician and Group 12-Month Survey with Patient-Centered Medical Home supplemental items: getting timely appointments, care, and information (2.1 percentage points); providers support patients in taking care of their own health (3.8 percentage points); and providers discuss medication decisions with patients (3.2 percentage points). 
  • Results suggest that transforming care during the first 2 years of CPC did not negatively affect patient experience but did generate some small improvements.
CMS is seeking to tie 50% of payments to quality or value through alternative payment models by 20181 by working with payers around the country to test the patient-centered medical home (PCMH) and similar models to improve primary care delivery and pay for value instead of volume.2 Thus, it is important to measure how this transformation is affecting the way in which patients experience care and to identify opportunities to continue to improve patient experience.

In a unique collaboration, CMS and 39 public and private healthcare payers launched the Comprehensive Primary Care (CPC) initiative in October 2012 to improve primary care delivery in the United States. CPC helped practices implement 5 key functions in their care delivery—1) access and continuity, 2) planned chronic and preventive care, 3) risk-stratified care management, 4) patient and caregiver engagement, and 5) coordination of care across the medical neighborhood—supported by continuous data-driven improvement, enhanced accountable payment, and optimal use of health information technology. CMS selected 502 practices in 7 US regions to participate. To help participating practices improve their care delivery, CPC provided them with enhanced payment, a learning system, and data feedback during the 4-year initiative.3-5

CPC aimed to improve cost, quality, and patient experience of care. This paper focuses on patient experience, examining how the ratings of more than 25,000 Medicare fee-for-service (FFS) beneficiaries attributed to 496 practices participating in CPC at the time of the first survey changed between the first and second years of CPC. This paper also identifies how ratings of CPC practices changed relative to the ratings of comparison practices, selected using propensity score matching,6 and areas where practices could still improve. 

Patient-centeredness was a core tenet of the model, and several aspects of CPC aimed to improve patient experience of care. Practices were expected to provide better access to care, engage patients in order to guide quality improvement through surveys and/or a patient and family advisory council, integrate culturally competent self-management support and shared decision-making tools into care, coordinate care across the medical neighborhood, and use a personalized plan of care for high-risk patients. In addition, patient experience was used to help determine eligibility for shared savings payments.

METHODS

Overview 

We conducted a repeated cross-sectional study using a large sample of Medicare FFS beneficiaries attributed to CPC practices and to a set of comparison practices selected using propensity score matching to have similar market-, practice-, and patient-level characteristics before CPC began. We examined changes in patient ratings and used difference-in-differences (DID) to evaluate how CPC practices’ ratings improved relative to comparison practices between 1 year (8 to 12 months) and 2 years (21 to 24 months) after CPC began. We did not draw inferences about effects from tests of each hypothesis separately, but rather from the findings across the set of questions and composites, particularly the summary composites. 

Setting 

CPC practices are primary care practices in 7 US regions: 4 states (Arkansas, Colorado, New Jersey, and Oregon) and 3 geographic areas (Cincinnati–Dayton [Ohio and Kentucky], Capital District–Hudson Valley [New York], and Greater Tulsa, Oklahoma). We drew comparison practices from: 1) those that had applied to CPC in the same regions as the CPC practices but were not selected, and 2) those in areas near the CPC regions that had reasonably similar demographics and market factors and had enough practices for matching.

Sample and Response Rates 

Using Medicare claims data, we attributed Medicare FFS beneficiaries to practices where they had most of their evaluation and management visits to primary care clinicians over the prior 2 years; using survey data, we identified attributed Medicare FFS beneficiaries who had visited the practice at least once in the 12 months before the survey round began. 

In each survey round, we mailed questionnaires to a random sample of an average of 119 attributed Medicare FFS patients from each CPC practice and an average of 24 attributed Medicare FFS patients from comparison practices. These sample sizes aimed to yield completed surveys with at least 40 attributed Medicare FFS respondents per CPC practice and 14 respondents per matched set of comparison practices (the larger sample in CPC practices supported practice-level feedback). We followed the National Committee for Quality Assurance’s sampling guidelines for the number of patients to sample in each practice; more patients were sampled in practices with more clinicians.7 The average number of completed surveys was 53 per CPC practice and 18 per comparison practice set, exceeding our targets of 40 and 14, respectively.

In 2013, we obtained response rates of 45% and 46% for CPC and comparison practices, respectively. We then excluded patients from 2 of the 497 CPC practices and their comparison matched sets because the calculated weights of the patients in those practices—which combined matching weights and nonresponse weights—were large outliers and would have unduly influenced the results. This left samples of 25,843 Medicare FFS patients in 495 CPC practices and 8949 Medicare FFS patients in 818 comparison practices. For the 2014 survey, we sampled patients from 496 CPC practices: 2 of the 497 total CPC practices in 2013 closed in summer/fall 2013 and 1 split into 2 practices in 2014. Similarly, the number of comparison practices in our sample fell from 818 in 2013 to 792 in 2014. In 2014, response rates were 48% and 47% for CPC and comparison practices, respectively. The final sample for the 2014 survey contained 26,356 Medicare FFS patients in 496 CPC practices and 8865 Medicare FFS patients in 792 comparison practices. About 15% of respondents replied in both survey rounds.   

Measurement of Patient Experience

Our patient survey instrument contains items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group 12-Month Survey with Patient-Centered Medical Home supplemental items.8 The survey asks patients about their experience of care over the previous 12 months across 6 dimensions of primary care: 1) patients’ ability to get timely appointments, care, and information; 2) how well providers communicate; 3) providers’ knowledge of the care patients received from other providers; 4) if providers support patients in taking care of their own health; 5) if providers discuss medication decisions with patients; and 6) patients’ overall rating of their primary care provider. To summarize patient experience of care, we created 6 summary composite measures using 19 questions following the CAHPS Clinician and Group survey scoring instructions.9 Table 1 details the specific patient care experiences that the 6 summary composite measures evaluate. Although CMS and some other payers used these composite measures to help determine whether practices received shared savings, CPC did not focus explicitly on each item. In addition to the 19 questions in the 6 summary measures, 25 other questions gauged patient experience of care, yielding 44 total questions (listed in eAppendix Table A [eAppendices available at ajmc.com]).

Survey Administration

We administered 2 rounds of the survey: 1) June through October 2013, 8 to 12 months after CPC began; and 2) July through October 2014, 21 to 24 months after CPC began. All surveys were administered by mail, following the CAHPS Clinician and Group survey instructions, with slightly modified timing of mailings. 

Analysis

We analyzed both the proportion of patients who gave the best (most favorable) response (response scales varied from 2-point [yes/no] to 11-point [0 to 10 rating scale]) and mean response. Our main analysis is on the best responses. Examples of these responses are: 1) the provider always explained things to the patient in a way that was easy to understand; 2) in the last 12 months, between visits, yes, the patient did receive reminders about tests, treatment, or appointments from the provider’s office; and 3) the patient got an appointment for care needed right away that same day. 

We first calculated the likelihood that patients responded to a question with the best response using logistic regressions, controlling for baseline patient and practice characteristics and education level reported on the survey. We calculated predicted probabilities for each of the 44 questions (eAppendix Table A reports results from each question).

In addition to analyzing responses to individual questions, we looked at the 6 summary composite measures containing 19 of the 36 questions asked in both rounds, following the CAHPS Clinician and Group survey scoring instructions.9 We first calculated patient-level composite measures by averaging nonmissing binary indicators for whether the patient’s response was the best option across each question in the composite. (That is, if the composite contained 4 questions and the respondent answered all 4 and gave the best response for 3, the patient’s score was 0.75.) Ordinary least squares regressions controlled for baseline patient and practice characteristics and the respondent’s education level. 

PDF is available on the last page.
Compendia
Adult ADHD Compendium
COPD Compendium
Dermatology Compendium
Diabetes Compendium
GI Compendium
Immuno-oncology Compendium
Lipids Compendium
MACRA Compendium
Oncology Compendium
Pain Compendium
Rare Disease Compendium
Reimbursement Compendium
Rheumatoid Arthritis Compendium
Know Your News
HF Compendium
Managed Care PODCAST