Currently Viewing:
The American Journal of Accountable Care March 2018
Medicare Accountable Care Spending Patterns: Shifting Expenditures Associated With Savings
David B. Muhlestein, PhD, JD; Spencer Q. Morrison, BA; Robert S. Saunders, PhD; William K. Bleser, PhD, MSPH; Mark B. McClellan, MD, PhD; and Lia D. Winfield, PhD
Currently Reading
ACO Quality Over Time: The MSSP Experience and Opportunities for System-Wide Improvement
William K. Bleser, PhD, MSPH; Robert S. Saunders, PhD; David B. Muhlestein, PhD, JD; Spencer Q. Morrison, BA; Hoangmai H. Pham, MD, MPH; and Mark B. McClellan, MD, PhD
A Managed Care Organization's Call Center–Based Social Support Role
Zachary Pruitt, PhD; Pamme Lyons Taylor, MBA, MHCA; and Kristopher M. Bryant, BS
The Skills of the Ambulatory Intensivist: A Review
Craig Tanio, MD, MBA, FACP; and Arnold R. Eiser, MD, MACP
Thirty-Day Readmissions: Relationship to Physician Attending Type and Social Connectedness
Carey C. Thomson, MD, MPH; Nathalie Bloch, MD, MPA; Tafadzwa Muguwe, MD, MS; Kendell Clement, PhD; Shani Legore, BA; Orissa Viza, MSW, MPH; Joanne Kerwin, PhD; and Valerie E. Stone, MD, MPH
Outpatient Referral Rates in Family Medicine
Maribeth Porter, MD, MS; John Malaty, MD; Charlie Michaudet, MD; Paulette Blanc, MPH; Jonathan J. Shuster, PhD; and Peter J. Carek, MD, MS
Predictive Factors of Discharge Navigation Lag Time
Charles Walker, MD; Sayeh Bozorghadad, BS; Leah Scholtis, PA-C; Chung-Yin Sherman, CRNP; James Dove, BA; Marie Hunsinger, RN, BSHS; Jeffrey Wild, MD; Joseph Blansfield, MD; and Mohsen Shabahang, MD, PhD

ACO Quality Over Time: The MSSP Experience and Opportunities for System-Wide Improvement

William K. Bleser, PhD, MSPH; Robert S. Saunders, PhD; David B. Muhlestein, PhD, JD; Spencer Q. Morrison, BA; Hoangmai H. Pham, MD, MPH; and Mark B. McClellan, MD, PhD
From 2013 to 2016, Medicare Shared Savings Program accountable care organizations (ACOs) improved quality. Continued infrastructure development funding, better relationships with postacute care facilities, and shared learnings among diverse ACOs would maximize quality improvement.

Objectives: To investigate accountable care organization (ACO) quality improvement over the first 4 Medicare Shared Savings Program (MSSP) years.

Study Design: Fixed-effects analysis examined associations of within-ACO MSSP quality metric changes with key time-variant ACO traits: changes in postacute care (PAC) expenditure and size (attributed beneficiaries). Fixed-effects subgroup analyses and linear regression were used for key time-invariant traits: ACO taxonomy (physician-led, hospital-led, or co-led), risk-bearing maturity, commercial contract presence, and rurality.

Methods: The sources of data were secondary MSSP public use files linked to the Leavitt Partners ACO Database (ACO panel: n = 528; 2013-2016).

Results: Confirming early federal findings, MSSP ACOs, on average, improved most quality measures. Larger ACOs had higher quality, but ACOs grew rapidly for the first 3 years, bringing “growing pains” in quality measures related to clinical care for at-risk populations, before plateauing in size in the fourth year. By comparison, PAC expenditures increased in the first year but then decreased in all remaining years, and PAC spending changes were inversely associated with quality, especially in quality measures related to care coordination and patient safety. Successes and challenges varied most notably by ACO taxonomy, risk-bearing maturity, and rurality.

Conclusions: MSSP ACOs improved quality despite their sicker, older population, suggesting that the model might work in other settings and populations and could shift to more advanced risk and payment models (eg, population-based prospective payment). Continued ACO infrastructure development funding, better relationships with PAC facilities, and opportunities for diverse ACOs to share their learnings would maximize quality improvement.

The American Journal of Accountable Care. 2018;6(1):e1-e15
Accountable care organizations (ACOs) receive shared savings by establishing a local healthcare delivery collaboration to coordinate across the full care continuum while improving care quality and reducing care costs below a specified benchmark.1 They are a promising approach to address care fragmentation and achieve the Triple Aim in the US health system.2-6

Increasingly widespread, ACOs cover about 10% of the US population.7 Congress established the Medicare Shared Savings Program (MSSP) as 1 of several ACO programs implemented by CMS to incorporate ACOs into Medicare.8 As of early 2018, there are 561 MSSP ACOs representing 10.5 million lives,9 making it the largest of Medicare’s ACO programs. CMS tracks MSSP ACO performance and reports that these ACOs have achieved reductions in cost while improving, and outperforming Medicare fee-for-service (FFS) providers on, most quality measures.10,11

Given the widespread nature of this delivery model, there is a need for evidence of its impact on healthcare quality. Longitudinal examinations of ACO quality have found that ACOs were associated with reduced utilization of postacute care (PAC), length of skilled nursing facility (SNF) stays,12 mortality,13 readmissions,14,15 and chronic obstructive pulmonary disease or asthma admissions16, and improved patient experience,17 chronic disease management, and preventive and pediatric care, especially among populations with low socioeconomic status.18,19 Cross-sectional study findings further indicate that beneficiary race/ethnicity,20 ACO size and leadership style,21 provider mix,22 and urban/rural county23 are associated with quality.

Existing longitudinal literature focuses on comparing the ACO model with a counterfactual of other care models (namely, FFS), which is valuable and needed evidence given that the literature is still young and mixed. By comparison, given the expansion and prevalence of the MSSP, we examined key factors affecting within-ACO quality improvement in the program (ie, the counterfactual is an ACO compared with itself at an earlier time; we did not compare ACOs with non-ACOs), an area needing better evidence,21 so that we may understand how ACOs function maximally within the MSSP to improve quality, a key program goal.


Data Sources and Study Population

We utilized 2 data sources. First, we used all publicly available MSSP ACO data files, containing quality scores, financial performance, and descriptive data on ACOs, their providers, and beneficiaries.24-27 Performance year 1 contains 220 ACOs from when they began (at certain points in 2012 or on January 1, 2013) through the end of 2013, and performance years 2, 3, and 4 contain 333, 392, and 432 ACOs, respectively, during calendar years 2014, 2015, and 2016. Second, we used proprietary data on ACO taxonomy type, geography, and risk-bearing arrangements from the Leavitt Partners ACO Database, which has tracked ACOs since 2010 using news releases, public reports, industry directories, surveys, and interviews.15,28-30 These were merged by MSSP ACO identifiers to construct a panel of ACOs from 2013 to 2016.


The objective was to examine how quality metrics changed within ACOs over time. This is represented by MSSP ACO quality performance metrics across 4 quality domains: patient/caregiver experience, care coordination/patient safety, clinical care for at-risk populations, and preventive health. Across 2013-2016, there were 44 measures (42 single measures plus 2 composites) that came from claims, patient survey, or ACO-reported data.31 In any given year, there were 33 to 34 single measures (some changed or were replaced over time). After careful review of measure definitions (see Table 1), in regression analyses we omitted certain years of measures before or after significant specification changes in a way that maximized data-years included but avoided spurious findings, a potential issue with the federal government’s early MSSP quality examinations.

We were primarily interested in 6 key independent variables related to ACO structure, function, relationships, and geography discussed or hypothesized in the literature as important to ACO care quality and success: 1) taxonomy (primarily led by a hospital, physicians, or both),32,33 2) percentile of risk-bearing maturity compared with other ACOs, 3) presence of a commercial contract,34 4) expenditures on PAC,12,32,35,36 5) size (attributed beneficiaries), and 6) rurality.23 These were drawn or derived from the Leavitt Partners ACO Database (except size and PAC expenditures, which are publicly available).

Other variables can confound these relationships, so we included the following as covariates (ACO-level): beneficiaries’ age, sex, dual-eligible status, and Hierarchical Condition Category risk scores (to attempt case-mix adjustment); provider mix; patient-to-provider ratio; per capita benchmark; and market context (ratio of market cost relative to national cost).

Statistical Analyses

First, we used linear regression to examine cross-sectional adjusted associations between average quality performance and covariates (between-ACO associations), and then we used fixed-effects linear regression to longitudinally examine quality improvement (within­ACO associations). Fixed-effects methodology mechanically controls for time-invariant traits (measured or not).37 We regressed each quality measure onto all time-variant independent variables, covariates, and fixed-effects dummies for ACO and year. Only 2 key independent variables, PAC expenditures and size, had time-variant data. However, we incorporated key time-invariant traits by examining how significant PAC and size coefficients from fixed-effects models differ when stratified by ACO taxonomy, commercial contract presence, rurality, and risk-bearing maturity (“subgroup analyses”). All analyses were conducted in Stata/SE 15.1 (StataCorp LP; College Station, Texas) using heteroskedasticity-robust standard errors.37


Table 2 presents descriptive statistics of quality metrics and independent variables by year and change scores (ie, differences in scores) between years. Almost all quality measures improved across the 4 years, except that roughly half of the patient/caregiver experience measures showed small decreases until year 4, when they increased. As many quality measures are assessed on different scales, our analysis also calculated relative change scores to mimic early CMS methods.38 We found similar but more conservative results than did early federal government reports,10,11 likely due to our study only focusing on trends involving no measure definition changes. The largest, most consistent quality improvements were in preventive health. Larger improvements were notable in heart failure admission rates, screening for fall risk, pneumonia vaccination, and screening/follow-up for clinical depression.

Table 3 shows key independent variable coefficients from linear regressions of each quality measure onto all covariates (comparison groups: ACO vs other ACOs). Table 4 shows statistically significant key time-variant independent variable coefficients (PAC expenditures change and ACO size change) from fixed-effects regressions of each quality measure with 2 or more years of data onto all tim­e-variant coefficients and subgroup analyses, decomposing these coefficients by key time-invariant independent variables (comparison groups: ACO vs only itself at different times). Below, we summarize key stories from these tables.

First, cross-sectionally, there were significant positive associations between size and quality, particularly in clinical care for at-risk populations (Table 3). However, within-ACO change in size was inversely associated with quality changes, also notably among clinical care for at-risk populations (Table 4). Subgroup analyses showed that this inverse association varies substantially by ACO taxonomy and maturity, the highest magnitude being among physician-led ACOs and those less experienced in terms of program time and risk-bearing maturity. Figure 1 provides an illustrative example using 2013-2014 change scores in the coronary artery disease composite: Although most ACOs succeeded in improving quality, ACOs in the highest quartile of size change (+2289 to +40,091 beneficiaries) had much smaller improvements than those in the lowest quartile (–8209 to –294), varying substantially by ACO taxonomy and maturity.

Second, there is a clear relationship between PAC expenditures and quality. Cross-sectionally, there are consistent inverse associations between PAC expenditures and most quality measures, except for patient/caregiver experience measures (Table 3). Within-ACO changes in PAC expenditures had inverse associations with changes in care coordination/patient safety, notably all measures related to unplanned hospital admissions and readmissions (Table 4). Subgroup analyses showed this inverse association to generally be mitigated among physician-led ACOs and ACOs more experienced in terms of time in the program and risk-bearing maturity. Figure 2

provides an illustrative example using 2014-2015 change scores in all-cause readmissions: Although most ACOs decreased readmissions, ACOs in the highest quartile of PAC expenditure change (+0.5% to +10.5%) had much smaller decreases than those in the lowest quartile (–10.4% to –1.5%), varying substantially by ACO taxonomy and maturity.

The presence of commercial contracts or rural settings showed inconsistent findings in subgroup analyses, but we examined them and other time-invariant traits in cross-sectional results (Table 3). First, hospital-led ACOs had higher average quality in many measures of preventive care and clinical care for at-risk populations, although provider-led ACOs often had higher patient experience scores. Second, risk-bearing maturity had a consistent positive association with quality. Last, ACO rurality was generally associated with better quality, except in care coordination and patient safety.


This study examined in depth the first 4 years of MSSP data, finding 4 main quality improvement conclusions: 1) ACO quality appears to be broadly improving, highlighting the potential success of the MSSP model; 2) although ACO size was positively correlated with quality, ACOs experienced some quality challenges while growing; 3) PAC expenditures increased then decreased, which was associated with quality changes; and 4) these findings vary by key organizational traits. We discuss these findings below in the context of theory and prior work, then conclude with implications for practice, policy, and quality improvement in the broader healthcare system.

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up