Medicare Shared Savings Program ACO Network Comprehensiveness and Patient Panel Stability

August 29, 2019

Medicare Shared Savings Program accountable care organization (ACO) network comprehensiveness is associated with stable patient assignment year to year. Panel stability was significantly associated with improved diabetes and hypertension control in the short term.


Objectives: The current Medicare Shared Savings Program (MSSP) accountable care organization (ACO) attribution methodology creates unpredictability for ACOs that are developing and deploying strategic initiatives aimed at improving value. The goal of this study is to determine if ACO network comprehensiveness is associated with the stability of assigned Medicare beneficiaries from 2013 to 2014.

Study Design: We utilized a beneficiary-level logistic regression model to determine association of network comprehensiveness with stable attribution to an MSSP ACO.

Methods: Using 2013 and 2014 Medicare fee-for-service beneficiary and provider files, we developed a measure of network comprehensiveness based on 2013 provider contracts, determined beneficiary attribution, and generated market-level measures. Additional population and quality measures were obtained from the US Census and the ACO Public Use File.

Results: Of the 1,317,858 observed beneficiaries, 84.38% were attributed to the same ACO in 2013 and 2014, and mean (SD) ACO network comprehensiveness was 0.30 (0.20). We found that a 0.10 increase in network comprehensiveness score significantly increased the odds of remaining attributed to the same ACO by 4.5% (P = .001). Patient panel stability was significantly associated with improved diabetes (P = .01) and hypertension (P = .02) control, timely access to care (P = .001), and delivery of health education (P = .03) over the 2-year period.

Conclusions: The comprehensiveness of an MSSP ACO’s contracted provider network is associated with stable patient assignment year to year. Patient panel stability may aid in the longitudinal management of some conditions.

Am J Manag Care. 2019;25(9):e267-e273Takeaway Points

This study provides initial insights into accountable care organization (ACO) structural impact on attribution over time.

  • With all new payment models, it is critical to know how providers respond so that iterative improvements can be made to achieve cost and quality objectives.
  • Patient panel stability was associated with improved diabetes and hypertension control, validating the ACO model’s ability to affect patient outcomes.
  • By constructing a provider network that mitigates the effect of the attribution process, the ACO can position itself for better performance on select quality measures.

The Medicare Shared Savings Program (MSSP) is an alternative payment model (APM) to traditional Medicare fee-for-service (FFS) and incentivizes the provision of efficient and effective healthcare through various levels of risk sharing.1 To participate in the MSSP, healthcare providers and organizations voluntarily collaborate and enter into a contract to create an accountable care organization (ACO). The ACO’s providers become collectively responsible for the overall quality and cost of care for assigned Medicare FFS beneficiaries.

Beneficiaries are assigned to an ACO based on primary care utilization of contracted providers.2,3 CMS specifies only 1 ACO eligibility requirement related to the composition of the contracted provider network. The ACO simply needs to include a sufficient number of primary care physicians to be assigned 5000 beneficiaries.1 This design latitude has led to the proliferation of uniquely organized and structured ACOs.4

Researchers have determined that ACO structural differences are influenced by external market forces.4,5 These differences have been categorized into measures of size of provider network, scope of services, breadth of provider type participation, proportion of primary care in network, leadership type, integrated delivery system membership, performance management strategies, and prior payment reform experience.4,6,7 Researchers have used structural and organizational measures to evaluate the cost and quality outcomes of ACOs, but these evaluations have yielded mixed results.7-9 Although other studies have used proxies for breadth and size of contracts, comprehensiveness of ACO provider networks has not been examined. Comprehensiveness is the inclusion of a minimum number of primary care and specialty providers necessary to serve a population. A more comprehensive network could increase access, improve provider communication, and reduce external utilization. It potentially offers ACOs greater control of the attribution process.

The attribution process mediates the relationship between organizational structure and outcome performance. CMS uses a prospective attribution method with retrospective reconciliation to assign beneficiaries to ACOs in the Track 1 and Track 2 MSSP models.2 A beneficiary must receive a plurality of primary care services from one of the contracted providers to be assigned to an ACO. This directly links the ACO’s network to the attribution process. ACO leadership receive preliminary panels during the performance year based on historic utilization. These panels are adjusted at the year’s end to reflect actual performance year utilization.10,11 The ACO’s cost and quality performance is calculated using this final list.

Organizations rely on complete and perfect information to make operational decisions that minimize risk.12-15 The retrospective reconciliation process introduces large uncertainty about the beneficiaries for whom an ACO is responsible. ACOs want to optimize the investment of organizational resources to improve population health and achieve MSSP goals.16 However, it is difficult to maximize performance without knowing the target that one needs to reach. The uncertainty produced by MSSP attribution can potentially affect an ACO’s proactive development and deployment of strategic interventions and initiatives aimed at improving value.10,11

Furthermore, the implementation of strategic and targeted population health initiatives often does not result in immediate and lasting changes. It can take years to modify utilization patterns, curve costs, and improve condition management—tenets central to the MSSP. This adds another important dimension to the MSSP attribution process: patient panel stability over time. When an attributed population remains stable year to year, the ACO has greater opportunity to enact interventions and witness returns on investment.17,18 Research has shown that ACO patient panel stability is moderately associated with better ACO performance.18,19

It is critical to evaluate supply-side response and perceptions to understand program impacts and to identify effective strategies and best practices. In this study, we expand on prior ACO organizational literature by exploring the mediating relationship of the attribution process. Specifically, we analyze if ACO patient panel stability from 2013 to 2014 is a result of provider network comprehensiveness in 2013. We then determine if patient panel stability is associated with changes in patient and caregiver experience performance and select quality metrics.


Conceptual Framework

We created a conceptual framework to detail the relationship between the ACO’s provider network and shared savings (Figure 1). The direct relationship illustrates that the ACO’s provider network achieves shared savings through changes to utilization. We hypothesized that the attribution process has a mediating effect on performance. We were particularly interested in the effects of the comprehensiveness of the ACO’s provider network, which we defined as the necessary number of providers in each specialty to provide sufficient access and care to a patient panel, on attribution. Comprehensive networks would increase the availability and accessibility of various provider types within the ACO, reducing the need to receive care outside of the ACO’s contracted providers. We hypothesized that the more complete the network, the more likely a beneficiary would be to remain attributed to an ACO from year to year. We referred to this as patient panel stability.

We included organizational structure, market, and patient demographic/care-seeking behavior factors that could influence the attribution process as covariates.

Data and Analysis

We used complete 2013 Medicare FFS beneficiary and provider files and partial 2014 Medicare FFS beneficiary and provider files for those beneficiaries with Medicare Part D from CMS. We first compared final attribution in both years to determine if the beneficiary remained in the same ACO, which was our primary dependent variable.

Next, we created the network comprehensiveness measure using the Medicare Advantage (MA) minimum number requirement for network adequacy.20 We selected this measure due to its application in Medicare, its overall acceptability, and our available data. We modified the included specialties to better reflect MSSP objectives and CMS’ attribution methodology.21 We included primary care providers (PCPs; general practice, family medicine, internal medicine, and geriatric) and all provider types delivering more than 1% of evaluation and management (E&M) visits to beneficiaries without a visit to a PCP. This resulted in the inclusion of 20 provider specialties listed in Table 1.

We used the 2013 Medicare MSSP provider file to count the number of contracted providers by specialty for each of the 2013 MSSP ACOs. Based on 2013 attribution, we used the MA Health Services Delivery table and the beneficiaries’ state-county codes to average the minimum number of providers per 1000 necessary for network adequacy in the ACO’s service region.20 We adjusted the minimum number of providers needed by specialty category to account for the size of an ACO’s attributed beneficiary panel.22 We then created proportions of the number of contracted providers over the minimum number of providers needed in each specialty. We capped the proportions at 1 and weighted each specialty to capture its magnitude. The weights were calculated from the proportion of minimum providers required in each specialty to the total minimum number of all providers required in the ACO. We added the 20 weighted proportions together to create a score of network comprehensiveness in 2013.

In addition to network comprehensiveness, we controlled for other variables included in the conceptual framework. We adjusted for ACO contract changes by creating a proportion of providers remaining within the ACO contract from 2013 to 2014 using the Medicare MSSP provider files. At the market level, we included a measure of an ACO’s footprint, defined as the proportion of providers delivering E&M services within a hospital service area (HSA) who are contracted with the ACO. We also generated a Herfindahl-Hirschman Index (HHI) measure of all Medicare FFS E&M providers at the HSA level to account for variation in market concentration among ACOs.

To adjust for complexity of care in attributed beneficiary populations, we included age and number of chronic conditions as defined by the Chronic Conditions Warehouse. We included number of E&M events and number of emergency department visits from the beneficiary summary file and applied CMS’ attribution methodology to determine if the beneficiary had the same attributing provider year to year to capture utilization patterns. Next, we included a series of patient population demographic variables, such as race, sex, and being disabled and/or having end-stage renal disease. We also used the Zip Code Tabulation Area—level demographic variables of percentage of individuals with a college degree, percentage living in a rural area, and median income to account for differences in care-seeking behavior.23-25 After creating these variables, we dropped 6 of the 220 ACOs that did not continue into 2014.

Because of our restricted sample in 2014, we wanted to adjust for any population-level differences between Part D and all other Medicare FFS beneficiaries. The Part D beneficiaries were slightly younger and had a greater number of chronic conditions than those without that benefit. We ran a 2-part model to estimate the probability of inclusion in our analysis. We included the predicted probability in a beneficiary-level logistic regression to determine the impact of network comprehensiveness on attribution to the same MSSP ACO in 2013 and 2014.

Finally, using 2013 and 2014 CMS ACO Public Use Files, we obtained information on each ACO’s quality performance for 6 selected measures that we believe are sensitive to ACO network composition and population management over time. We merged the performance results with an ACO-level file for a subanalysis. We ran generalized estimating equations (GEEs) for normal distribution models of ACO quality performance measures, including patient/caregiver experience measures (ACO 1, 4, and 5) and population health management performance (ACO 21, 27, and 28) in 2013 and 2014 to determine within-ACO change.22,26 We selected GEEs to allow for correlation in performance over time by ACO and to provide a population-averaged effect. We included the ACOs’ patient panel stability percentage as a key explanatory variable.

All analyses were conducted using SAS 9.4 (SAS Institute; Cary, North Carolina) and Stata 14.1 (StataCorp LP; College Station, Texas).


Among all 220 ACOs in 2013, the average network comprehensiveness score was 0.30 (median = 0.25), and there was substantial variation (SD = 0.20). The most comprehensive network had a score of 0.90. ACOs were most sufficiently contracted with PCPs, averaging 0.63 of the minimum providers necessary. Gynecology had the second highest score and was the only other specialty with a capped score higher than 0.5 (mean = 0.53). Three specialties (vascular surgery, dermatology, and oncology [medical, surgical]) averaged less than 0.10 of the contracted providers necessary to be considered adequate. The primary care category carried the largest weight in the network comprehensiveness score (weight = 0.29). Table 1 includes the results for all 20 specialties.

There were 2,645,025 beneficiaries attributed to the included 214 ACOs in 2013. We matched 1,317,858 of these beneficiaries to our sample of 2014 Medicare beneficiaries with Part D coverage. A majority (84.38%) of these beneficiaries remained attributed to the same ACO between 2013 and 2014. Structurally, the ACO’s provider network was also relatively consistent, with an average of 84.06% (median = 89.06%) of providers remaining contracted to the same ACO. The ACOs were operating in markets with low concentrations (HHI <1500) and contracted with an average of 9.38% of the E&M providers in the HSAs served (median = 6.80%). Beneficiaries were aged an average of 71.42 years, and the majority were white (84.93%) and female (59.96%) (Table 2).

Using the logistic regression, displayed in Table 3, we found that after controlling for market influences, patient demographic/care-seeking behavior, and other organizational factors, a 0.10 increase in the network comprehensiveness score was associated with a 4.5% increase in the odds of a beneficiary remaining attributed to an ACO (P <.001). The percentage of providers remaining contracted to the ACO was also associated with greater odds of attribution stability (odds ratio [OR], 1.02; P <.001). From a market perspective, a 1% increase of providers contracted to an ACO within a Dartmouth Atlas—defined HSA increased the odds of maintaining an attributed patient from 2013 to 2014 by 2.0% (P <.001). Related to care-seeking behavior, greater numbers of E&M visits (OR, 0.99; P <.001) and hospital emergency department visits (OR, 0.98; P <.001) and being disabled or having end-stage renal disease (OR, 0.92; P <.001) were associated with lower odds of being attributed to the same ACO year to year. A beneficiary receiving a majority of their E&M services from the same provider in both years, captured by the same attributing provider covariate, increased the odds of remaining attributed to the same ACO by 5.99 times. This was significant even after adjusting for ACO consistency in provider contracts (P <.001).

To determine the association with quality, we ran 6 separate GEEs for normal distribution models. For patient/caregiver experience measures, patient panel stability was positively associated with the receipt of timely care, appointments, and information (β = 0.06; P = .01) and the provision of health promotion and education (β = 0.04; P = .03). The measure was not significantly associated with ACO performance related to access to specialists. For the population health management measures, we found that patient panel stability was significantly associated with decreases in the percentage of beneficiaries with glycated hemoglobin values indicating poor control (β = —0.115; P = .01) and with increases in the percentage of blood pressure readings under 140/90 mm Hg among patients with hypertension (β = 0.10; P = .02) (Figure 2; full results in eAppendix [available at]). The measure of network comprehensiveness was also significantly associated with better diabetes control (β = —7.92; P = .01). Patient panel stability was not associated with blood pressure screenings.


This study provides initial insights into ACO structural impact on attribution over time. We hypothesized that the current retrospective nature of the Track 1 and Track 2 MSSP ACO attribution process introduces many potentially mediating factors that influence the stability of patient panels. Stable patient panels may allow clinicians to efficiently and effectively build and target programs to achieve cost and quality benchmarks. We found that several structural and market influences increased the likelihood of a beneficiary staying with an ACO in 2013 and 2014. In particular, the more comprehensive the ACO provider network in 2013, the greater the odds of patient panel stability in 2014. We also found that patient panel stability is associated with several positive quality outcomes.

ACOs operate in HSA markets with low concentrations (HHI <1500), meaning that most beneficiaries have the choice of many E&M providers.27 If an attributed beneficiary sees an external provider, the ACO may not be able to coordinate care plans across providers or reduce unnecessary testing and imaging. Visits to noncontracted providers can affect attribution to the ACO, as well as cost and quality performance. By improving network comprehensiveness, ACOs increase the accessibility and availability of contracted providers. It can limit the need to seek care outside of the ACO’s network and reduces the threat of panel instability. Similarly, contracting with a greater number of providers within an HSA enlarges the ACO’s footprint, increasing odds of a beneficiary remaining attributed to an ACO year over year. This could be due to the reduced potential for leakage. Our results indicate that ACO leadership can better maintain and anticipate attributed patient panels by strategically structuring the organization’s provider network.

The quality models showed positive associations between certain disease management measures and patient panel stability. Specifically, patient panel stability was associated with improved diabetes and hypertension control. Our findings validate the conceptualization of the ACO model and its ability to affect longitudinal patient-level outcomes. Although the results are promising, we are unable to determine specific provider behaviors that are responsible for the improvements or explain why patient panel stability is associated with improvements only in certain measures. Qualitative studies should continue to be conducted to understand the impact of ACO care coordination, disease management, and provider—beneficiary relationship on measures over time.


The cross-sectional nature of this study limited our ability to understand the impact of organizational network comprehensiveness and other important covariates on patient panel stability over multiple years. Relatedly, because we were using administrative claims, we were restricted in our ability to control for management structure, use of care coordination and case management, and hospital-associated or provider-led status.7,28,29 We also recognize the potential for endogeneity between attribution to the same provider and the outcome variable. We decided to include attribution to the same provider, capturing consistency of provider—patient relationships, as an important measure of care-seeking behavior. We were limited by the data available to us for this study. We had complete demographic and claims information only for Part D beneficiaries in 2014. Although we tried to control for differences, the findings may not be generalizable to the full Medicare FFS population. Our design also excluded from this analysis those initiating care or aging into Medicare in 2014.

Finally, we chose to utilize a MA specification for network adequacy in our measure development. We selected this definition for our measure of network comprehensiveness because it is a widely accepted measure for the Medicare population and it allowed us to capture a wide scope of provider specialties. The use of time and distance from beneficiary to provider was not feasible in this study based on available data. Although we selectively chose provider types and weighted values to stratify specialty importance, selected provider types and the overall definition may not be appropriate for ACOs. The network adequacy specifications ensure access to care for members, and ACOs are not able to restrict access in the same way as those plans do. Still, this provides a benchmark for the Medicare population that can be used to understand ACO network comprehensiveness and how it is associated with patient attribution and performance.


As implementation of the Medicare Access and CHIP Reauthorization Act of 2015 continues, CMS will be providing bonus payments for Medicare provider participation in APMs like the MSSP ACO model. CMS hopes to have a majority of providers participating in advanced APMs in the near future, which requires downside risk. The push to participate in APMs and the escalating risk associated with participation will exert pressure on providers to plan, structure, and implement the models and programs in the strongest manner possible. It is critical to know how the supply side perceives and responds to incentives so that adjustments can be made to improve overall program performance.

This study contributes to the existing literature and informs future decision making. Our findings are important because they can empower ACOs to compose a provider network that mitigates the effect of the attribution process, positioning the ACO for better performance on select quality measures. Researchers will need to continue to explore organizational influences on ACO performance to identify and spread best practices.&ensp;


We would like to thank Dr Yuting Zhang for use of Medicare administrative claims data obtained as part of a grant (20150380) from The Commonwealth Fund.Author Affiliations: Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh (CL, EC, AEJ, JD), Pittsburgh, PA; Health Policy Institute, University of Pittsburgh (CL, AEJ), Pittsburgh, PA.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (CL, EC, JD); analysis and interpretation of data (CL, EC, JD); drafting of the manuscript (CL, EC, JD); critical revision of the manuscript for important intellectual content (CL, EC, AEJ, JD); statistical analysis (CL, JD); administrative, technical, or logistic support (AEJ); and supervision (AEJ, JD).

Address Correspondence to: Cassandra Leighton, MPH, Health Policy Institute, University of Pittsburgh, Scaife Hall, Ste S310, 3550 Terrace St, Pittsburgh, PA 15261. Email:

1. About the program. CMS website. Updated June 3, 2019. Accessed June 4, 2018.

2. Medicare Shared Savings Program: shared savings and losses and assignment methodology: specifications. CMS website. Published April 2017. Accessed June 4, 2018.

3. Gold J. Accountable care organizations, explained. Kaiser Health News website. Published September 14, 2015. Accessed June 4, 2018.

4. Shortell SM, Colla CH, Lewis VA, Fisher E, Kessell E, Ramsay P. Accountable care organizations: the national landscape. J Health Polit Policy Law. 2015;40(4):647-668. doi: 10.1215/03616878-3149976.

5. Lewis VA, Tierney KI, Colla CH, Shortell SM. The new frontier of strategic alliances in health care: new partnerships under accountable care organizations. Soc Sci Med. 2017;190:1-10. doi: 10.1016/j.socscimed.2017.04.054.

6. Fisher ES, Shortell SM, Kreindler SA, Van Citters AD, Larson BK. A framework for evaluating the formation, implementation, and performance of accountable care organizations. Health Aff (Millwood). 2012;31(11):2368-2378. doi: 10.1377/hlthaff.2012.0544.

7. Wu FM, Shortell SM, Lewis VA, Colla CH, Fisher ES. Assessing differences between early and later adopters of accountable care organizations using taxonomic analysis. Health Serv Res. 2016;51(6):2318-2329. doi: 10.1111/1475-6773.12473.

8. McWilliams JM, Chernew ME, Zaslavsky AM, Hamed P, Landon BE. Delivery system integration and health care spending and quality for Medicare beneficiaries. JAMA Intern Med. 2013;173(15):1447-1456. doi: 10.1001/jamainternmed.2013.6886.

9. McWilliams JM, Hatfield LA, Chernew ME, Landon BE, Schwartz AL. Early performance of accountable care organizations in Medicare. N Engl J Med. 2016;374(24):2357-2366. doi: 10.1056/NEJMsa1600142.

10. Re: CMS-1461-P Medicare Program; Medicare Shared Savings Program; Accountable Care Organizations. American College of Physicians website. Published February 6, 2015. Accessed July 22, 2018.

11. Lewis VA, McClurg AB, Smith J, Fisher ES, Bynum JPW. Attributing patients to accountable care organizations: performance year approach aligns stakeholders’ interests. Health Aff (Millwood). 2013;32(3):587-595. doi: 10.1377/hlthaff.2012.0489.

12. Mühlbacher AC, Amelung VE, Juhnke C. Contract design: the problem of information asymmetry. Int J Integr Care. 2018;18(1):1. doi: 10.5334/ijic.3614.

13. Folland S, Goodman AC, Stano M. The Economics of Health and Health Care. 6th ed. Upper Saddle River, NJ: Prentice Hall; 2009.

14. Frank RG. Behavioral economics and health economics. National Bureau of Economic Research working paper 10881. Published November 2004. Accessed July 28, 2018.

15. Kahneman D, Knetsch JL, Thaler RH. Anomalies: the endowment effect, loss aversion, and status quo bias. J Econ Perspect. 1991;5(1):193-206. doi: 10.1257/jep.5.1.193.

16. Casalino LP, Erb N, Joshi MS, Shortell SM. Accountable care organizations and population health organizations. J Health Polit Policy Law. 2015;40(4):821-837. doi: 10.1215/03616878-3150074.

17. Christensen EW, Payne NR. Effect of attribution length on the use and cost of health care for a pediatric Medicaid accountable care organization. JAMA Pediatr. 2016;170(2):148-154. doi: 10.1001/jamapediatrics.2015.3446.

18. McWilliams JM, Chernew ME, Dalton JB, Landon BE. Outpatient care patterns and organizational accountability in Medicare. JAMA Intern Med. 2014;174(6):938-945. doi: 10.1001/jamainternmed.2014.1073.

19. Ouayogodé MH, Colla CH, Lewis VA. Determinants of success in Shared Savings Programs: an analysis of ACO and market characteristics. Healthc (Amst). 2017;5(1-2):53-61. doi: 10.1016/j.hjdsi.2016.08.002.

20. Medicare Advantage network adequacy criteria guidance. CMS website. Published January 10, 2017. Accessed October 24, 2017.

21. Contract year (CY) 2017 Medicare Advantage health service delivery (HSD) provider and facility specialties and network adequacy criteria guidance and methodology. CMS website. Accessed October 24, 2017.

22. 2013 Shared Savings Program Accountable Care Organizations (ACO) PUF. CMS website. Published November 14, 2016. Metadata updated September 21, 2017. Accessed February 17, 2018.

23. US Census Bureau. Urban and rural: housing units. American FactFinder website. Published 2010. Accessed October 30, 2017.

24. US Census Bureau. Educational attainment: 2009-2013 American Community Survey 5-year estimates. American FactFinder website. Published 2013. Accessed October 30, 2017.

25. US Census Bureau. Median income in the past 12 months (in 2013 inflation-adjusted dollars): 2009-2013 American Community Survey 5-year estimates. American FactFinder website. Published 2013. Accessed October 30, 2017.

26. 2014 Shared Savings Program (SSP) Accountable Care Organizations (ACO) PUF. CMS website. Published October 30, 2015. Metadata updated September 21, 2017. Accessed August 3, 2018.

27. Herfindahl-Hirschman Index. US Department of Justice website. Updated July 31, 2018. Accessed August 3, 2018.

28. Shortell SM, Wu FM, Lewis VA, Colla CH, Fisher ES. A taxonomy of accountable care organizations for policy and practice. Health Serv Res. 2014;49(6):1883-1899. doi: 10.1111/1475-6773.12234.

29. Muhlestein D, Saunders RS, McClellan MB. Medicare accountable care organization results for 2015: the journey to better quality and lower costs continues. Health Affairs Blog website. Published September 9, 2016. Accessed August 14, 2018.

Medicare Shared Savings Program ACO Network Comprehensiveness and Patient Panel Stability