Carrie H. Colla, PhD

Because hospitals and health systems sponsored the majority of new accountable care organizations (ACOs) from 2010 to 2015, they influenced priorities and strategies of the policies designed to drive ACO adoption. In recent years, however, the majority of new ACOs have been sponsored by physician groups. This shift means that policies need to be developed with the characteristic strengths and weaknesses of physician-led ACOs in mind. Using data from the Leavitt Partners ACO database, we analyzed the types of providers becoming ACOs over time to look at their numbers and market potential. Because the market potential for further growth of physician group—led ACOs is much stronger than for hospital- or health system–led ACOs, policy makers need to create programs and policies that facilitate physician-led ACOs’ success by helping them develop the capacity to take on risk, finance investments in high-value healthcare, and partner with other organizations to provide the full spectrum of care.

New accountable care organizations (ACOs) are increasingly led by physician groups rather than hospital systems.

In 2018, physician group—led ACOs represented approximately 45% of all ACOs, hospital-led ACOs accounted for approximately 25%, and joint-led ACOs represented 30%.

There is greater market potential for new physician-led ACOs than for those led by hospital systems, so physician-led ACOs will likely be the dominant type of ACO in the future.

Because hospitals and health systems sponsored many early ACOs, policies, payment models, and care delivery models initially focused on building competencies for these groups. However, these policies and strategies may need to be reconsidered as the model moves forward.

From 2010 to 2015, hospitals or health systems sponsored the majority of new accountable care organizations (ACOs). This allowed them to influence the priorities and strategies of policy makers as they created regulations and polices designed to drive these organizations. However, in recent years, the ACO market has seen a shift in leadership as physician group organizations have begun to lead the majority of new ACOs (

). This trend, and the significantly higher market potential for physician groups, suggest that they will continue to lead a significant number of new ACOs. With this change, policy makers and practitioners must consider the unique needs and opportunities of physician groups in the transition to value-based care.

ACOs consist of healthcare providers that accept responsibility for the cost and quality outcomes of a defined population. The defining characteristic of an ACO is the contract that establishes the provider group as being financially accountable for value. However, the type of care delivery changes that can be implemented to achieve ACO goals is determined by an ACO’s organizational structure.

In response to the policy environment, many early ACOs were large organizations that included hospitals, had prior experience with risk-based contracts, and had access to capital to invest in new technologies and start-up costs.

 For example, of the 10 participants in the Physician Group Practice Demonstration—the precursor to the Medicare Shared Savings Program (MSSP)&mdash;8 were hospital affiliated, 8 had experience with performance-based payment arrangements,

 and 8 had prior access to, or funding for, electronic health records (EHRs) and other information technology (IT) systems to track data. Because many early ACOs were hospital affiliated, policies, payment models, and care delivery models originally focused on building competencies for similar organizations. However, these policies and strategies may need to be reconsidered as the model moves forward.

Presently, ACOs range from large integrated delivery systems to hospitals with partner practices to multispecialty physician groups to small primary care physician groups.

 To understand changes in care delivery, it is important to understand the organizations participating, their characteristics and capabilities, and new participants attracted to these models. Our study reveals that physician groups—rather than hospitals or health systems&mdash;are becoming the dominant type of ACO and represent the largest potential type of organization to join the model. We use a simplified provider type to separate ACOs into 3 categories: hospital-led ACOs, physician group—led ACOs, and ACOs that are led by both a hospital and physician group. ACOs with hospitals are generally well financed, include an established health IT (HIT) infrastructure, and either have implemented or are experimenting with sophisticated data analytics systems.

 In contrast, average physician group ACOs tend to have less sophisticated HIT capabilities and data analytic tools

 and tend to be smaller, averaging a patient population of 20,000 lives, whereas ACOs with hospitals average 44,000 lives. ACOs across and within these groups have varying needs, competencies, and capabilities.

Data on ACO provider types were obtained from the Leavitt Partners ACO database, which tracks organizations that are participating in accountable care payment arrangements and includes information on organizational structure.

 Hospital affiliation or ownership by an ACO in the database has been previously validated through surveys.

 Determination of whether the ACO is physician, hospital, or jointly led is based on qualitative assessment of the broader organization, not just the providers participating in an ACO contract. For example, a health system consisting of hospitals and physicians may participate in the MSSP but may only list primary care physician practices on the official participant list. Because no hospitals are officially listed as participants, some evaluators would consider this “physician led,” but we consider this to be jointly led because the health system, including the hospitals, established and directs the ACO. Physician-led ACOs, then, should be viewed as organizations that are involved in ACOs that do not involve hospitals directly in the payment arrangement or in the broader organization. The data include all ACOs in the database with information on their provider type, which represented 1221 of 1334 ACOs as of the end of 2018. For this paper, we track only the year of their first contract and include organizations that subsequently dropped out of all ACO programs in the aggregate estimates. Estimates of total market size are derived from Torch Insight,

 a commercial healthcare data aggregator.

Figure 1 provides an overview of the types of providers that are becoming ACOs over time. The figure should be interpreted as the percentage of ACOs that fall into each category for each year. In 2010, 22% of ACOs were led by physician groups only, 63% of new ACOs were led by both a hospital system and a provider group, and 16% were led by hospital systems only; 79% of all ACOs in 2010 included a hospital in the ACO’s structure.

By 2016, the market changed, and 55% of new ACOs did not have a hospital participant. This continued into 2017, when 62% of all ACOs were physician group—led ACOs, although in 2018, only 45% were. The broader trend has been for the proportion of both hospital-led ACOs and joint hospital and physician group–led ACOs to decline.

In the aggregate in 2018, physician group—only ACOs represented approximately 45% of all ACOs, whereas hospital-led ACOs accounted for approximately 25% and joint-led ACOs represented 30%. This indicates that although physician-led ACOs represent the majority of new entrants, they are a minority of total ACOs, but they may become the dominant type of ACO in the country within the next few years if the trend of new entrants continues. A figure representing overall ACOs is available in the 

Estimating the market potential for ACOs is challenging, but we performed back-of-the-envelope calculations to get a sense for ACOs’ market penetration and potential. We first estimated that health systems or independent hospitals that included short-term acute care hospitals could potentially form an ACO and that physician groups with at least 15 providers could potentially form an ACO (based on the minimum number of physicians for the smallest ACOs). Ongoing consolidation could decrease this total market potential when larger groups or health systems combine, or it could increase this number as smaller physician groups merge to become large enough to enter into an accountable care contract. Also, some physician groups are owned by hospital systems, which may decrease the market potential for potential physician-led ACOs that do not include a hospital.

 We estimate that as of December 2018, 28% of existing health systems or independent hospitals were participating in an ACO of the more than 1700 hospitals or systems that could potentially form an ACO. In comparison, only 6% of the more than 8200 physician groups that are large enough to ultimately form an ACO have done so. 

Although 45% of ACOs now comprise physician groups without a hospital partnership, the market potential for further growth of physician group—led ACOs is much stronger than for hospital- or health system–led ACOs. Our market potential estimates are admittedly rough and should be viewed as only directionally correct, but they suggest that there is more room for physician-led ACOs to develop than for ACOs that include hospitals. The sheer number of physician groups suggests that the dominant type of provider moving forward that has the potential to bear risk will not be hospital based. Additionally, the Medicare Access and CHIP Reauthorization Act, which is targeted to physicians, may further the adoption of ACO models by physician groups so that they can avoid the Merit-based Incentive Payment System and potentially receive the additional 5% boost from participating in an advanced alternative payment model.

 with control over much of the care spectrum were early adopters of the ACO model, physician groups are increasingly becoming the predominant provider type involved in accountable care. As physician groups tend to have less experience managing risk, less access to capital, lower overhead, and less-integrated EHRs,

 policy makers need to understand how increased numbers of physician group—led ACOs will necessitate changes in policy initiatives.

Physician-led ACOs have done better at achieving savings and improving quality scores despite less access to capital, less experience managing risk, and less sophisticated HIT systems.

 MSSP results have consistently shown that smaller, physician-led ACOs are more likely to earn shared savings than hospital-led or integrated hospital and physician group—led ACOs.

 From a business perspective, physicians are able to focus on eliminating costly hospital admissions without suffering a drop in inpatient revenue.

 ACOs with hospitals may struggle with decreasing inpatient admissions, which can cause issues with maintaining capital-intensive inpatient facilities.

Although physician group ACOs tend to achieve better savings results, lack of capital is a major barrier to market entry and the building of competencies for success. Physician group ACOs often lack the capital to invest in sophisticated HIT systems, data analytics, and other technology capabilities.

 Without technology, care coordination and management are significant challenges, although some can be met with key management partners.

Financial risk may be another concern as more physician groups enter the ACO market. With little capital or experience in managing care and costs across populations, smaller physician group organizations may not be capable of adequately managing risk. Provider groups in the 1990s attempted to assume full capitation risk and eventually suffered massive financial losses.

 Policy makers should be wary of pushing too much risk onto physician-led ACOs too quickly and consider initiatives that promote risk management among smaller organizations. The Advanced Payment Model demonstration program and ACO Investment Model did furnish capital to physician-led ACOs, but these programs have been closed to new entrants, and restarting a similar model could be considered.

 The recent Pathways to Success program does limit the amount of risk that smaller organizations must bear through a “low-revenue” option, and it is apparent that CMS is seriously considering the differences between ACOs with and without hospitals.

 Despite this focus, recent ACO dropouts have disproportionately been physician led, suggesting the need for policies that support physician group success under accountable care models.

As physician-led ACOs are increasingly forming and have significant market potential, it will be important to create programs and policies that facilitate their success. Much work has been done in identifying the skills necessary to successfully manage a patient population,

 such as modernizing technology and accelerating real-world evidence to learn from successful organizations,

 but more is needed, including learning how to finance necessary investments for smaller groups. Policy makers should encourage the development of tools to evaluate physician-led ACOs and continue to assess how vendors and ACO enablers or management partners can help these organizations succeed.

 CMS is focusing on reducing the length of time that providers can be in an upside-only ACO arrangement while giving additional time to “low-revenue” providers—a measure they believe is a proxy for physician-led ACOs and rural hospitals.

 But even with a delay before 2-sided risk is required, there is a continued need to quickly help organizations develop the competencies to succeed. This can be encouraged with additional investment in peer learning communities, sharing examples of successful organizations, and policies that favor the use of ACO enablers (organizations that work with many ACOs to help them be successful and share in the financial success).

Additionally, policies should encourage and facilitate physician-led ACOs to partner with other organizations. By definition, physician-only ACOs will not provide the whole spectrum of necessary medical care. Mechanisms need to be created to help physician groups work effectively with hospitals, postacute care providers, pharmacies and pharmacists, mental and behavioral healthcare providers, and others.

 Policy makers can identify barriers, such as antikickback and Stark laws, that inhibit partnerships across organizations but do not result in consolidation and resulting price increases—in late 2019, HHS did propose changes

 to these laws, which, if adopted, may encourage these partnerships.

Physician groups are becoming the dominant type of new entrant into the ACO space and have been the most successful in achieving savings to date. It is uncertain whether this trend will continue, given the Trump administration’s push to move ACOs to take on risk earlier, which may hinder the participation of smaller physician group ACOs. Policy makers, vendors, and developers of payment programs must recognize that physician-led ACOs have unique needs and opportunities to address care for patients, and they should create policies that focus on identifying the skill sets necessary for physician-led ACOs to manage the full spectrum of care while concurrently eliminating barriers to partner with other providers and break down care silos.

 Leavitt Partners (DM), Salt Lake City, UT; The Dartmouth Institute at Dartmouth College (DM, CHC), Lebanon, NH; Margolis Center for Health Policy at Duke University (DM), Durham, NC; University of Utah (TT), Salt Lake City, UT.

 Dr Muhlestein is employed by Leavitt Partners, which provides consulting services around accountable care organizations (ACOs) and sells ACO-related data. Ms Tu was a contracted employee at Leavitt Partners. Dr Colla reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (DM, CHC); acquisition of data (DM); analysis and interpretation of data (DM, TT, CHC); drafting of the manuscript (DM, TT); critical revision of the manuscript for important intellectual content (DM, TT, CHC); statistical analysis (DM); administrative, technical, or logistic support (DM); and supervision (DM).

 David Muhlestein, PhD, JD, Leavitt Partners, 299 S Main St, Ste 2300, Salt Lake City, UT 84111. Email: david.muhlestein@leavittpartners.com.

1. CMS, HHS. Medicare program; Medicare Shared Savings Program: accountable care organizations. 

2. Tu T, Muhlestein D, Kocot SL, White R. Origins and future of accountable care organizations. Brookings website. brookings.edu/wp-content/uploads/2016/06/Impact-of-Accountable-CareOrigins-052015.pdf. Published May 2015. Accessed April 8, 2020.

3. Wu FM, Shortell SM, Lewis VA, Colla CH, Fisher ES. Assessing differences between early and later adopters of accountable care organizations using taxonomic analysis. 

. 2016;51(6):2318-2329. doi: 10.1111/1475-6773.12473.

4. Colla CH, Lewis VA, Tierney E, Muhlestein DB. Hospitals participating in ACOs tend to be large and urban, allowing access to capital and data. 

. 2016;35(3):431-439. doi: 10.1377/hlthaff.2015.0919.

5. D’Aunno T, Broffman L, Sparer M, Kumar SR. Factors that distinguish high-performing accountable care organizations in the Medicare Shared Savings Program. 

. 2018;53(1):120-137. doi: 10.1111/1475-6773.12642.

6. Muhlestein D, Gardner P, Merrill T, Petersen M, Tu T. A taxonomy of accountable care organizations: different approaches to achieve the Triple Aim. Leavitt Partners website. leavittpartners.com/whitepaper/a-taxonomy-of-accountable-care-organizations-different-approaches-to-achieve-the-triple-aim. Published June 16, 2014. Accessed April 8, 2020.

7. Muhlestein D. Continued growth of public and private accountable care organizations. 

 website. healthaffairs.org/do/10.1377/hblog20130219.028313/full/. Published February 19, 2013. Accessed March 21, 2013.

8. Torch Insight/Leavitt Partners Insight website. www.leavittpartnersinsight.com. Accessed April 8, 2020.

9. Henry TA. Employed physicians now exceed those who own their practices. American Medical Association website. ama-assn.org/about/research/employed-physicians-now-exceed-those-who-own-their-practices. Published May 10, 2019. Accessed September 30, 2019.

10. MACRA. CMS website. cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/MACRA-MIPS-and-APMs/MACRA-MIPS-and-APMs.html. Updated November 18, 2019. Accessed April 8, 2020.

11. Bleser WK, Muhlestein D, Saunders RS, McClellan MB. Half a decade in, Medicare accountable care organizations are generating net savings: part 1. 

 website. healthaffairs.org/do/10.1377/hblog20180918.957502/full/. Published September 20, 2018. Accessed September 20, 2018.

12. Khullar D, Burke GC, Casalino LP. Can small physician practices survive?: sharing services as a path to viability. 

. 2018;319(13):1321-1322. doi: 10.1001/jama.2017.21704.

13. Lewis VA, Tierney KI, Colla CH, Shortell SM. The new frontier of strategic alliances in health care: new partnerships under accountable care organizations. 

. 2017;190:1-10. doi: 10.1016/j.socscimed.2017.04.054.

14. Frakt AB, Mayes R. Beyond capitation: how new payment experiments seek to find the “sweet spot” in amount of risk providers and payers bear. 

. 2012;31(9):1951-1958. doi: 10.1377/hlthaff.2012.0344.

Articles

Since 2015, the majority of new accountable care organizations (ACOs) have been led by physician groups rather than hospital systems. This shift requires policies that address the characteristic strengths and weaknesses of physician-led ACOs.

Accountable Care Organizations Are Increasingly Led by Physician Groups Rather Than Hospital Systems

 Accountable care organizations (ACOs) are groups of healthcare providers responsible for quality of care and spending for a defined patient population. The elimination of low-value medical services will improve quality and reduce costs and, therefore, ACOs should actively work to reduce the use of low-value services. We set out to identify ACO characteristics associated with implementation of strategies to reduce overuse.

 We used the National Survey of ACOs to determine the percentage of responding ACOs aware of the Choosing Wisely campaign and to what degree ACOs have taken steps to reduce the use of low-value services. We identified characteristics of ACOs associated with implementing low-value care—reducing strategies using 3 statistical models (stepwise and LASSO logistic regression and random forest).

 Responding executives of 155 of 267 ACOs (58%) were aware of Choosing Wisely. Eighty-four of those 155 ACO leaders said that their ACOs also actively implemented strategies to reduce the use of low-value services, largely through educating physicians and stimulating shared decision making. All 3 models identified the presence of at least 1 commercial payer contract and prior joint experience pursuing risk-based payment contracts as the most important predictors of an ACO actively implementing strategies to reduce low-value care.

 In the first year of implementation, just one-third of ACOs had taken steps to reduce the use of low-value medical services. Safety-net ACOs and those with little experience as a risk-bearing organization need more time and support from healthcare payers and the Choosing Wisely campaign to prioritize the reduction of overuse.

Collective risk taking in financial contracts is the most influential determinant for accountable care organizations (ACOs) in taking steps to reduce unnecessary care. Safety-net ACOs are not likely to take steps like educating physicians on low-value medical services and encouraging shared decision making. ACOs with less experience in risk bearing will likely start to prioritize overuse when they acquire more risk as an organization.

ACOs with little experience in risk bearing and safety-net ACOs should be specifically stimulated to reduce overuse with targeted advocacy efforts of healthcare payers and the Choosing Wisely campaign.

Research should focus on identifying efficient strategies for waste reduction with specific attention to audit-and-feedback mechanisms on overuse and underuse.

Accountable care organizations (ACOs) are voluntary groups of provider organizations that are collectively held accountable for both

quality of care and total spending for a defined group of patients through payment contracts. A promising strategy to improve quality and financial sustainability involves the reduction of low-value medical services. Indeed, prior research shows a modest decrease in the use of low-value care, and thus in spending rates, for ACOs compared with non-ACO providers with predominantly fee-for-service payment models.

 While utilization and related spending have decreased in ACOs, quality scores and care satisfaction have remained similar or improved compared with other organizations.

 However, it is not clear what strategies ACOs deploy to lower unnecessary care, nor what features predict a commitment toward overuse reduction.

One way to tackle low-value care is to embrace the Choosing Wisely campaign.

 Choosing Wisely aims to reduce the delivery of low-value medical services by promoting conversations between patients and physicians on the appropriateness of care. More than 70 US specialty societies have defined concise lists of 5 to 10 wasteful interventions that “physicians and patients should question.”

 The synergies of ACOs and Choosing Wisely regarding care improvement and overuse reduction suggest that ACOs committed to reducing low-value care should be aware of this campaign and also work toward actively lowering the utilization of these medical services. In this study, we analyze data from the National Survey of ACOs (NSACO) to determine which strategies are used to reduce low-value care and identify the ACO characteristics that predict the use of such methods.

The NSACO is an online survey designed by researchers at the Dartmouth Institute for Health Policy and the University of California, Berkeley. It questions ACOs (Medicare Shared Savings Program [MSSP] ACOs, Medicare Pioneer ACOs, state Medicaid ACOs, and commercial payer ACOs) on their composition, characteristics, contracts, and capabilities.

 A total of 752 ACOs were identified through public documents, provider surveys, scientific literature, and certification by the National Committee for Quality Assurance and invited to participate. Senior ACO executives, including chief executive officers, executive directors, and chief medical officers, filled out the survey. At the time of our analysis, the survey was fielded in 3 consecutive waves, with each wave questioning newly formed ACOs (wave 1, October 2012-May 2013; wave 2, September 2013-March 2014; wave 3, November 2014-May 2015). The median duration between the implementation of the ACO contract to the time of the survey was 11.6 months (interquartile range, 7.1-13.2 months).

 Over all 3 waves, 64% of ACOs filled out the survey. Waves 2 and 3 of the survey were more elaborate, and therefore, ACOs from wave 1 were approached with a follow-up survey asking additional questions during the fielding of wave 3. Questions about Choosing Wisely were not asked in wave 1, excluding from our analysis 93 ACOs that participated in the first wave but not the follow-up survey. Survey questions related to Choosing Wisely included: (1) “Are you aware of the Choosing Wisely program?” and, if the response was positive, (2) “What steps have you taken to reduce the use of Choosing Wisely tests and procedures?”

We divided our sample into 2 groups: (1) ACOs not aware of the Choosing Wisely campaign or aware but not taking steps to support it and (2) ACOs taking steps to actively reduce the use of low-value medical services.

We compiled an a priori list of 62 survey responses that could be associated with the decision to take steps to support Choosing Wisely (the 

] lists these characteristics). Based on existing hypotheses about how these characteristics might affect an ACO’s decision to take steps to reduce overuse, as well as on simple pairwise significance tests, we then selected a subset of 22 variables from this list. We excluded ACO characteristics on quality behavior to prevent potential reverse causality with waste-reducing efforts. To identify the main drivers behind the decision to take steps to reduce overuse within those 22 variables, we used both logistic regression (stepwise regression and LASSO regression) and classification techniques (random forest).

 Stepwise logistic regression was performed both backward and forward. LASSO imposes shrinkage constraints on the variables, resulting in an optimal model with only those characteristics that have a coefficient greater than 0. Random forest stratifies the predictor space in regions with nonlinear boundaries between variables, producing multiple decision trees that are combined into a single consensus prediction. The 3 statistical approaches identified 3 sets of prediction variables, and we subsequently assessed consistency of associations across these 3 models. Furthermore, we evaluated the relative predictive merits of each model by comparing their receiver operating characteristic (ROC)

curves and confusion matrices on the basis of their implied misclassification rates (fraction of false positives and false negatives).

CMS publicly reports shared savings payments and the outcomes of 33 quality measures per performance year for each participating ACO. The quality scores are in 4 domains: patient experience (including a measure on shared decision making), care coordination, at-risk measures, and preventive care. We calculated an overall quality score and a quality score per domain for the first 2 performance years of each ACO (the year in which the ACO filled out the survey) using the CMS sliding scale approach, as described elsewhere.

 We compared these quality scores and the savings per beneficiary attributed to the ACO according to CMS in the first 2 years with reference to historical expenditure benchmarks, between ACOs taking steps to reduce overuse and ACOs not taking steps, using a 2-sample 

Characteristics of ACOs Taking Steps to Reduce Overuse 

Of 305 potential ACOs (wave 1 follow-up, 82 ACOs; wave 2, 95 ACOs; wave 3, 128 ACOs), survey respondents for 267 ACOs answered the question “Are you aware of the Choosing Wisely program?” Of these, 58% (155) reported awareness of Choosing Wisely, but only 31.5% (84) said they had also taken steps to reduce the use of low-value services (the eAppendix Table characterizes these ACOs). Consequently, 183 ACOs (68.5%) did not take such steps, partly because they were not aware of the Choosing Wisely campaign.

Compared with ACOs not taking steps, ACOs that had implemented strategies to reduce waste included more hospitals in their largest contract (

<.01) and were significantly more likely to consider themselves an integrated delivery system (

<.01) (the eAppendix Table compares these 2 groups). Provider organizations within ACOs implementing those strategies had more often jointly pursued risk-based payment contracts in the past (

<.01), and this group of ACOs had previously participated in a higher number of payment reform efforts than ACOs that did not take steps (

<.01). More ACOs taking steps to reduce waste had at least 1 commercial contract (

<.01), and Medicare contracts were less prevalent (

= .03). Although their commercial contracts were more often characterized by both bonus and downside risk (

= .03), this bonus was less often contingent on quality metrics (

= .02). In addition, ACOs that actively reduced low-value services were more likely to allocate shared savings bonus payments across participating members (

= .04), compensate physicians based on clinical quality measures (

= .01), and share cost measures among their physicians (

= .02). A larger proportion of ACOs that were not aware of Choosing Wisely or not actively reducing waste were safety-net organizations (

<.01), defined as having more than 25% uninsured or Medicaid beneficiaries.

Of the 84 ACOs that reported using strategies to reduce low-value care, just 57 (68%) were asked to specify the steps they had taken, because this question was added to the survey after wave 2. On average, these ACOs took 3 different steps. The most frequently reported waste-lowering strategies included educating physicians on low-value tests and procedures (82.5% of ACOs), encouraging discussions between physicians and patients about appropriate care (73.7%), and disseminating Choosing Wisely materials (68.4%) (

). Other strategies included handing out decision guides for patients, pop-ups in the electronic health record reminding physicians of the low value of certain tests and procedures, and audit and feedback on individual physician performance. Few ACOs (3.5%) reported taking steps to reduce waste by changing physician payment incentives.

Predictors for Implementing Strategies to Reduce Low-Value Care

Backward and forward stepwise logistic regression identified the same 6 variables and similar importance for predicting the use of low-value care—reducing strategies (

). The presence of at least 1 commercial contract and prior participation of all provider organizations within the ACO in risk-based payment contracts were the most important factors. These were also the 2 most significant predictors in the LASSO model (

). This model gave only 1 negative predictor, which predicted that an ACO would not actively seek waste reduction. The characteristic, more than 25% uninsured or Medicaid patients, was fourth in order of importance, and also 1 of the 6 characteristics in the stepwise regression (Table 2). In the consensus prediction tree from the random forest, the third model we used, the same 2 variables (at least 1 commercial contract, prior joint experience with risk-based payment contracts) were the 2 highest branches in the decision tree (

). Similarly, having more than 25% uninsured or Medicaid patients was the most influential negative predictor in this model. In conclusion, our working model to predict which ACOs would deploy strategies to reduce low-value care consisted of 2 positive predictors indicating joint experience in risk taking in the form of financial models, as well as 1 negative predictor (namely, a large contingent of safety-net patients).

The 3 models exhibited similar minimum misclassification rates on their ROC curves. The lowest possible misclassification rate for both stepwise regression and LASSO was 24% (

 shows the true- and false-positive rates for the minimum misclassification of each model), whereas it was 26% for random forest (data not shown). This attributes to these models a power to identify 33% to 39% of true positives, or ACOs that take steps to reduce low-value care.

Relationship Between Taking Steps and CMS Data on Quality and Savings

Waste-reducing strategies were not associated with differences in CMS quality measures or savings per beneficiary for the 158 ACOs for which CMS data were available in performance year 1 (46 taking steps, 112 not taking steps) (

). Furthermore, changes in these parameters from performance year 1 to year 2 were similar (32 and 70 ACOs, respectively; data not shown).

Based on 3 waves of the NSACO, we show that just one-third of all ACOs are taking steps to reduce low-value care. The best predictors for an ACO to deploy strategies for waste reduction were consistent across the 3 models with various statistical approaches. ACOs with a commercial contract and those whose provider organizations have jointly pursued risk-based payment contracts in the past are more likely to actively reduce overuse. Our finding is consistent with those of a recent study wherein ACOs with commercial payer contracts were more actively implementing efficiency measures compared with ACOs with public payer contracts only.

We found that ACOs with a large contingent of uninsured or Medicaid patients probably do not take steps to reduce overuse. However, the delivery of low-value care is as common among uninsured or Medicaid patients as among the privately insured.

 In prior research, minorities and those with poor and fair health were notably at a higher risk of receiving wasteful medical services.

 Both groups are overrepresented in the Medicaid population.

 Therefore, safety-net ACOs should pay attention to overuse of medical services. However, with relatively fewer resources for quality improvement, these ACOs may be prioritizing tackling the underuse of high-value practices over limiting low-value practices.

Furthermore, we did not find correlations between taking steps to reduce overuse and CMS quality measures (including the use of shared decision making) or between such steps and overall savings for ACOs participating in the MSSP in the first 2 performance years. Indeed, although waste in healthcare is ubiquitous,

 quality measures and savings are determined by many factors other than waste reduction alone. In addition, the effect of strategies other than clinical decision support and performance feedback to lower low-value care may be limited, certainly if they do not address both patient and provider roles.

 For example, audit-and-feedback methods are efficient in reducing unnecessary antibiotic prescriptions, but only if the intervention is perpetuated.

 Decision aids—but not shared decision making or the dissemination of educational materials&mdash;have moderate effect on the use of discretionary surgery.

 Although not yet extensively studied, certain pay-for-performance models have resulted in only modest improvements in care processes and outcomes.

 In our study, just half of ACOs aware of Choosing Wisely took active steps to reduce low-value care, and in most of those ACOs, the interventions were low impact and did not include the most promising strategies for waste reduction (namely, decision aids, computer decision support, and audit and feedback on individual physician behavior). To be meaningful, increasing awareness of Choosing Wisely should go hand in hand with practical advice for provider organizations on how to enhance appropriate use of care efficiently. Therefore, more research is needed to determine the correct design of strategies to reduce low-value care, the potential to lower utilization of such care in ACOs, and the numeric effect of reductions on quality and savings in the long run.

Several limitations to studies involving NSACO data have been recognized,

 notably, the fact that survey questions were answered by ACO executives who may not have been aware of efforts to reduce low-value care in their organization and the short period (less than 1 year) between the start of the ACO contract and the survey. Within this time frame, ACOs may not yet have been able to initiate strategies to reduce low-value care. Of note, previous comparisons of ACOs filling out the NSACO survey with those that failed to respond have not shown a significant nonresponse bias in terms of beneficiary or provider composition, organizational structure, overall quality, or savings distribution.

Physicians are an important source of low-value care utilization. It explains the focus chosen by Choosing Wisely on conversations between physicians and patients on appropriate care. However, collective risk taking in financial contracts influences ACOs to actively reduce low-value care more than physician leadership does. It is also possible that other unobserved characteristics in linkage with risk-bearing experience are influencing an ACO’s decision to seek a reduction in low-value care. In addition, reverse causality (ACOs most confident in their capacity to reduce waste signing risk-bearing contracts) cannot be excluded. However, this would not negatively affect the relationship we detected between collective risk taking and waste reduction.

The positive influence of risk bearing on efforts to reduce low-value care may be explained by a combination of resources, incentives, and opportunity. First, ACOs contracting with commercial payers may have more stringent contracting requirements that force them to prioritize waste reduction and more resources dedicated to quality improvement than do ACOs involved exclusively in public payer contracts. Second, ACOs with prior experience in risk-based contracting may have been able to develop the culture, systems, and technical know-how to tackle challenging issues such as overuse over time. Consequently, ACOs with less experience in risk bearing and safety-net ACOs will likely start prioritizing the reduction of overuse as they acquire more risk. In the meantime, those ACOs should be otherwise stimulated to reduce overuse and specifically targeted by advocacy efforts of healthcare payers and the Choosing Wisely campaign. Furthermore, researchers, policy makers, and the Choosing Wisely campaign should focus on defining waste-reducing efforts that are efficient and practical in use, with special attention to audit-and-feedback mechanisms on individual physician performance for both underuse and overuse. With US healthcare spending as high as 18% of gross domestic product, and healthcare outcomes in the United States lagging behind in comparisons with other high-income countries,

 all ACOs should use available levers for waste reduction, including Choosing Wisely materials, and implement strategies with proven efficacy in reducing low-value care to decrease costs and increase the quality of healthcare.

Dr Haverkamp and Dr Peiris, both Harkness fellows 2015-2016, gratefully acknowledge their sponsor, the Commonwealth Fund (New York), which has also supported the work with grant 20160616. In addition, Dr Haverkamp recognizes the Dutch Ministry of Health Welfare and Sport for sponsoring her fellowship. Furthermore, this project is supported by Agency for Healthcare Research and Quality (AHRQ) grants R01HS023812 and U19HS024075. The latter grant falls under AHRQ’s Comparative Health System Performance Initiative, which studies how healthcare delivery systems promote evidence-based practices and patient-centered outcomes research in delivering care.

 Department of Health Policy and Management, Harvard T.H. Chan School of Public Health (MHH, DP, MBR), Boston, MA; The George Institute for Global Health, University of New South Wales (DP), Sydney, Australia; The Dartmouth Institute for Health Policy & Clinical Practice (AJM, CHC), Lebanon, NH; Scientific Center for Quality of Healthcare (IQ healthcare), Radboud University Medical Center, Radboud Institute for Health Sciences (GPW), Nijmegen, Netherlands; Division of General Medicine, Brigham and Women’s Hospital (TS), Boston, MA; Department of Health Care Policy, Harvard School of Medicine (TS), Boston, MA.

 This study was funded by the Commonwealth Fund (New York) through grant 20160616, and personal grants to Dr Haverkamp and Dr Peiris (both Harkness fellows 2015-2016). In addition, the Harkness fellowship of Dr Haverkamp was sponsored by the Dutch Ministry of Health Welfare and Sport. Furthermore, AHRQ supported the work with grants R01HS023812 and U19HS024075.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (MHH, DP, GPW, MBR, TS, CHC); acquisition of data (MHH, CHC); analysis and interpretation of data (MHH, DP, AJM, MBR, TS, CHC); drafting of the manuscript (MHH, AJM); critical revision of the manuscript for important intellectual content (MHH, DP, AJM, GPW, MBR, TS, CHC); statistical analysis (MHH); provision of patients or study materials (CHC); obtaining funding (MHH, CHC); administrative, technical, or logistic support (AJM, CHC); and supervision (GPW, TS, CHC).

 Margje H. Haverkamp, MD, PhD, Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Kresge Bldg, Rm 431, 677 Huntington Ave, Boston, MA 02115. Email: mhhaverk@hsph.harvard.edu.

1. Schwartz AL, Chernew ME, Landon BE, McWilliams JM. Changes in low-value services in year 1 of the Medicare Pioneer Accountable Care Organization program. 

. 2015;175(11):1815-1825. doi: 10.1001/jamainternmed.2015.4525.

2. Nyweide DJ, Lee W, Cuerdon TT, et al. Association of Pioneer Accountable Care Organizations vs traditional Medicare fee for service with spending, utilization, and patient experience. 

. 2015;313(21):2152-2161. doi: 10.1001/jama.2015.4930.

3. McWilliams JM, Chernew ME, Landon BE, Schwartz AL. Performance differences in year 1 of Pioneer Accountable Care Organizations. 

. 2015;372(20):1927-1936. doi: 10.1056/NEJMsa1414929.

4. Kelleher KJ, Cooper J, Deans K, et al. Cost saving and quality of care in a pediatric accountable care organization. 

. 2015;135(3):e582-e589. doi: 10.1542/peds.2014-2725.

5. Choosing Wisely: promoting conversations between providers and patients. Choosing Wisely website. choosingwisely.org. Accessed November 11, 2016.

6. Colla CH, Lewis VA, Shortell SM, Fisher ES. First national survey of ACOs finds that physicians are playing strong leadership and ownership roles. 

. 2014;33(6):964-971. doi: 10.1377/hlthaff.2013.1463.

7. Peiris D, Phipps-Taylor MC, Stachowski CA, et al. ACOs holding commercial contracts are larger and more efficient than noncommercial ACOs. 

. 2016;35(10):1849-1856. doi: 10.1377/hlthaff.2016.0387.

The Elements of Statistical Learning: Data Mining, Inference, and Prediction

Experience with risk-based contracting best predicts active engagement of accountable care organizations in reducing low-value medical services, mainly through physician education and encouraging shared decision making.

ACOs With Risk-Bearing Experience Are Likely Taking Steps to Reduce Low-Value Medical Services

 Alternative payment models, such as accountable care organizations, hold provider groups accountable for an assigned patient population, but little is known about unassigned patients. We compared clinical and utilization profiles of patients attributable to a provider group with those of patients not attributable to any provider group.

 Cross-sectional study of 2012 Medicare fee-for-service beneficiaries 21 years and older.

 We applied the Medicare Shared Savings Program attribution approach to assign beneficiaries to 2 mutually exclusive categories: attributable or unattributable. We compared attributable and unattributable beneficiaries according to demographics, dual eligibility for Medicaid, nursing home residency, clinical comorbidities, annual service utilization, annual spending, and 1- and 2-year mortality. We estimated multivariate regression models describing correlates of attribution status.

 Most beneficiaries (88%) were attributable to a provider group. The remaining 12% were unattributable. Beneficiaries unattributable to any provider group were more likely to be younger, male, and from a minority group; to have disability as the basis for enrollment; and to live in high-poverty areas. Unattributable beneficiaries included 3 distinct subgroups: nonusers of care, decedents, and those with healthcare service use but no qualifying evaluation and management visits. Many unattributable Medicare beneficiaries had minimal use of healthcare services, with the exception of a small subgroup of beneficiaries who died within the attribution year.

 Attribution approaches that more fully capture unattributable patients with low service use and patients near the end of life should be considered to reward population health efforts and improve end-of-life care.

Given the reliance of alternative payment models on assignment of patients to provider groups, we compared clinical and utilization profiles of patients attributable to a provider group with those of patients unattributable to any provider group.

We estimated multivariate regression models describing correlates of attribution status.

Most beneficiaries (88%) were attributable to a provider group, whereas 12% remained unattributable.

Beneficiaries unattributable to any provider group included 3 distinct subgroups: nonusers of care, decedents, and those with healthcare service use but no qualifying evaluation and management visits.

Attribution approaches that more fully capture unattributable patients with low service use and patients near the end of life should be considered to reward population health efforts and to improve end-of-life care.

By holding networks of healthcare providers responsible for the total cost and quality of care for a designated population, the accountable care organization (ACO) model creates incentives to coordinate care across providers, reduce unnecessary spending, and improve the quality of care.

 and by January 2017, Medicare held ACO contracts with 525 organizations serving more than 10 million beneficiaries.

 Medicare ACO contract participants that meet quality benchmarks are eligible to share the savings that they generate.

 An ACO’s performance is evaluated based on Medicare methods of measurement and attribution. Researchers have evaluated outcomes of ACOs among attributable beneficiaries,

 but no study has described unattributable beneficiaries.

In Medicare ACO contracts, beneficiaries are attributed to organizations based on their use of primary care services from eligible providers,

 so attribution can influence an organization’s performance under an ACO contract. Among organizations serving patients with complex clinical and psychosocial needs, for example, quality metrics may be hard to achieve, giving organizations incentives to avoid accepting such patients. Conversely, ACO participants that are effectively managing population health may not achieve shared savings for patients who, appropriately, do not use

the primary care services that would make them

eligible for attribution. ACOs have little financial incentive to deliver preventive care that might decrease the chance that healthy patients are attributed to an ACO.

 As advanced payment models mature to include downside risk, and as models like Comprehensive Primary Care expand, it is important to understand which patients are left out so that policy makers can develop regulations that encourage participation and improved care in new payment models.

To date, we have no information on the composition of patients who are not attributable to any provider group under Medicare Shared Savings Program (MSSP) regulations. This paper examines beneficiary characteristics associated with attribution and compares hospitalization, mortality, and spending across attributable and unattributable beneficiaries. Results from our analyses can guide policy on whether additional actions are necessary to adequately give provider participants incentives to improve population health and to ensure that vulnerable beneficiaries—who may benefit the most from improved care coordination&mdash;are not excluded from new payment models.

Using Medicare claims and CMS attribution rules for the most widely adopted model, the MSSP,

 we categorized beneficiaries into 2 mutually exclusive groups: patients who were attributable to provider groups and patients who were not attributable to any provider group. We included only beneficiaries with full parts A and B coverage, limiting the sample to those for whom traditional fee-for-service (FFS) Medicare was the primary payer. We combined all beneficiaries attributable to provider organizations (ie, MSSP, Pioneer ACO, and non-ACO medical groups) into a single category because the purpose of our research was to investigate the characteristics of those falling through the cracks of the current attribution methodology. In 2012, Medicare ACO contracts included 32 Pioneer program participants and 114 MSSP participants (over the first performance period from April 2012 or July 2012 through December 2013) that were responsible for more than 2 million beneficiaries.

 Although Pioneer program participants, which were responsible for nearly 700,000 beneficiaries in 2012,

 faced downside risk for spending, virtually all MSSP participants (n = 110) were in Track 1 and eligible for upside savings only.

 We conducted cross-sectional analyses examining characteristics of beneficiaries according to their attribution status.

Beneficiary Attribution to Provider Organizations 

Following MSSP’s 2-step attribution process, a beneficiary who has at least 1 face-to-face outpatient evaluation and management (E&M) visit is assigned to the provider group that has the highest allowed charges by primary care clinicians (ie, general practice, family practice, internal medicine, and geriatric medicine practitioners) for those visits.

 Patients not seeing primary care clinicians are attributed based on visits to qualifying non—primary care clinicians (ie, physicians in other specialties, nurse practitioners, physician assistants, and clinical nurse specialists).

 Beneficiaries who received care only from nonqualifying clinical providers (eg, interventional cardiologists or certified registered nurses) or in nonstandard settings (eg, emergency departments [EDs]) or who had no visits at all were not able to be assigned to any provider group using current MSSP attribution rules; they formed our “unattributable” group.

For beneficiaries in parts A and B who were 21 years or older, data were drawn from a 40% random sample of 2012 Medicare FFS claims and 2012-2013 beneficiary summary files. Beneficiary summary files provided patient demographics, enrollment status, and date of death. Attribution status, diagnoses, utilization, and spending were obtained using claims. We chose 2012 because that was the most recent year of claims available without the suppression of claims indicating substance use disorder in research files

 and within the period of Medicare ACO implementation.

Long-term residence in nursing homes was determined from the 2012 Minimum Data Set (MDS). Patients’ zip codes were used to identify hospital referral regions (HRRs) of residence; HRRs represent regional healthcare markets in the United States defined by the Dartmouth Atlas. We further used beneficiaries’ zip codes to identify rural and urban areas

 and whether they lived in high-poverty Census tracts (≥20% of residents below the federal poverty line).

The main outcome of interest was whether a beneficiary was attributable to a provider group or not. We estimated correlations between attribution and beneficiary characteristics, including age, sex, race/ethnicity (ie, non-Hispanic white, non-Hispanic black, Hispanic, Asian or Pacific Islander, or other race), residence in a high-poverty Census tract or a rural area, dual eligibility for Medicaid, residence in long-term nursing home (≥100 days in a nursing home according to MDS), disability as a basis of original Medicare entitlement, and date of death.

International Classification of Diseases, Ninth Revision

This article compares clinical and utilization profiles of Medicare patients who are attributed to provider groups with those of patients unattributed to any provider group in accountable care organization models.

Carrie H. Colla, PhD

Articles

Accountable Care Organizations Are Increasingly Led by Physician Groups Rather Than Hospital Systems

ACOs With Risk-Bearing Experience Are Likely Taking Steps to Reduce Low-Value Medical Services

Forgotten Patients: ACO Attribution Omits Those With Low Service Use and the Dying

Overuse and Insurance Plan Type in a Privately Insured Population