The Ingredients of Success in a Medicare Accountable Care Organization

June 21, 2016

For 2 successive years, the Hackensack Alliance Accountable Care Organization achieved cost savings and maintained quality by using physicians with patient-centered medical homes and nurse care coordinators focused on high-risk patients.


Objectives: Accountable care organizations (ACOs) have rapidly become one of the preferred approaches to providing more efficient healthcare to the Medicare population. The ingredients of a successful ACO have not been well described in the literature, however. We present the descriptive epidemiology of our ACO to illustrate the elements in our organization that are critical to its success. We explain how we gained cost savings and show the quality measures that were monitored, which are essential to whether savings are received or not.

Study Design: Medicare provided expenditure data through claims files collected for the HackensackAlliance ACO. Quality data on 33 measures were collected from 3 sources: the Medicare claims files, a chart review of physician notes in patient charts, and a Medicare patient experience survey of a sample of ACO beneficiaries.

Methods: The ACO submitted 9 Claim and Claim Line Feeds (CCLFs) to Verisk Health for analysis. Premier Inc reviewed the analysis and discussed the results with the principals of the HackensackAlliance ACO.

Results: As a whole, the ACO had fewer hospital admissions, readmissions, and emergency department visits compared with a group of ACOs—these were the major factors that reduced our expenses, as we focused on maintaining a high level of quality. The number of patient office visits far exceeded those found in the comparable ACO group, resulting in better healthcare for all—especially for those with chronic disease, such as diabetes and heart failure. The complex process for calculating savings or losses is also carefully described. The HackensackAlliance ACO saved expenses in 2 successive years, which is an uncommon feat.

Conclusions: One reason our ACO succeeded in having savings in 2 successive years is the initial requirement that physician practices be certified as patient- centered medical homes by the National Committee on Quality Assurance. These doctors understood and accepted what would be required in the current evolution of medicine care. Also critical, was instituting nurse care coordination of high-risk patients from the beginning of the ACO contract. These 2 factors helped eliminate waste in the system and changed physician behavior, not physician practice.The accountable care organization (ACO) model has captured the imagination of more than 700 healthcare sites in the United States. However, there is little in the academic literature describing how the model works, and most of the reports in the medical literature are editorials on whether ACOs will succeed, whether the philosophy for their creation is reasonable, and whether progress has occurred in the 5 years since they were created.1,2 Within the ACO model, there are multiple choices based on whether the organization agrees to assume downside risk and whether the federal government or a private enterprise supports them. The federal government, under the auspices of CMS, created most of the ACOs as either Pioneer ACOs or Medicare Shared Savings Programs (MSSPs). CMS also created other payment models, including Bundled Payments for Care Improvement (BPCI), Value-Based Payment for Physicians, and Comprehensive Primary Care Initiative (CPCI).3,4 This article will focus on the descriptive epidemiology of the MSSP model, how it assesses the ability to save expenses, and how it evaluates compliance with the 33 quality measures.

Most MSSPs are in Track 1, which aims for upside savings when costs are reduced, and does not assume downside risk when there are losses. Accepting both upside gains and assuming downside risk is referred to as Track 2, Track 3, or Next Generation5,6—each of the latter 3 has other features that distinguish them from one another. We will focus on Track 1, as that is the model selected by most ACOs.

An important point to emphasize is that the current approach to ACOs and other alternative payment models is different from the managed care programs of the 1990s. At that time, saving money was the sole goal. Now, under the ACO approach for MSSPs, it is important to achieve a high level of quality in the care of the patient in order to be eligible for sharing any significant savings generated. The ultimate goal for the ACO is to achieve the “Triple Aim”: namely, better care for the individual, better health for the population, and lower growth of expenditures.7


Each quarter, we receive the claims files from Medicare—referred to as Claim and Claim Line Feeds (CCLFs). They detail claims Medicare receives from each physician, hospital, and other healthcare service providers, such as skilled nursing facilities. We send the CCLFs to a data analytics firm, Verisk Health, for further analysis, and their findings are forwarded to Premier—a healthcare performance improvement alliance of 3000 hospitals—to be digested and to cull out important clinical findings. These findings are then reviewed by our data analyst (author ME) and clinician (author PG) to select the key clinical findings for presentation to our physicians—mainly, our primary care providers (PCPs).

It should be noted that claims may not be available for all patients, as patients have the opportunity to opt out of sharing their information with their physician. This occurred in less than 3% of our patients, however.

The types of information sent to the PCPs include basic demographics and information on clinical areas that have a significant impact on expenses, such as hospital admissions, readmissions, and emergency department (ED) visits. For example, Table 1 shows that the largest individual practices—namely practices A, B, and C—are cost-efficient and have fewer admissions, readmissions, and ED visits than the ACO norm, as well as many more office visits than the ACO norm.

The clinical quality provided by the PCPs is based on the 33 measures required by Medicare (Table 2). These measures cover 4 major areas: patient/caregiver experience; care coordination/patient safety; preventive health; and at-risk populations for diabetes, vascular disease, hypertension, heart failure, and coronary artery disease. A full score is achieved when the measure compliance achieves the 90th percentile level.

The savings achieved by the ACO are determined by an elaborate risk-based formula (Table 3).8 The following presentation describes our experience in the second year (January-December 2014) of the CMS contract. In the first year, our total savings was $10,747,669 and our savings share came to $5,266,358. It should be noted that the “first year” was 21 months long (April 2012-December 2013) and the quality was measured on reporting, not on compliance. In both contract years, current expenses were compared with a 3-year historical expense benchmark. To eliminate the possibility of random variation being responsible for a savings or a loss, Medicare sets a minimum savings rate (MSR) that has to be exceeded on the upside to be considered a true savings, and a minimum loss rate (MLR) that has to be exceeded on the downside to be considered a true loss.9 The size of the minimum rates is dependent on the number of Medicare beneficiaries at the individual ACO—the larger the ACO, the smaller the minimum rate.


We will review the major steps leading up to identifying whether the ACO achieves cost savings or not in the second year (2014).

Beneficiary Assignment

If a beneficiary has at least 1 primary care service from a physician in the ACO, the beneficiary will be assigned to the ACO after they complete a 2-step process.5 First, the beneficiary receives the plurality of their primary care services from a primary care physician in the ACO. PCP is defined as a physician specializing in internal medicine, general practice, family practice, or geriatric medicine. Second, if step 1 did not occur, a beneficiary could still be assigned to the ACO if the beneficiary receives the plurality of their primary care services from other non—primary care professionals within the ACO, including physician specialists, nurse practitioners, clinical nurse specialist, and physician assistants.

Overview of Shared Savings Calculations

Let’s consider the method for determining shared savings (Table 3).8 First, we have to identify the number of beneficiaries assigned by CMS to our ACO and, more accurately, the person-years (#2 in Table 3). Next, the per-capita historical benchmark expenditures (#3 in Table 3) and the actual per-capita expenditures (#4 in Table 3) are identified. The total benchmark expenditures and actual expenditures are calculated as shown in Table 3. The total savings is the difference between the two. The minimum savings rate percentage is selected from a table (not shown) that illustrates how the rate varies inversely with the population size. In our case, the minimum savings rate in dollars is less than the total savings10; therefore, our ACO has the opportunity to share in the savings with Medicare. Potential net savings for the ACO are 50% of the total savings, and Medicare keeps the other 50%. How the ACO performs on the 33 quality measures from CMS is critical to receiving all or part of the savings.11 As our final quality score is 89%, we are able to keep 89% of 50%, or 44.7%, of the potential net savings. Then, from that number, a portion is taken out for the sequester adjustment (#15 in Table 3) when the federal government was shut down in 2014. The final net earned savings performance payment becomes $2,832,988.

Population Segmentation

There are some finer points that are considered in the calculation. Knowing the distribution of the population is also relevant. There are 4 enrollment groups that CMS reports: 1) patients with end-stage renal disease (ESRD), 2) patients who are disabled, 3) aged and dual- eligible patients who have both Medicare and Medicaid coverage, and 4) aged but non—dual-eligible patients who have only Medicare coverage. The patients with ESRD are the most expensive, but make up only 0.6% of our total patient population; consequently, focusing on the expenses generated by patients with ESRD, while important, is not an initial priority. The aged/non–dual eligible patients are the largest percentage of our population (86.5%). Even though they are the least expensive, they receive our immediate attention because this group incurs the largest overall expense. Fee-for-service expenses are then analyzed for each of the 4 enrollment types, annualized, truncated, and a payment completion factor is added in. Indirect Medical Education (IME) payments and Disproportionate Share Hospital (DSH) payments, as well as uncompensated care payments, are not included.

Historical Benchmark

Next, using a complicated process, we identify the historical benchmark for per-capita expenditures in the previous 3 calendar years. The first benchmark year is 2009, and it is weighted 10%; the second is 2010 with a 30% weight, and the third is 2011 with a 60% weight. In addition, the federal Office of the Actuary (OACT) updates the benchmark to account for changes in beneficiary characteristics and the absolute amount of growth in per-capita expenditures. The historical benchmark is updated for subsequent years.

Risk Scoring

Beneficiary risk scoring uses the CMS-Hierarchical Condition Categories (HCCs) retrospective risk-adjustment model to adjust the benchmark years when updating the historical benchmark in subsequent years. Each year, the historical benchmark is updated for changes in health status and demographic factors that occur in the originally assigned beneficiaries. Consequently, the risk scores could go up or down depending on the diagnostic codes gathered by CMS claims. In the Verisk Health application, the DxCG risk scores for Models 125— a risk score that indicates how much the individual is predicted to cost (includes age, gender, eligible months, diagnostic codes, selected utilization, and prior cost) by multiplying the risk score by the plan average cost per person per year—were developed in concert with the HCCs.

Distribution of Net Savings Within the ACO

There are many different approaches to dividing up the shared savings. In our case, this amount is divided into 2 parts. One part goes to Hackensack University Medical Center to repay part of their initial investment in the ACO—the balance goes to the primary care practices. The practices then divide up the amount among their physicians. The actual amount each practice or tax identification number (TIN) receives is based on the number of beneficiaries cared for by the TIN.


The story on the ability of MSSP to reduce costs is mixed. In performance year 1, which was 21 months long, 53 of 220 ACOs (24.1%) earned shared savings.11 In performance year 2, 97 of 353 ACOs (27.5%) earned shared savings.12 Behind the numbers is the fact that approximately half of the ACOs saved money in each of the performance years, but only the numbers shown exceeded the minimum savings rate to allow those few MSSPs to earn savings. For the 2 performance years, only 38 of 353 ACOs (10.8%) generated savings (according to a memo from Holly Wittenberg, Avalere Health, February 2016). The key factors that we believe most likely made a difference and permitted us to generate shared savings are as follows.

1. Selection of Membership

In the initial design for our ACO, the selection of the membership was critical. We decided to select practices that were already certified as patient-centered medical homes (PCMHs). A PCMH is defined as “a way of organizing primary care that emphasizes care coordination and communication to transform primary care into ‘what patients want it to be.’ Medical homes can lead to higher quality and lower costs, and can improve patients’ and providers’ experience of care.”13,14 The National Committee for Quality Assurance grants the certification. For those physicians who want to join the ACO but are not certified as PCMHs, we insist that they acquire the software to practice with an electric health record and become certified as a PCMH within 1 year. We agree to provide, free-of-charge, a trainer to facilitate their acquisition of PCMH certification.

2. Providing a Nurse Care Coordinator

The other major commitment we made initially was to provide a nurse care coordinator for the larger practices. Their goal is to identify the high-risk patients, get to know them well, and assure those patients relate to them whenever they felt ill to avoid the knee-jerk response of going to the ED immediately.

To minimize waste and avoid unnecessary hospitalizations, the combination of physician and nurse coordinator working in concert enabled the identification of areas of wasteful spending, creation of work plans, and provision of timely appointments to see the PCP team.

3. Informing About CMS Quality Measures

The physicians and nurses were informed of the 33 quality measures required by CMS. It was made clear that compliance with these measures determines whether savings generated by more efficient care will accrue to the ACO.

4. Identifying a Data Analytics Firm

The next part was not so easy. We knew that information would be the key to success for this project. Consequently, we had to identify a data analytics firm to help determine which practices were and were not doing well. This information came from CMS in several reports—namely, the CCLFs and Financial Reconciliation Reports.15 We identified 3 companies—Health Endeavors, Verisk Health, and Milliman—that are helping us with various parts of our data analysis. The data from Verisk Health and Milliman go through a second filter at Premier Inc, which helps us identify key areas of required attention and areas where more investigation is needed.

5. Using the Data

Using these data analytics firms, we found from the Verisk Health analysis that our risk score for our population was higher (24.34) than many ACOs (17.17). This factor, in turn, meant that CMS determined that our historical expenses should be relatively high.

6. Examining Regional Differences

Although there is some evidence that ACOs with higher initial benchmark expenditures are more likely to be able to reduce expenditures, 16 this factor did not guarantee a reduction in expenditures for all high-cost ACOs. Regional differences need to be examined more closely to fully understand this phenomenon.

There are many areas where CMS and the ACOs themselves need to improve their programs, but time does not permit serious consideration of all these areas. For CMS, some of the areas, for example, include revision of the retrospective attribution of beneficiaries, reducing the complexity of determining cost savings, reexamining the “normal variation” in savings and losses, considering regional effects on cost, the role of merit-based incentive payment system, and competition from other alternative payment models such as bundled payments and the comprehensive primary care initiative. ACOs, themselves, need to deal with leakage of beneficiaries to other ACOs, performing annual wellness visits on all assigned beneficiaries, interconnectivity of numerous EHRs, choosing other areas to reduce waste such as skilled nursing facilities and other post-acute care options, and selecting appropriate data analytic software.


The MSSP’s ACOs appear to offer ample opportunity to improve the quality of care for the patient while at the same time, reduce the cost of care. Initial factors that should facilitate these changes are having physicians certified as patient-centered medical homes by NCQA and providing financial support for increased care coordination. In this setting, high-risk patients are likely to be seen more often in the office in order to prevent deterioration and to better manage their chronic conditions. PCMHs and care coordinators will provide the greatest opportunity to impact a large segment of medical care costs by decreasing hospital admissions, readmissions, and ED visits, while maintaining a higher quality of healthcare.

We should emphasize that we do not change physician practice, we change physician behavior. By creating the appropriate interventions, we eliminate waste in our bloated healthcare system. We learn to address patient needs better. Although we are still good at disease management, we learn how to perform health management better from the PCMH model.


The authors appreciate the editorial assistance provided by Regina G. Gross, MA.

Author Affiliations: Hackensack Physician-Hospital Alliance Accountable Care Organization, LLC (dba HackensackAlliance ACO) (PAG, ME, MM, EG, IS, RCG, RLG), Hackensack, NJ; Hackensack University Medical Center (PAG, ME, MM, EG, IS, RCG, RLG), Hackensack, NJ; Rutgers New Jersey Medical School (PAG, IH), Newark, NJ; Premier Inc (EP, VC), Charlotte, NC; Verisk Health Inc (JH), Salt Lake City, UT.

Source of Funding: The Hackensack University Medical Center supplied the startup and continuation funds.

Author Disclosures: Dr Sawczuk is an employee of Hackensack University Medical Center CMO. MS Hart is an employee of Verisk Health. Mr Garrett is a member of the Hospital and Network Board of Hackensack and is the CEO of Hackensack University Health Network. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (PAG, EG, MM, VC, IS, RCG, RLG); acquisition of data (PAG, EG, ME, EP, VC); analysis and interpretation of data (PAG, EG, ME, MM, EP, VC, JH); drafting of the manuscript (PAG, MM, JH); critical revision of the manuscript for important intellectual content (PAG, ME, MM, JH); statistical analysis (PAG, ME, EP, VC); provision of study materials or patients (PAG, EG); obtaining funding (PAG, IS, RCG, RLG); administrative, technical, or logistic support (PAG, EG, ME, EP, VC, IS, RCG, RLG); and supervision (PAG, EG, MM, IS, RCG, RLG).

Send Correspondence to: Peter A. Gross, MD, Chair, ACO Board of Managers, 30 Prospect Ave, Hackensack, NJ 07601. E-mail:


1. Blumenthal D, Abrams M, Nuzum R. The Affordable Care Act at 5 years. N Engl J Med. 2015;372(25):2451-2458. doi: 10.1056/ NEJMhpr1503614.

2. Casalino LP. Accountable care organizations—the risk of failure and the risk of success. N Engl J Med. 2014;371(18):1750-1751. doi: 10.1056/NEJMe1410660.

3. Howell BL, Conway PH, Rajkumar R. Guiding principles for Center for Medicare & Medicaid Innovation model evaluations. JAMA. 2015;313(23):2317-2318. doi: 10.1001/jama.2015.2902.

4. Ryan AM, Press MJ. Value-based payment for physicians in Medicare: small step or giant leap? Ann Intern Med. 2014;160(8):565-566. doi: 10.7326/M13-1715.

5. Finalized changes to the Medicare Shared Savings Program regulations. CMS website. MediaReleaseDatabase/Fact-sheets/2015-Fact-sheetsitems/ 2015-06-04.html. Published June 4, 2016. Accessed January 3, 2015.

6. Kocot SL, White R, McClellan M. The revised Medicare ACO program: more options…and more work ahead. Health Affairs website. aco-program-more-options-and-more-work-ahead/. Published June 16, 2015. Accessed January 3, 2015.

7. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood). 2008;27(3):759-769. doi: 10.1377/hlthaff.27.3.759.

8. Medicare Shared Savings Program: shared savings and losses and assignment methodology specification [version 3]. CMS website. Payment/sharedsavingsprogram/Downloads/Shared-Savings- Losses-Assignment-Spec-v2.pdf. Published December 2014. Accessed May 2016.

9. Pope GC, Kautter J. Minimum savings requirements in shared savings provider payment. Health Econ. 2012;21(11):1336-1347. doi: 10.1002/hec.1793.

10. Table: 33 quality measures. CMS website. https://www.cms. gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/ Downloads/ACO-Shared-Savings-Program-Quality- Measures.pdf. Accessed May 30, 2016.

11. ACO performance results demonstrate savings to Medicare, minimal savings to individual ACOs. Avalere website. http:// results-demonstrate-savings-to-medicare-minimal-savings- to. Published September 18, 2014. Accessed May 2016.

12. Evans M. Few Medicare ACOs earned bonuses in 2014. Modern Healthcare website. 20150825/NEWS/150829922. Published August 25, 2015. Accessed May 2016.

13. Patient-centered medical home recognition. National Committee for Quality Assurance website. Recognition/Practices/PatientCenteredMedicalHomePCMH. aspx#sthash.AHJiRhZq.dpuf. Accessed May 2016.

14. Nielsen M, Buelt L, Patel K, Nichols LM. The patient-centered medical home’s impact on cost and quality: annual review of evidence 2014-2015. Patient-Centered Primary Care Collaborative website. The%20Patient-Centered%20Medical%20Home’s%20Impact%20 on%20Cost%20and%20Quality,%20Annual%20Review%20 of%20Evidence,%202014-2015.pdf. Published February 2016. Accessed May 2016.

15. Webex: Medicare Shared Savings Program—Open Mic Q&A Session. Published March 31, 2014. Accessed May 30, 2016.

16. Blazej L. Surprising facts about successful ACOs: a dive into the data. LaunchMed website. blog/surprising-facts-about-successful-acos-a-dive-intothe- data.html. Published May 1, 2015. Accessed May 2016.