 The main objectives of this study were to show a simple approach to detect inappropriate requests of laboratory tests and to monitor success after establishing interventions. These objectives were monitored through process and outcome indicators customized according to the type and phase of the appropriateness strategy.

 Based on evidence regarding laboratory test utilization differences among different geographical areas of Spain, we identified serum calcium (s-Ca) testing to be underrequested and total bilirubin (tBil) testing to be overrequested in primary care patients who undergo testing at the Public University Hospital of San Juan, in San Juan de Alicante, Alicante, Spain. Additionally, the ratio of free thyroxine (FT4) tests to thyrotropin (also called thyroid-stimulating hormone [TSH]) tests was well above the published 0.25 goal in primary care. Finally, numerous laboratory tests were overrequested in hospitalized patients due to repetitive testing. We designed and implemented a variety of strategies to correct such inappropriateness and designed different indicators to monitor the intervention success over time.

 After implementation of the different strategies, the absolute number of s-Ca tests increased. The number of tBil tests in primary care, and numerous other tests repeated too frequently in hospitalized patients, decreased. The FT4/TSH indicator goal was reached and maintained over time. Regarding the outcome indicators, the strategy of reducing tBil tests in primary care and reducing the aggregate of unnecessary tests in hospitalized patients resulted in savings of $3543.80 and $9825.50, respectively, from January 2012 to December 2014. The s-Ca strategy, from November 2011 to December 2014, detected 62 subjects’ primary hyperparathyroidism at a cost of $137.80 per case.

 The study demonstrates a simple approach to detect inappropriate requests of laboratory tests, and how to assess the potential success of interventions using process and outcome indicators.

Addressing inappropriate laboratory orders is a great challenge for laboratory professionals. 

The aim of this study was to show a simple approach to detecting inappropriate laboratory orders. 

Indicators customized according to the type and to the phase of the appropriateness strategy are essential to being able to measure the potential impact on the patient’s diagnosis.

Addressing inappropriate requests of laboratory tests is a great challenge for healthcare professionals; yet, due to the significant adverse effects of such requests, it is worth tackling.

 Although underrequesting may result in missing a diagnosis, overrequesting may generate as many as 3 major adverse effects: 1) economic costs—although the individual cost of a single test may seem low, the cumulative effect of unnecessary tests is high, as they generate high costs due to their high request

; 2) adverse effects of false-positive results—these can produce additional side effects, including the costs of additional medical consultations and diagnostic tests, and the collateral effects of Ulysses

; and 3) increasing commoditization of the laboratory—the laboratory may be seen more as a “data vending machine” than as a provider of knowledge,

 and it might deliver results that are possibly misinterpreted or create a scenario in which those results with high clinical value are “hidden.”

 Given these issues, it is crucial to reduce variability of laboratory testing in clinical practice.

 Indeed, establishing interventions in collaboration with all the stakeholders involved in healthcare is a key element for better efficiency in clinical decision making.

The first objective of this study is to propose an intervention that can detect inappropriate requests of laboratory tests. The second is to monitor the course of the intervention over time through process and outcome indicators, customized according to the type and phase of the appropriateness strategy.

Through the National Health System, every citizen of Spain has access to public health services. The system divides Spain into health departments, each of which covers the healthcare necessities of a population living in a particular geographic area through several primary care centers and a hospital. The laboratory located at each hospital attends to the needs of every inhabitant of that health department.

The study was conducted from January 1, 2010, to December 31, 2014, in the clinical laboratory at the Public University Hospital of San Juan, a suburb of Alicante, Spain. This 370-bed suburban community hospital serves a population of 234,551 who utilize 9 different primary care centers (PCCs). It receives samples from hospitalized patients, outpatients, and primary care patients (PCPs).

Hospitalized patients’ samples are collected in every ward by nurses and then transported to the laboratory. The hospitalized patient and outpatient orders are entered manually in the laboratory information system (LIS); however, PCPs’ requests are ordered electronically by the general practitioners (GPs). Blood is drawn for PCPs in the PCC. Samples are transported by couriers following scheduled routes covering the PCCs and are delivered to the laboratory sample reception desk. Subsequent reports are sent electronically from the LIS to the PCP’s electronic health record.

In 2010, based on test utilization differences among Spanish geographical areas,

 overrequests of free thyroxine (FT4) tests were noted in primary care via the indicator that measures the ratio between the tests requested of FT4 and of thyrotropin (also known as thyroid-stimulating hormone [TSH]). The ratio of FT4/TSH equaled 0.42, which did not reach the published <0.25 indicator goal.

 We also detected a significant underrequest of serum calcium (s-Ca) tests and an overrequest of total bilirubin (tBil) tests in PCPs, through the indicators, “test requests per 1000 inhabitants” (37 and 56 tests per 1000 inhabitants for s-Ca and tBil, respectively).

For hospitalized patients, redundant requests of brain natriuretic peptide (BNP); ferritin; folate; glycated hemoglobin (A1C); high-density lipoprotein cholesterol (HDL-C); immunoglobulin G, A, and M (IgG, IgA, IgM, respectively); iron; and prostatic specific antigen (PSA); as well as rheumatoid factor, transferrin, triglycerides, and vitamin B12, were identified. We considered a test redundant when it was requested before an established interval; for instance, requesting a rheumatoid factor test when one had been performed just 2 days earlier was considered redundant.

To address the problem, we designed and established 4 strategies in consensus with GPs and the test-requesting hospital physicians. The first strategy, implemented in PCCs, is that the LIS discards FT4 when the TSH value is in reference range, unless the FT4 measurement is specifically requested by the GP. The second strategy, also implemented in PCCs, is that the LIS automatically adds s-Ca to the GP requests made for patients 45 years or older who have not had the test in the previous 3 years.

GPs agreed with these 2 interventions during various meetings; patients provided verbal consent for conducting a biochemical examination at the moment of the request by the GP. However, if, for some particular reason, a clinician justifies the relevance of an automatically cancelled FT4 test for a specific patient, this test is registered again.

The third strategy consists of measuring tBil only when the icteric index is above 2 mg/dL (34.2 µmol/l).

 The icteric index is a very accurate semi-quantitative surrogate of tBil. When the index is below 2 mg/dL (34.2 µmol/l), tBil is reported through a comment, such as “with a confidence interval of 99%, tBil result is below 1.2 mg/dL (20.5 µmol/l).” This strategy was applied to every type of patient.

The fourth strategy involved hospitalized patients. If previously requested and completed in the past 7 days, the LIS would automatically negate requests for tests of total cholesterol, HDL-C, or A1C. If previously requested and completed in the past 3 days, the LIS would automatically negate requests for tests of BNP, ferritin, folate, IgG, IgA, IgM, iron, PSA, rheumatoid factor, transferrin, triglycerides, or vitamin B12.

) illustrates the different steps of the proposed approach to identify and correct potential inappropriate use of laboratory tests. The strategies to reach appropriate request levels of FT4, s-Ca, and tBil in primary care, and the intervention for the aforementioned tests in hospitalized patients, were established in May 2011, November 2011, January 2012, and March 2012, respectively. From October 2012 to January 2013, the s-Ca strategy stopped for preliminary evaluation, then the strategy was reinstituted in January 2013. The rest of the measures were continuous.

This study was approved by the Hospital Research Committee of the Public University Hospital of San Juan.

Monitoring Through Process and Outcome Indicators

The strategy that discards FT4 when the TSH value is in the reference range was monitored through the ratio of requests of both tests. The strategy to increase the measurement of s-Ca in PCPs was monitored through the process indicator: the absolute number of s-Ca tests that were added. This strategy was also monitored through the indicator that relates s-Ca to serum glucose (s-Glu) requests. To control that the predicted increase in s-Ca requests was indeed due to the intervention and not a general trend, we also calculated the ratio of serum creatinine to s-Glu requests. The intervention to request tBil testing through a comment, when the icteric index is below 2 mg/dL (34.2Ã¢â‚¬â€¦µmol/l), was monitored through the process indicator of number of patients whose tBil was not measured.

The strategy that eliminated the aforementioned tests (total cholesterol, BNP, etc) in hospitalized patients when requested too soon (as described) after previous identical measurements was evaluated monthly, utilizing the counts of tests that were cancelled.

We calculated the following outcome indicators: number of new cases of primary hyperparathyroidism (HPT) detected; the economic cost per new HPT diagnosis derived from an s-Ca test added from the clinical laboratory (calculated by dividing the total amount of s-Ca reagent costs by the number of new HPT cases); and economic savings that resulted from the decrease in tBil measurement and from cancelling the hospitalized patients’ tests that were deemed redundant because identical ones had recently been performed. To calculate these savings, the tBil and redundant tests that would have been performed were counted, and the total was multiplied by the price of the reagent that would have been used (other “saved” laboratory fees were not included). eAppendix Table B summarizes the different indicators that were used to track and monitor the different interventions after their establishment.

 shows the number of requests and tests received annually on behalf of PCPs and hospitalized patients from January 1, 2010, to December 31, 2014.

 shows the evolution of the FT4/TSH indicator value for primary care on a monthly basis. The indicator goal was reached in May 2011 and has been maintained ever since.

There was a significant increase in the number of s-Ca tests from primary care that were added at the beginning of the implementation of the strategy, then 3 months with no extra s-Ca because of the intervention evaluation period, and finally a gradual decrease in the number of s-Ca added as the number of patients screened for PHPT increased (Figure 2). The indicator that relates s-Ca to s-Glu request followed the same pattern (

). There was also an increase in the total requests of tBil, (24,658 in 2012, 25,437 in 2013, and 30,979 in 2014) and tBil not measured. The number of tBil tests that were not measured, but reported through a comment, is shown in 

 shows the total number of tests that were deemed unnecessary and were cancelled in hospitalized patients.

The outcome indicators demonstrate that, thanks to the s-Ca strategy, 62 new, previously unidentified cases of HPT were detected between November 2011 and December 2014, with each detection costing $137.80.

From an economic perspective, regarding the outcome indicators, the strategy of reducing tBil tests in primary care and reducing the unnecessary tests in hospitalized patients resulted in savings of $3543.80 and $9825.50, respectively, from January 2012 to December 2014.

Our study demonstrates how simple approaches can detect inappropriate requests of laboratory tests, as well as monitor—at a glance&mdash;postintervention success, using process and outcome indicators that have been customized according to the type and stage of the strategy. The measurement of outcome indicators is essential to calculate if the clinical laboratory is contributing appropriately in terms of patient results and economic savings (ie, achieving the highest contribution to the diagnosis, monitoring, and prevention of disease at the lowest cost).

The first step in any intervention of this kind is to determine whether there is over- or underrequesting of laboratory tests; the second is to correct such inappropriateness through strategies approved by requesting clinicians; and the third is to monitor the corrective interventions after their establishment.

Studies to identify inadequacies in test requests by reviewing patients’ medical records are expensive, cumbersome, and difficult to perform. However, identifying such inadequacies is possible—using indirect means&mdash;by measuring differences in test utilization among geographical areas (ie, multiple countries or regions within a single country).

 Indicators that can be used in such scenarios include the number of tests per 1000 residents in a certain area

 or by comparing the number of requests with guidelines or disease prevalence.

 In our case, we detected that the relative ratio of FT4 and TSH testing was not in accordance with a published, generally accepted (in Spain) target of <0.25.

 Additionally, the underrequesting of s-Ca tests was detected in our region by utilizing the indicator “requests per 1000 inhabitants,” which was significantly lower than in other Spanish areas.

 Testing for tBil was considered to be inappropriate since it is not recommended as a screening method for liver disease in primary care.

A second way to detect whether over- or underrequesting is taking place is through retrospective studies of the number of requests in the LIS patient database. In fact, we utilized this approach to identify inappropriate test requests for hospitalized patients. Unnecessarily repetitive testing was observed for certain tests—probably due to human error, as hospitalized patients’ medical records were kept manually, not with computers.

Once inappropriate testing levels have been identified and addressed, indicators play a crucial role in evaluating the success of interventions and to check if they are maintained over time. Regarding process indicators, for instance, as was expected, once the s-Ca strategy was established, the requests for this test began to increase, both in absolute numbers and relative to the s-Glu test requests. When relative indicators are used, it is important to also measure, simultaneously, a warning indicator referring to a test whose demand is not influenced by the strategy to identify potential external confounding factors not related to the established intervention. The success of the strategies to diminish overrequesting of FT4 and tBil tests, and to eliminate redundant tests in hospitalized patients, also needs to be assessed through the use of indicators. Of particular interest is that indicators that measure request ratios of 2 related tests, such as FT4 and TSH, have 2 different applications: identification of inappropriate request and monitoring after intervention.

A very significant issue in implementing any major change is the ability to sustain the intervention. Most—such as educational or administrative strategies&mdash;may produce excellent results during the first months of application that, unfortunately, are not maintained over time.

 However, laboratory professionals have at their disposal excellent information systems that can be utilized to improve requesting in a sustained manner,

 and the study findings did show that the results of interventions were maintained over time in every strategy.

Just as process indicators are, outcome indicators must be designed before establishing the strategy. Although the outcome indicators are as straightforward as measuring the decrease of unnecessary treatments,

 they are crucial to discovering if the laboratory has become more efficient and/or is enhancing its contribution to patient management. Indeed, the importance of monitoring after intervention establishment is always important, but it is especially crucial when it comes to correcting the underrequesting of certain tests. In a case like ours, in which we were investigating increased requesting of s-Ca tests and, consequently, increased economic expenses, it was crucial to be able to monitor the strategy’s success in such a systematic and detailed manner so that we could decide in a very short period of time whether to stop or continue the intervention. In fact, in the case of s-Ca, a preliminary pilot study—with very carefully chosen outcome indicators&mdash;was designed, established, and evaluated before maintaining the intervention over time. The preliminary pilot study strategy ended after 4 months, at which time we reviewed patient medical records to obtain the preliminary results of outcome indicators.

 In view of the number of HPT cases detected, and the low cost of every case identified, we decided to restart the strategy and maintain it indefinitely. Moreover, the strategy regulates itself. As time passes, there will be fewer patients who haven’t had their s-Ca tested in the previous 3 years, and consequently fewer s-Ca tests will be added by means of the strategy.

The study results indicate that the strategies do not just enhance laboratory contribution to diagnosis. In fact, through s-Ca strategy, the clinical laboratory becomes the protagonist in diagnosis. It is LIS that, in a continuous way over time, is identifying new cases of HPT. Additionally, the economic savings accrued by correcting laboratory test overrequests will improve the effective use of laboratory resources.

This study has certain limitations. First, the study indicator results, referred to as tests per 1000 inhabitants (residents in the investigators’ area), could be considered as research with local importance. These settings might not be appropriate for large testing elsewhere because they were designed for a local population/health service system in Spain. Additionally, in certain environments, a written test request is required for that test to be performed; our s-Ca strategy would not be possible under this requirement.

Another potential limitation is that the strategy we designed for hospitalized patients was to avoid test redundancy due to human errors in manual requesting because the hospitalized patients’ medical records are not computerized. It was 

 based on previous studies regarding minimal retesting intervals, defined as the minimum time before a test should be repeated, based on the properties of the test and the clinical situation in which it is used.

 Finally, the calculated economic savings of the study may not apply to other countries or settings, since our laboratory belongs to the Public Health Network, where reagent prices are relatively low.

The study demonstrates a simple approach to detect inappropriate requests of laboratory tests and to monitor results after appropriate interventions, using process indicators. Indicators that are customized according to the type and to the phase of the strategy are essential to measure the potential impact on patient results. Outcome indicators measure the laboratory’s contribution to the diagnosis; to monitoring or preventing diseases; to becoming a protagonist in diagnosis; and to leading the healthcare system towards the most effective uses of its resources.

The authors would like to express their deep gratitude to José Manuel Ramos Rincón, MD, PhD, for his valuable and constructive suggestions during the review of this paper.

 Clinical Laboratory, Hospital Universitario de San Juan, San Juan de Alicante (MS, ML-G, EF, JL), Alicante, Spain; Department of Biochemistry and Molecular Pathology, Universidad Miguel Hernández (MS, ML-G, ML-S), Elche, Alicante, Spain; Department of Dermatology, Hospital General Universitario de Alicante (ML-S), Alicante, Spain; Primary Care Center of Muchamiel, Alicante-San Juan Health District (AA), Muchamiel, Alicante, Spain; Department of Radiology, University of Virginia (CL-S), Charlottesville.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (AA, EF, ML-G, JL, CL-S, ML-S, MS); acquisition of data (EF, ML-G, MS); analysis and interpretation of data (EF, ML-G); drafting of the manuscript (AA, EF, ML-G, JL, CL-S, ML-S, MS); critical revision of the manuscript for important intellectual content (AA, EF, ML-G, JL, CL-S, ML-S, MS); statistical analysis (EF, ML-G); and supervision (MS).

Articles

WEB EXCLUSIVE

This study shows automatic, practical, simple, and effective strategies designed in the laboratory, in consensus with requesting clinicians, to improve laboratory test appropriateness.

Managing Inappropriate Requests of Laboratory Tests: From Detection to Monitoring

 We analyzed changes in healthcare spending associated with the implementation of Cigna’s Collaborative Accountable Care (CAC) initiative in a large multi-clinic physician practice.

 We compared claims from 2009, prior to the CAC initiative, against claims for 2010 to 2011, contrasting the patients covered by Cigna’s CAC initiative with patients in other practices in the same geographic area covered by Cigna’s medical plan.

 We used a propensity weighted difference-in-differences approach, adjusting for age, sex, health status, and secular trends to isolate the treatment effect of the CAC.

 The CAC initiative resulted in a 5.7% reduction in net spending per patient for 2010 to 2011, relative to what spending would have been without the initiative. This reduced spending was evident in multiple service categories: evaluation and management, procedures, imaging, tests, and durable medical equipment. Professional payments, inpatient facility, and outpatient facility payments for Medical Clinic of North Texas enrollees all experienced significant cost savings relative to the control group. About half of the savings resulted from using lower-priced sources.

 The CAC initiative, which includes an embedded care coordinator and a list of recommended providers, was associated with cost savings similar to those reported by other initiatives, such as global budgets and risk-based contracts.

	Despite the proliferation of private accountable care organizations, few studies have examined whether they yield potential cost savings relative to fee-for-service medicine. We tested whether a shared savings contract between a large insurer and a multi-clinic physician practice saved money over a 2-year period. 

		Cigna’s Collaborative Accountable Care initiative achieved a 5.7% reduction in net spending per patient for 2010 to 2011, relative to what spending would have been without the initiative. 

		Significant cost savings, relative to the control group, were achieved with inpatient facility, outpatient facility, and other medical service payments.

		About half of the savings resulted from using lower-priced sources of care.

Accountable care organizations (ACOs) are provider-led organizations with a strong base of primary care providers who are collectively accountable for the quality and per-capita costs across the full continuum of healthcare for a population of patients.

 As part of the Affordable Care Act, many provider organizations have formed ACOs that are reimbursed for the care provided to Medicare patients. Under Medicare’s program, ACOs share in any savings and, in some cases, excesses, in the cost of care provided, relative to historic benchmarks.

A recent report estimates that there are 782 ACOs across the country, with 54% operated by private payers.

 Private ACOs differ greatly in form and size. They can be led by a hospital or hospital system, or include physician practices only.

 Provider reimbursement could take the form of shared savings in a fee-for-service (FFS) environment or as limited or substantial capitation arrangements. Private insurers also differ in the performance measures used to track quality.

 The criteria used for provider selection also differ across insurers, as does the level of technical assistance provided to participating providers (eg, disease management or health information exchange support).

Several policy experts and clinicians have written about the potential advantages of ACOs for improving the quality of healthcare and controlling healthcare cost growth,

 and many other papers have focused on advice for structuring or regulating ACOs.

 There has been extensive literature—we are unable to fully acknowledge it here&mdash;examining the cost and/or quality implications of integrated care, which is one important feature of ACOs.

 Meanwhile, there are fewer studies documenting the actual performance of ACOs, which include incentive payments for providers on cost or quality metrics.

A 2012 publication reported the early results on patient costs and outcomes for Cigna’s Collaborative Accountable Care (CAC) initiative for physician practices in 3 parts of the country: Arizona, New Hampshire, and Texas.

 This initiative is a shared savings program that offers practices, in their first year of participation, an upfront care coordination fee to pay for investments in infrastructure that furthers progress toward quality and cost targets. Cigna provided substantial support in informatics, care coordination, and consultation to aid participating practices. After the first year of intervention, Cigna analyzed medical claims and found no significant decrease in medical costs compared with expected costs for 2010. Other than this paper, we are aware of only 5 other studies of the cost effects of private ACO programs that have included a control group in their performance assessment.

 All of these studies examined the cost and/or quality impact of the Blue Cross Blue Shield (BCBS) Alternative Quality Contract in Massachusetts; the combined results suggest that the program lowered spending and improved quality overall, although effects differed for specific patient populations and conditions.

In this study, we report more detailed results after 2 years of CAC implementation for the physician practice in Texas (Medical Clinic of North Texas, P.A.). Using longer-term data, we compare the cost trends for physicians participating in Cigna’s initiative, relative to physicians in the surrounding market. We also compare the trends in costs by type of service and site of care in order to assess whether these trend differences were consistent with the financial incentives introduced by Cigna. It is beyond the scope of this current analysis to examine changes in the quality of care after CAC implementation; however, the physician practice received a payout from the initiative, implying that the quality criteria in the contract were met and quality did not deteriorate as a result of the initiative.

The CAC arrangement with the Medical Clinic of North Texas, P.A. (MCNT), was similar to the CAC initiatives that were described in a previous study.

 To summarize, Cigna paid MCNT a care coordination fee in addition to standard FFS payments. In the first year of implementation, the amount of the fee was based on an estimate of the projected benefits to Cigna’s patients, and in subsequent years, the payments were supplemented based on a formula that employed cost and quality measurements from the prior year. Cigna also supplied MCNT with reports on their patients to monitor progress and to identify those most likely to benefit from special assistance.

In return, MCNT took steps to provide Cigna patients with a level of care consistent with a patient-centered medical home. MCNT hired a nurse who served as an embedded care coordinator for Cigna patients, with primary activities that included the following: hospital discharge coordination for patients at increased risk of readmission, outreach to patients identified through Cigna’s predictive modeling programs as likely to incur high medical costs, and patient education and coaching regarding gaps in care, such as lack of preventive care or medication adherence. The arrangement also included active use of Cigna’s preferred provider list for referrals. A referral template in MCNT’s electronic health record (EHR) identified preferred specialists based on their in-network and value-based designations. MCNT providers and staff were also educated on the importance of in-network usage and how to use the template in the EHR.

MCNT had 141 primary care physicians (PCPs) in 42 practices, treating 7109 patients covered by Cigna in the Dallas—Fort Worth area in 2010. We compared the expenditures for these patients to Cigna patients who were aligned with other PCPs in the same geographic region between 2009 and 2011. This control group contained 192,655 patients in 2010. Study and control group patients were included only if they were aligned to PCPs and had 12 months of eligibility in any of the 3 years; there were no age restrictions. The methodology for aligning customers to PCPs has been described in a previous study.

 All payments to both groups were FFS. The data sources were the Cigna claim line and enrollment databases, contracts between Cigna and MCNT, and Cigna fee-payment reports.

The study design closely follows that used by previous researchers to analyze the effects of the Massachusetts BCBS Alternative Quality Contract.

 We compared changes in total spending pre-intervention versus post intervention for the CAC intervention patients versus the control group of patients in other Texas clinics. In this difference-in-differences analysis, the pre-intervention period is 2009 and the postintervention period is 2010 to 2011.

We decomposed the overall 2-year effect on spending into year 1 and year 2 effects. We also decomposed the spending effect by clinical category, as defined by the Berenson-Eggers Type of Service (BETOS) classification system,

 and by site and type of care. We describe our methodology for assigning claims to BETOS categories in eAppendix 1 [

]. We then decomposed the spending effect result into a price effect and a utilization effect by repricing claims for each service to their median prices across all providers in 2011. Spending results generated using standardized claims reflect only differences in utilization.

The dependent variable was aggregate medical spending per enrollee (in 2011 dollars) per 6-month period. Total medical costs included allowable charges typically covered under medical plan benefits (eg, inpatient facility, outpatient facility, professional, ancillary expenses); pharmacy expenses were not included. The care coordination fees that were in the claims were included in the charges for professional services. Medical spending included enrollees’ cost sharing. Aggregate medical spending was truncated at $100,000 for each patient in a given time period.

We controlled for age categories, interactions between age and sex, the patient’s retrospective risk score, and secular trends. We also categorized patients based on the cost-sharing characteristics of their insurance plan benefits. The cost-sharing category of each plan was based on the mean percent paid out of pocket for all individuals in the sample covered by the same set of benefits. Sets of benefits with fewer than 100 customers were assigned to the “undefined” category due to insufficient sample size.

Retrospective risk scores were calculated from the medical claims using commercially available software (Symmetry Episode Risk Groups, release 7.5 [Optum, Eden Prairie, Minnesota]). Claims for the current year were aggregated for each individual, creating a yes/no decision on 167 medical condition categories (eg, acute bronchitis, malignant neoplasm of the breast), called Episode Risk Groups (ERGs). Each ERG has a relative risk value, and the sum of these values is the risk score. The relative risks were derived from claims costs and utilization in a managed care population with more than 8 million members.

We analyzed spending (in 2011 dollars) at the semi-annual—enrollee level using a multivariate linear model with propensity weights calculated using age, sex, risk, and cost sharing (eAppendix 2). Additional independent variables included indicators for intervention status (MCNT vs non-MCNT), 6-month period, the postintervention period, and the interaction between the postintervention period and the intervention. This final indicator produced our initial estimate of the policy effect.

However, not all of the care coordination fees were embedded in the claims. Cigna was in the early stages of developing the CAC program, which led to delays in some payments. Therefore, we adjusted the regression estimate of the overall 2-year effect of the program for the additional care coordination fees that Cigna paid to MCNT that were not part of the claims data. We lacked sufficient information to conduct a similar adjustment of the regression estimates of the program effects by BETOS categories or site and type of care.

Standard errors were robust, based on clustering at the practice level.

 We tested the sensitivity of our results to restricting the analysis to patients who were continuously covered by Cigna for all 3 years of the study. We also tested omitting the adjustment for cost sharing and omitting the propensity weights.

 provides descriptive statistics comparing the treatment and control groups. Prior to the implementation of the CAC initiative, the MCNT clinic treated 8266 Cigna patients in 2009, and 7171 patients after the initiative was introduced. The experiences of these patients were compared with that of 180,278 Cigna patients treated by other providers in north Texas in 2009, and 185,561 patients in 2010 to 2011. Patients treated by the MCNT clinic were slightly older than patients covered by Cigna in other parts of north Texas, which translated into slightly higher risk scores. These differences are accounted for in the regression analyses. The truncation of medical spending in a given 6-month period, at $100,000, was necessary for only 5 observations in the sample. All of these observations were in non-MCNT clinics, and spending exceeded $1 million in each case.

 indicates that overall, the average 6-month expenditures declined between 2009, and 2010 to 2011, for patients covered by Cigna in the Dallas—Fort Worth area in both MCNT and non-MCNT clinics. The difference-in-differences analysis tests whether the decline in spending for MCNT under the CAC initiative was greater than the decline for non-MCNT clinics, after controlling for other factors that might explain observed spending patterns during the study period.

After implementation of the CAC initiative, average healthcare spending fell substantially for patients associated with an MCNT clinic, but declined only slightly for enrollees in the control group. Overall, from 2010 to 2011, statistical estimates indicated that the intervention was associated with a $128.30 (

 <.001) decrease in average spending per enrollee per 6-month period, relative to what spending would have been without the intervention (

 and Table 2). This decrease was a 6.5% reduction compared with what the spending would have been without the intervention. Further analysis indicates that most of the cost savings achieved by MCNT occurred during 2011. Spending per 6-month period did not change significantly between 2009 and 2010, but fell by $236.13 from 2009 to 2011.

Because only a portion of the care coordination fees were included in the 2010 and 2011 claims, the estimates do not include all of the costs associated with the CAC initiative. After subtracting all the fees paid in 2010 and 2011, the average overall net savings per 6-month period in 2010 to 2011 was $106.25 (5.7%).

This reduced spending was evident in multiple BETOS service categories—procedures, durable medical equipment, and “other” spending&mdash;although there were noticeable differences in the percentage reductions in spending across these BETOS categories. Average spending on procedures fell 10.2% between 2009 and 2011, while spending for durable medical equipment fell 13.4% and spending on “other” services fell 22.5%. The “other” services BETOS category is heterogeneous and includes ambulance services, as well as chemotherapy and other drugs. In most categories, the spending declines were larger in magnitude in year 2 of the intervention compared with year 1. Moreover, the decline in spending was statistically significant for all 7 BETOS categories in 2011 versus 2009.

Inpatient and outpatient facility payments and other medical service payments for MCNT enrollees all experienced significant cost savings relative to the control group. The average 6-month cost savings were larger in magnitude in 2011 than in 2010. The percentage declines in inpatient and outpatient facility spending were similar in magnitude (8.4% and 7.9%, respectively), while other medical services fell 16.7 percentage points. There was no significant decline in professional payments for patients covered by MCNT.

When we estimated cost savings using normalized prices, the estimated average 6-month savings over the period 2010 to 2011 fell to 3.7%. Therefore, some of the cost savings from the CAC initiative resulted from changes in service utilization. Of the remaining savings, approximately half resulted from using sources that had lower average costs per service.

The sensitivity analyses yielded cost savings estimates that were consistent with the main result. Limiting the sample to patients who were continuously covered by Cigna from 2009 through 2011 yielded an estimated average 2-year cost savings of 5.1% (—$103.76; 

<.001). Omitting the adjustment for cost sharing implied an estimated cost savings of 6.8% (—$128.12; 

 <.001). Omitting the propensity weights implied an estimated cost savings of 8.1% (—$142.80; 

The 2-year estimate of 5.7% average net cost savings from the CAC initiative may seem large when compared with initiatives such as BCBS Massachusetts Alternative Quality Contract, which reportedly saved 2.8% over a similar time span; however, most of the patients with 2 years of utilization under the BCBS intervention were from practice groups that had prior experience managing risk-based contracts.

 For patients treated by physician practice groups with no prior experience with risk-based contracts, the 2-year BCBS estimated savings was 8.2%, and, similar to the CAC initiative, the cost savings amount was substantially larger in the second year compared with the first year.

We would have liked to continue the analysis of MCNT and the CAC initiative beyond 2011; however, MCNT joined a larger integrated care organization, USMD Health System, in 2012, so data from later years may not be comparable. We shared our study results with MCNT’s managers and asked them to comment on their experience with the CAC initiative. The managers stated that MCNT devoted additional unquantified resources, including analytics and managers’ time, which were not fully covered by the care coordination fee paid by Cigna. MCNT managers stated that they were willing to do this because they recognized a change from volume to value among payers. Therefore, they viewed these additional costs as an investment in population health, which would be essential for future business.

Cigna bore additional costs to develop predictive models to assist MCNT and to develop the collaborative incentive relationship. Since then, Cigna has since been able to apply the tools developed for the MCNT relationship to over 140 incentive arrangements. Therefore, the marginal administrative cost of providing these tools to MCNT was likely minimal. It is possible that some other change in MCNT practices that occurred simultaneously with the CAC initiative could explain the cost savings identified in this study; however, MCNT management could not think of any other major changes in the practice that could explain the study results. Some offices expanded their hours during this time period, but the MCNT physicians could not see a link between this change and the measured savings.

Among BETOS categories, the 2-year average decline in spending for evaluation and management was statistically insignificant between 2009 and 2011, even though this category accounts for the most dollars spent among the 7 service categories. When spending was categorized by site and type of care, there was also no statistically significant average decline in professional payments from 2009 to 2011. Large and statistically significant reductions were observed for inpatient and outpatient facility spending, as well as other medical service payments. There was a similar finding when a risk-based contract was implemented among several hundred physicians by a health maintenance organization in the 1990’s.

 A conclusion in that study was that physician practices were less likely to decrease spending on services resulting in direct reimbursement to their own offices, and more likely to cut spending on services delivered to their patients by other healthcare providers. Results such as these suggest that physician-owned ACOs, such as MCNT, may be more effective in reducing spending than hospital-led ACOs.

It is possible that the CAC initiative led MCNT physicians to provide services in their clinics that lowered the need for more expensive services from outside providers. This might have been accomplished through preventive medicine or by performing procedures and tests in their offices that could have been referred. It is also possible that patients were referred to less expensive in-network or out-of-network providers or encouraged to use nurse practitioners instead of doctors or urgent care centers instead of emergency departments. Unfortunately, we do not have sufficient data to test these hypotheses.

A recent survey of private ACOs found that these organizations differ significantly in features, such as patient attribution and performance measurement and targets.

 Given the many features in which contracts between physician practices and insurers can differ, more extensive comparisons of the Cigna approach, relative to other ACO and medical home models, should be conducted. The upfront funding of coordinated care and local benchmarking may be highly effective tools in achieving cost savings for many other physician practices.

Although many studies describe the potential advantages of ACOs, only a handful of papers document the actual performance of these organizations. We found that after 2 years, a private ACO model with an embedded care coordinator and a list of recommended providers led to an estimated average of 5.7% net cost savings compared with what costs would have been without the initiative. Both upfront funding of coordinated care with local benchmarking or the responsiveness of a physician-owned ACO versus a hospital-led ACO may have contributed to these cost savings. Given the wide variety of organizational and contractual differences across private ACOs, future research should aim to distinguish between these 2 potential cost-saving mechanisms and others to determine which ACO models are most cost-effective.

Dr Ho acknowledges support from the Kuwait Foundation Fellow endowment at Rice University’s Baker Institute.

 Rice University’s Baker Institute for Public Policy (VH, MHK-G), Houston, TX; Department of Economics (VH), Rice University, Houston, TX; Department of Medicine, Baylor College of Medicine (VH), Houston, TX; Cigna (TA, UK, WPK, MS), Hartford, CT.

A private accountable care organization model with an embedded care coordinator and a list of recommended providers yields cost savings similar to initiatives with risk-based contracts.

Measuring the Cost Implications of the Collaborative Accountable Care Initiative  in Texas

 Complex patients incur a majority of healthcare costs in the United States, in part, because their care is poorly coordinated. We sought to determine the leading edge of health information technology (IT) tools for care coordination of complex patients.

 Qualitative discussions and technical expert panels.

 We conducted 35 discussions with clinical leaders, technology and startup executives, government officials, academic researchers, and 2 technical expert panels.

 Although health IT has the potential to improve care coordination, the types of IT tools available to clinicians and patients are currently limited. We found substantial barriers to developing technical capabilities for improving care coordination, including lack of knowledge of users’ needs; lack of standardized roles, responsibilities, and protocols; required changes in providers’ work activities to achieve coordination; and an unclear value proposition.

 We found several innovative tools, but existing efforts suffer from important limitations, including minimal engagement by physicians, lack of standardized definitions of what the tools do, and challenges integrating with clinical workflows. For health IT to facilitate coordination of care for complex patients, user needs and workflows must be better understood and used to guide the development of technology and policy.

	Coordination of complex patients’ care has great potential for producing savings. Although health information technology (IT) has the potential to help achieve this goal, the types of IT tools available to coordinate care for complex patients are currently limited. Specific study findings include: 

		Substantial barriers to developing technical capabilities exist and include a lack of knowledge of users’ needs and a lack of standardized roles and responsibilities. 

		Current tools engage physicians minimally and have challenges integrating with clinical workflows. 

		For health IT to facilitate coordination of care for complex patients, user needs and workflows must be better understood and used to guide the development of technology and policy.

Arguably, the greatest opportunity to save on healthcare spending in the United States is through more effective care of complex patients—defined as those with multiple comorbidities, high risk for poor outcomes, and high cost.

 Given that such patients typically visit multiple providers, improved care coordination is one important means of improving the effectiveness of their care.

 Despite this need, the care of complex patients is generally poorly coordinated.

 Few interventions have been shown to be effective in addressing the care coordination needs of these patients, and thus, reducing costs.

Information technology (IT) has been widely adopted outside of healthcare to support coordination among individuals. For example, project management technology coordinates workplace tasks, and configuration management programs help coordinate the work of software developers. However, recent studies have found a relative lack of such technical capabilities in healthcare, despite the existence of many of the components required for such capabilities (eg, cloud computing platforms, widespread use of smartphones, internet connectivity).

Although spending on health IT has increased, little effort has been devoted to developing care coordination technologies for complex patients. In addition, little is known about the latest generation of technologies.

 We sought to understand the key barriers to progress and the technical capabilities (ie, functionalities) at the leading edge of innovation. Building on our findings, we provide suggestions for how to better harness health IT to coordinate care for complex patients.

We conducted a series of discussions with experts from a variety of stakeholder perspectives, identifying them through a targeted literature review, recommendations from other experts, and snowball sampling. We selected 35 experts to represent a wide range of stakeholder perspectives from the private sector, academic institutions, and federal agencies. We conducted 2 phases of in-depth discussions, ranging from 45 to 90 minutes, using a guide that identified key topics to be covered, including IT capabilities for care coordination. After the first phase, we convened a technical expert panel to discuss the preliminary results and refine a second phase of expert discussions. The second phase focused on IT capabilities designed to facilitate care coordination through communication among providers caring for the same patient.

From the expert discussions, we identified key themes, including barriers to progress and types of technical capabilities that overcome these barriers. We then identified potential opportunities for further advancing care coordination IT tools to better meet the needs of complex patients and their care teams.

This study was approved by the RAND Institutional Review Board and all discussion subjects gave informed consent.

Experts identified 4 key barriers to the development of new IT tools for care coordination.

 The types of coordination needs, and activities that address them, are poorly understood, which makes it difficult for IT designers to create solutions. Even members of our technical expert panel disagreed about what activities should be called coordination. Some experts drew a sharp distinction between activities related to coordination versus treatment; in their view, physicians were not a necessary part of coordination. Others believed that such a distinction was unhelpful because many coordination activities are also related to treatment and involve physicians (eg, developing care plans, communicating with other physicians). From their viewpoint, it would be a mistake to not fully engage physicians as users of a functionality that facilitates care team communication.

 The users’ roles, responsibilities, and protocols are not standardized, except in special circumstances, like postsurgical care within an institution.

 The lack of standardized shared definitions for care team membership and associated workflows makes it difficult to operationally establish current care team roles. Even creating names for roles, such as “care coordinator,” is challenging, because they lack universal meaning and the roles change over the course of a patient’s care. Experts described challenges defining user roles even for common tasks: the management of a patient’s problem list—a list of the most important health problems facing a patient, including current and active diagnoses as well as relevant past diagnoses&mdash;does not have agreed-upon protocols defining which member of a care team is responsible for updates or corrections. Providers tend to be reluctant to modify problems added by other providers.

 Additionally, product design is complicated if users must be able to handle significant diversity, which can include variation among providers’ practice patterns and workflows, and patients’ needs and preferences.

 The use of IT tools for care coordination forces changes in the typical provider workflow. Joint use of a shared platform outside of a patient visit may require a work culture change for providers accustomed to visit-centric care. Including the patient in such communication is also new and unfamiliar.

 Incorporating this kind of work into routine clinical practice may require substantial training and the development of new work processes, such as establishing dedicated time for using the tools, and protocols for identifying and prioritizing patients’ coordination needs.

 Changes in incentives—such as accountable care&mdash;may make a better business case for providers to purchase and use care coordination IT capabilities, but the shift to newer payment arrangements has been slow. Even under such incentives, there is no evidence that use of IT tools for coordination will result in enough savings to justify purchasing and using the tools. Part of the challenge is the need to provide a compelling value proposition that takes into account the many types of users (eg, patients with varying conditions, patients’ families, physicians of all specialties, nurses, social workers, medical assistants) that vary both across and within organizations.

Technical Capabilities for Care Coordination

Despite the barriers described above, we identified 5 types of IT tools that can facilitate coordination: care plans, dashboards, patient relationship managers, event alerts, and referral tracking

). Some products combine and integrate these capabilities and focus on certain types of care coordination activities. We found no efforts that were comprehensive care coordination solutions. For some of these capabilities, basic forms are in more widespread use (eg, dashboards of quality indicators, text-based care plans). More advanced functionalities (eg, a goal-oriented care plan that supports communication among all members of a care team, alerts for a range of event types that can be “subscribed” to) are largely still under development or being tested in pilots. Care plans were the care coordination functionality described by experts as being in use most frequently, as well as being the most complex and varied. Other functionalities included dashboard, patient relationship managers, event alerts, and referral tracking. We describe care plans below and provide additional details on this functionality and others in the 

Experts described several care plan products developed by companies, including several startups, and those as part of research projects. Other experts described the desired functionality based on their experience as leaders in large provider organizations. Below, we summarize the experts’ input on the following aspects of care plans related to health IT: the definition of care plan and its varying content, types of users involved, definitions of user roles and responsibilities, and methods for tracking members of a care team. The eAppendix contains a summary of how care plans supported task tracking and communication among the care team.

 We found that the term “care plan” was poorly defined and meant different things to different experts. Experts differed in their understanding of the content and purpose of the care plans within a health IT system: a) static text that described brief action steps: annual ophthalmology visit and foot exam; reach a target weight; take BP every day; b) ability to track gaps in care, such as overdue screening tests, and issue reminders to patient, clinicians, or care coordinator; c) care plan structured around a list of problems, each with an associated goal and its assigned intervention; and d) “dynamic” care plans that are accessible between visits and can be updated (experts did not provide any more detailed specifications).

 Care plan functionalities mentioned by several experts were accessible to members of a care team and facilitated interactions among them. Users included clinicians, the patient, patient surrogates (eg, parents or children), care coordinators, and social services. Typically, the care plan functionality was designed primarily for use by care coordinators who would work with other supporting services, such as social workers, diabetic coaches, or fitness coaches. Although physicians usually had access to the functionality, they were rarely actively involved in using it, and when they did, the degree of use varied. For example, one startup company found that some physicians wanted to be fully engaged, overseeing the whole process surrounding a surgical episode of care. Others wanted to create the postsurgical care plan, hand it off to the rest of the care team, and not be involved further.

Definitions of user roles and responsibilities.

 Patients used the care plan functionalities to varying degrees. In some cases, patients were not actively using, or not offered access, to their care plan, partly because experts stated that older, sicker patients often were not interested in using the software to actively manage their care. In other cases, the patient was allowed control of many aspects of the care plan, such as the ability to selectively share specific types of data only with specific team members (eg, patients may not want to share psychological issues with fitness coaches). Some care plans allowed for structured interactions with patients, such as the use of patient assessment surveys that ask about patients’ available resources, goals, and symptoms. These assessments are used in some products to automatically produce or update tasks within the care plan (eg, triggering a care coordinator to give patients educational materials). Experts also described how the care plan functionality defined control and ownership. In one project that used a care plan functionality, only the care coordinator was allowed to edit the care plan; other team members could make suggestions, but could not make official changes. Other care plans are more flexible and allow multiple users to change the care plan.

Methods for tracking members of a care team.

 Implementing care plan functionality requires identifying and tracking the current members of the care team. One expert said that having clinical logic that would suggest or automatically add or remove providers to the team for a specific patient would facilitate the use of care plans. Several experts described products that allowed the patient to specify members of their care team. Other products were more provider-focused and required the provider or care coordinator to “invite” providers who might practice in the same organization or a different one. We found no widely agreed-upon definition of a care team member, and given that care team membership changes over time, this complicates the efforts of those developing these functionalities

Although the 5 IT capabilities above are clear signs of innovation, we found 3 important limitations. First, few care coordination products are designed for or used by physicians—this is particularly true for care plans. Although there are certainly care coordination activities that may not need to involve physicians, complex patients commonly receive care from multiple physicians,

 and some coordination-related issues that involve physicians are well-documented (eg, missing or unclear information at the point of care, unclear accountability for next steps, “dropping the baton” on referrals).

 Yet, few efforts described by discussants engaged physicians to address these issues.

Second, the types of activities that the IT tools support vary widely and lack standardized definitions, which reflects the barriers cited above; ultimately, user roles and responsibilities are poorly understood. For example, “care plans” could mean anything from a simple text box to a structured list of problems with associated goals and assigned interventions. Other ways in which care plans varied included the types of users for whom the capabilities were designed, the ability granted to patients to perform tasks, tracking members of a care team (patient invite vs provider invite), support for tracking tasks as part of the care plan, and lastly, the ability to communicate through messages embedded within the care plan. This variation and lack of standard definitions for describing the technical capability was a source of frustration for executives from provider organizations and made it challenging for them to understand the capabilities or compare products.

Third, care coordination products are often not easily integrated into existing physician workflows. Some experts believed that electronic health record (EHR) integration was required for physicians to agree to use the capabilities. Others said that care coordination capabilities could serve as a kind of bridge between different EHRs, because they would contain data from multiple systems. However, in order for such capabilities to bridge this EHR gap, care coordinators were often required to manually translate information (eg, physician orders) from the EHRs into separate care plan software. EHR integration challenges were less of a problem when the care plan capability was offered as part of an EHR vendor’s integrated solution, as certain data elements were populated automatically from the patients’ medical record, facilitating the generation of the care plan. Some experts expressed optimism in a “post-EHR” world in which innovative care coordination tools could be more easily integrated into the products used by providers.

Although the potential for IT to improve the coordination of complex processes has been demonstrated in many industries, the use of IT to improve coordination of care for complex patients is less clear. Substantial barriers impede development and the use of these tools in healthcare. Recent efforts are focusing on developing innovative solutions, but are nevertheless struggling with the barriers. Below, we discuss 3 key challenges, as well as policy opportunities to address them.

Challenge 1: The information and communication needs of users are diverse and poorly understood. 

Understanding user needs is a prerequisite for building effective technology. To improve care coordination of complex patients, the needs of every type of user (eg, physicians of every specialty, nurses, social workers, patients, family members) and the types of coordination activities in which they participate must be understood. This knowledge is needed to allow IT tool developers to accommodate different types and levels of engagement by various types of users as they perform a range of coordination activities. Amassing this knowledge is complicated by the wide variation in workflows across the healthcare system, as well as diversity of users’ preferences. For example, one recent study assessed preferences of elderly patients for information sharing with their caregivers and found substantial complexity and diversity.

 Furthermore, information needs will undoubtedly change; as with other technologies, the roles, responsibilities, and workflows of users will need to be redesigned to make full use of new capabilities.

Two activities could help address this problem. First, conducting an extensive assessment of user needs. This work will require characterizing the workflows and preference of each type of user, establishing standard definitions for user needs, and describing prototypical roles and responsibilities. Second, experimenting with the redesign of both workflows and IT capabilities to address a subset of critical user needs. It will be important to perform rapid cycles of testing, evaluation, and refinement of workflows and IT capabilities, so that care coordination interventions can be improved efficiently.

Challenge 2: Care coordination capabilities interrupt provider workflows.

 To fully engage physicians and other providers in care coordination, the burden of accessing care coordination IT capabilities must be minimized. If EHR vendors developed the capabilities to be well-integrated into existing products, workflow interruptions could easily be reduced for some patients. However, patients are likely to continue to receive care from providers in different organizations given existing patterns of fragmentation.

 Thus, there will be an ongoing need for care coordination IT capabilities to integrate with multiple EHR vendors to avoid workflow interruptions for patients who receive care from provider who use different EHRs.

Improvement in integration could be achieved through novel care coordination capabilities and/or improved interoperability of health IT more generally. Given that EHR vendors have many competing priorities, organizational capacity to focus specifically on developing novel care coordination functionalities may be limited. Therefore, we believe that the most promising path to innovation is through improved interoperability. Ongoing health IT interoperability efforts are attempting to develop standards that would make it easier for healthcare applications to integrate with each other.

 Such standards could facilitate integration of care coordination capabilities and ultimately reduce workflow interruptions.

The challenge to this approach, however, is to develop technical standards for capabilities that have not yet been specified. To better specify care coordination capabilities, particularly care plans, policymakers may consider convening experts to decide on key use cases (specifications of how and by whom the technical capabilities will be used) to guide standards efforts. Such efforts may be informed by the user needs assessments described above.

Challenge 3: Existing incentives are not sufficient.

 The lack of reimbursement for the care coordination supported by IT capabilities means that only risk-sharing or other emerging forms of accountable care provide incentives for adopting these technologies. However, there may be a “chicken and egg” problem: providers may resist taking on risk without proven tools and methods to manage the care of their highest-cost patients. Yet, without a market of providers to drive innovation, such tools may not be developed. Policy makers may overcome this problem by sponsoring research the helps to develop the tools as described above, demonstrate their efficacy, and identify best practices.

This work has several limitations. First, we limited our expert discussions to the United States because the international situation was beyond the scope of this project. Second, this was not a formal qualitative research study and we did not transcribe the discussions. We did conduct the discussions rigorously, requiring at least 2 project team members at each discussion and following up with e-mail for clarifications as needed. Finally, new innovations could have emerged since our discussions given the high level of interest in cost-effective coordination strategies related to payment reform.

Although health IT capabilities are being developed to facilitate coordination of care of complex patients, much is still unknown. Developing successful health IT capabilities should begin with a deep understanding of user needs and will require iterative development in which changes are made to both the technology and workflows. There may be a role for policy to address these critical gaps in knowledge and kick-start the development of better solutions.

 RAND Corporation (RR, ZP, CG), Boston, MA; The Commonwealth Fund (ES), New York, NY.

 This research was funded under contract #HHSP23320095649WC from the Assistant Secretary for Planning and Evaluation (ASPE), US Department of Health and Human Services (HHS). The findings and conclusions of this report are those of the authors and do not necessarily represent the views of ASPE or HHS.

 Concept and design (CG, RR, ES); acquisition of data (ZP, RR); analysis and interpretation of data (ZP, RR); drafting of the manuscript (CG, ZP, RR); critical revision of the manuscript for important intellectual content (CG, ZP, RR, ES); obtaining funding (CG, RR); administrative, technical, or logistic support (ZP); and supervision (RR, ES).

 Robert S. Rudin, PhD, RAND Corporation, 20 Park Plaza, Ste 920, Boston, MA 02116. E-mail: rrudin@rand.org.

1. Medicaid: a small share of enrollees consistently accounted for a large share of expenditures. Government Accountability Office website. http://www.gao.gov/assets/680/670112.pdf. Published May 2015. Accessed February 22, 2016.

2. Osborn R, Moulds D, Schneider EC, Doty MM, Squires D, Sarnak DO. Primary care physicians in ten countries report challenges caring for patients with complex health needs. 

. 2015;34(12):2104-2112. doi: 10.1377/hlthaff.2015.1018.

This study examines the leading edge of health information technology used to coordinate the care of complex patients.

September 2016

Managing Inappropriate Requests of Laboratory Tests: From Detection to Monitoring

Measuring the Cost Implications of the Collaborative Accountable Care Initiative in Texas

Knowledge Gaps Inhibit Health IT Development for Coordinating Complex Patients' Care

Mapping US Commercial Payers' Coverage Policies for Medical Interventions

Opportunities to Improve the Value of Outpatient Surgical Care

Implications of Evolving Delivery System Reforms for Prostate Cancer Care

The Opportunities and Challenges of the MSSP ACO Program: A Report From the Field

Geographic Variation in Surgical Outcomes and Cost Between the United States and Japan

New Strategies for Aligning Physicians With Health System Incentives

Cost-Benefit of Appointment-Based Medication Synchronization in Community Pharmacies