To identify Medicaid patients, based on 1 year of administrative data, who were at high risk of admission to a hospital in the next year, and who were most likely to benefit from outreach and targeted interventions.

Observational cohort study for predictive modeling.

Claims, enrollment, and eligibility data for 2007 from a state Medicaid program were used to provide the independent variables for a logistic regression model to predict inpatient stays in 2008 for fully covered, continuously enrolled, disabled members. The model was developed using a 50% random sample from the state and was validated against the other 50%. Further validation was carried out by applying the parameters from the model to data from a second state’s disabled Medicaid population.

The strongest predictors in the model developed from the first 50% sample were over age 65 years, inpatient stay(s) in 2007, and higher Charlson Comorbidity Index scores. The areas under the receiver operating characteristic curve for the model based on the 50% state sample and its application to the 2 other samples ranged from 0.79 to 0.81. Models developed independently for all 3 samples were as high as 0.86. The results show a consistent trend of more accurate prediction of hospitalization with increasing risk score.

This is a fairly robust method for targeting Medicaid members with a high probability of future avoidable hospitalizations for possible case management or other interventions. Comparison with a second state’s Medicaid program provides additional evidence for the usefulness of the model.

	Predictive models are powerful tools that can be used to estimate future healthcare costs and opportunities for interventions for individuals.

		Administrative data can be successfully used to identify individuals for care management.

		This study provides a robust method for developing a predictive model to identify these individuals.

		The model is based on available data; most of the derived variables are relatively easy to generate from the data; and risk scores, either developed on a proprietary basis or open source, are easily incorporated into the model.

According to a report from the Kaiser Commission on Medicaid and the Uninsured, in 2011, due to the recession, there were 2 major factors driving state needs to control costs in their Medicaid programs: reduced state budgets and increased enrollment in programs. However, despite these economic pressures, a survey of Medicaid directors conducted by the commission showed a commitment to assuring access to high-quality care delivered in the most efficient manner possible.

 States want to improve the care and well-being of participants in their Medicaid programs and at the same time bring some control to rising program costs.

One method of meeting this commitment would be to identify those segments of the Medicaid population accounting for disproportionate percentages of the costs and within those groups to identify the highest risk members and intervene early in order to avoid unnecessary highcost care. In 2003, the elderly and disabled constituted approximately 25% of the Medicaid population. However, they accounted for about 70% of Medicaid spending that year: 43% by people with disabilities and 26% by the elderly. Only 30% of the spending was accrued by the remaining 75% of the Medicaid population.

Predictive modeling may assist Medicaid plans in identifying program participants at highest risk of future health problems. According to Knutson and Bella,

 “Predictive models are data-driven, decision-support tools that estimate an individual’s future potential healthcare costs and/ or opportunities for care management.” Predictors can be derived from administrative data. This was an approach taken by Billings et al

 when developing a predictive model for the National Health Service in England to identify patients at high risk for rehospitalization. Claims and enrollment data are readily available to payers and can be used in models to target specific groups of interest and to provide risk scores for individuals. These scores could then be provided to case/care/disease managers to help them more readily identify those in need of their services.

A randomized trial conducted by Wennberg et al

 has shown that a targeted care management program can be successful in reducing medical costs and hospitalizations. Billings and Mijanovich

 showed that care management for chronic disease Medicaid patients who had been hospitalized could be cost-effective and could improve the health of this population. Of importance to Medicaid plans, they showed that existing data resources can be used to predict patients at greatest risk of future hospital readmissions within 12 months of an index admission.

These studies also stress the need for developing models and care management plans specific to Medicaid populations. Eligibility requirements, such as low income and/or disability, and other factors typically associated with this population (eg, homelessness, substance use, or low educational achievement) distinguish them from those populations typically covered by commercial plans and their vendors.

Hospitalizations are known to be high-cost events and are easily identifiable and categorized from claims and encounter data. It is also well known from the literature that patients with chronic diseases and multiple comorbidities are at high risk for hospitalization or rehospitalization.

 This situation should hold true regardless of which state Medicaid plan is under study. For these reasons, developing a model predictive of hospitalization for patients with chronic diseases and multiple comorbidities would provide the best opportunity for targeting patients for case/care management that could reduce avoidable costs and be generalizable across states.

In this article, we describe the development of a model to predict hospitalizations among enrollees identified as disabled in a state Medicaid program. Its purpose was to identify Medicaid patients, based on 1 year of administrative data, at high risk of admission to a hospital in the next year and most likely to benefit from outreach and targeted interventions. Previous studies have examined a similar population, but specific to readmissions,

 Applying the model to Medicaid data from a second state supports the generalizability of the model to other programs in other states.

Data for a 2-year period (2007 and 2008) were extracted from a data mart containing a subset of membership and claims information for all Medicaid enrollees in 1 state Medicaid program. (The state was in the southern part of the United States; however, contractual agreements prevent the authors from identifying the actual state studied.) Data extracted for 2007 (measurement year) were used to derive the predictors for the model; outcomes were derived from 2008 (prediction year) data.

The claims experience of the eligible members included information from all measurement year claims/encounters. Inpatient, outpatient, professional, and pharmacy claims were used to obtain predictors based on utilization and diagnosis. Eligibility and enrollment files were used to establish study eligibility and to provide demographic predictor variables.

The Medicaid population is composed of numerous subpopulations defined by the state’s eligibility categories and benefits structures. It would be inappropriate to develop 1 model based on the entire eligible population. Each group has its own characteristics, risk factors, and outcomes.

The population chosen for this study included disabled enrollees who were fully covered by Medicaid and were continuously enrolled for both measurement year and prediction year. Members were identified as disabled if they were enrolled in one of the aid categories defined by the state for the disabled.

The disabled were chosen because they comprised a large portion of the enrolled Medicaid members for the state and were more likely than most other enrollees to be continuously enrolled  for at least 2 years. In 2009, for the state under study, the disabled comprised 19.2% of the Medicaid population and accounted for 44.4% of the costs. Fully covered, continuously enrolled members were chosen because of the need for the most complete claims picture possible. Those enrollees with full Medicaid coverage were chosen to reduce loss of information due to incomplete Medicare claims data from the states involved.

Logistic regression was used to provide predicted probabilities for the occurrence of an inpatient hospital stay for individuals. The regression was performed in Stata and a stepwise process was used for including variables in the model (P was set at .05 for inclusion). The model was specified in a prospective manner. Demographic, utilization, diagnosis, and prescription drug data for 2007 were used to predict hospitalizations in 2008. The coefficients for the most powerful (ie, statistically significant) variables were used.

The dependent variable was the occurrence of an inpatient stay in the prediction year. Inpatient stays were identified by the occurrence of a valid, paid inpatient claim. Admissions due to major trauma or pregnancy were omitted, as these were felt to be unpredictable from the data and less amenable to intervention. Trauma that could be treated outpatient or through an emergency department (ED) was not addressed in this study. The independent variables used in the model included the following:

1. Inpatient stay(s) in the measurement year

3. Primary/preventive office visit in the measurement year

9. Mental health diagnosis in the measurement year (substance abuse)

10. Mental health diagnosis in the measurement year (other than substance abuse)

12. Polypharmacy (8 or more different drugs prescribed in the measurement year)

13. The disease categories included in the chronic disease score (cystic fibrosis, end-stage renal disease [ESRD], human immunodeficiency virus [HIV], anxiety and tension, asthma, bipolar disorder, cardiac disease, coronary/peripheral vascular, depression, diabetes, epilepsy, gastric acid disorder, glaucoma, heart disease/hypertension, hyperlipidemia, hypertension, inflammatory bowel disease, liver failure, malignancies, pain, pain with inflammation, Parkinson’s disease, psychotic illness, renal disease, rheumatoid arthritis, thyroid disorder, transplant, and tuberculosis)

The model was developed by using a 50% sample of claims data for 1 state. The model was validated in 2 ways. It was tested against the other 50% of the eligible population and also against a second state (in the Midwest) to evaluate its generalizability. Stepwise logistic regressions were run separately for each of the samples and the results were compared. Variables were included and excluded based on their significance level in relation to the other variables included in the model. Because the chronic disease score (CDS) and Charlson Comorbidity Index (CCI) and individual disease categories were included in the model, multicollinearity was a possibility. The standard errors of the parameter estimates were examined to determine if multicollinearity existed. Multicollinearity was a possibility if any of the standard errors were large (ie, over 2). All of the standard errors were much lower than 2.

The performance of the model was evaluated using the receiver operating characteristics (ROC) curve. Based on data from the first 12-month period, the model also assigned scores reflecting each member’s risk of hospitalization in the second 12-month period.

Articles

WEB EXCLUSIVE

Predictive modeling can be used to identify disabled Medicaid beneficiaries at high risk of future hospitalizations who could benefit from appropriate interventions.

A Predictive Model of Hospitalization Risk Among Disabled Medicaid Enrollees

Randomized controlled trials (RCTs) reflect priorities established by regulators. Recently, pragmatic clinical trials (PCTs) have begun to attract interest. Unlike RCTs, PCTs aim to better inform post-regulatory decision making by using head-to-head comparisons of alternative treatments, diverse patient populations, and outcomes meaningful to patients, prescribers, and payers.

 To describe how US insurers and public payers perceive the value of PCTs for assessment of new prescription drugs.

 Criterion-based sample of US insurers and public payers.

 We gathered qualitative evidence from intensive interviews with formulary decision makers at 15 payers, representing 10 major types of US payers. Prior literature and exploratory interviews informed our question selection.

Payers viewed PCTs favorably despite wariness of drug company—sponsored trials. Payers would accept results from PCTs as part of payers’ synthesis of multiple sources of evidence. Payers were enthusiastic about 2 PCT features—a diverse population (compared with the more homogeneous populations typical of RCTs) and an active comparator drug (not placebo). Payers did not anticipate that PCTs would displace their own analyses of internal data. Pharmaceutical companies’ financial interest in obtaining trial results that favor their own drugs reduces PCTs’ perceived value and dampens their appeal to payers; nonetheless, payers would seek PCT results and review them carefully, as they do other evidence.

 Recommendations to trial designers based on payers’ views include tailoring different types of PCTs to different disease conditions, building in head-to-head comparisons in phase IIIb PCTs, and designing phase IV PCTs to include broader populations.

US payers desire both scientific rigor in clinical trial design and real-world applicability of trial results. Payers find pragmatic clinical trials (PCTs) appealing because PCTs provide:

		Information that allows evaluation of the comparative value of drugs against active comparators.

		Opportunities to triangulate PCTs’ evidence of effectiveness from real-world settings with data from regulatory RCTs and internal claims data sets.

		Health outcomes for patients who more closely reflect the diversity of patients they enroll (eg, patients with comorbidities, minorities), as well as meaningful data on subpopulationsfor which a drug works.

Interest in comparative-effectiveness research (CER) suggests a need to address both regulatory requirements and requests from post-regulatory decision makers (eg, payers, patients) for evidence of the relative value of comparator treatments. Pragmatic clinical trials (PCTs) have been proposed as potential means for satisfying both regulators and post-regulatory decision makers.

 Although the definition of PCTs is emerging, a recent definition is indicative:

The RCTs whose explicit purpose is to be most informative to decision makers are called pragmatic or practical clinical trials (PCTs). . . . Common elements of such trials include clinicallyeffective comparators, study patients with common comorbid conditions and diverse demographic characteristics, and providers from community settings. Primary and secondary outcomes are patient-centered, chosen to reflect what matters most to patients and clinicians.

Examples of PCTs include head-to-head comparisons of different pharmacologic agents for treatment of various conditions such as hypercholesterolemia

Randomized controlled trials (RCTs) have long been considered the highest level of evidence on clinical efficacy.

 However, regulatory trials historically have focused on narrow patient populations and used placebo comparisons. In contrast, PCTs aim to answer meaningful scientific questions about investigational drugs via study designs with (1) broad patient populations, (2)  active comparators, or both. Traits of PCTs may also include (3) outcomes of interest to patients, prescribers, and payers

; (4) usual care (in contrast to a strict clinical protocol); and (5) fewer data collection elements than RCTs have (to reduce burden of  data collection). Our study defines a PCT as a randomized trial with at least 2 of these 5 “pragmatic” traits; we used this working  definition to create a shared frame of reference with our interviewees. Others, including the pragmatic-explanatory continuum indicator summary (PRECIS) group, have further discussed the extent of pragmatism in PCTs.

Pragmatic clinical trials are seen as a bridge between RCTs (randomized) and observational studies (using real-world data).7 However, according to some observers, payer decision makers often perceive several factors as barriers to their accepting PCTs (

Moreover, because payers’ awareness and acceptance of PCTs have been understudied, uncertainty exists about whether and how US payers, key stakeholders for trials, would use results from PCTs to guide coverage and formulary decision making.

We gathered qualitative evidence to address our primary study question: 

Assuming that PCTs are randomized, how would payers view a PCT’s strength of evidence, compared with an RCT on the one hand and an observational study on the other?

 We pursued specific questions that explored which facets of PCTs payers find more compelling or more problematic than others (See 

 for interview questions). We conducted structured, in-depth interviews of formulary decision makers and senior staff  members at private health insurers and public payers. This study was funded by Eli Lilly and Company.

We selected a criterion-based (purposive) sample

 of 15 payers and related organizations drawn from 10 categories of payers  located throughout the United States (

). In selecting payers, our main criterion was that the sample reflect the diversity of US payer types—from managed care organizations offering comprehensive medical services and prescription drugs to pharmacy benefit managers; from national insurers to regional insurers; and from commercial insurers with nonelderly enrollees to public payers serving seniors.These organizations reflect most major types of US healthcare payers.

Our use of a nonprobability sample restricts the results’ generalizability. But unlike a conventional survey, the study’s purposive, interview-based approach offers a feasible method of obtaining reliable, credible, and detailed information about the use, perceived merits, and perceived difficulties of PCT evidence within US payers. Our method ensured that respondents were formulary decision makers (not junior staff assigned to fill out a website survey). Moreover, we believe our interviewing method reduced miscommunication.

Interview Format and Protocol Development 

The 1-hour interviews typically included 1 to 3 formularydecision makers, drawn from the following: vice president for pharmacy, director of pharmacy, chief medical officer, other physician, vice president for drug evaluation and pharmacoeconomics, and pharmacy and therapeutics committee chair. The interviews followed a protocol of questions, although interviewees’ own logic dictated occasional changes in the order of topics. The protocol permitted interviewees to convey both interest in and concerns about PCTs. After exploratory interviews at 5 payers, we sharpened the protocol’s focus on important, under-researched topics. Moreover, early in each interview, we defined a PCT as a randomized, prospective controlled trial with the traits listed above. This established a common terminology for interviewees and interviewers.

The study protocol posed questions that led interviewees to explore a decision tree (

). To elicit preferences between complex alternatives, we generally separated each alternative into constituent elements. Decision makers were asked to compare element A with element B, and then to explain the considerations that informed their choice. This process was repeated for various features of a PCT and for comparisons between a PCT and other types of studies. Our approach overcame 1 difficulty decision makers displayed in the initial interviews: describing their preferences regarding alternatives, each of which could involve multiple features of a trial or other type of evidence.

In addition, the protocol elicited each payer’s global assessment of PCTs as a type of evidence. This question guarded against misinterpreting a payer’s whole view of PCTs as being the sum of responses to piecemeal questions. Payers answered our questions in terms of clinical trials that met our definition of a PCT and typically did not refer to a particular PCT that they had reviewed, because their own experience with PCTs was limited.

To develop findings, the 2 investigators who conducted the interviews independently reviewed their own notes, identified interviewees’ main points, and classified them as “consensus” or as representing 2 or more interviewee groupings. These  investigators (who were not pharmaceutical company employees) then reviewed each other’s readings of the main points and  their classification, resolved differences by reexamination of their notes, and produced findings that synthesized the interviews.

Theme 1: Payers’ Views of Place of Pragmatic Clinical Trials in Chain of Evidence

. All formulary decision makers consider PCTs to be rare (to date). Some decision makers did cite a head-to-head trial funded by a drugcompany as an example of a PCT, but most struggled to recall PCTs they had used in decision making. Even “classic” PCTs, the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) and Clinical Antipsychotic Trials in Intervention Effectiveness (CATIE),

 did not come to mind for many. We could not determine whether the issue was faulty memory (decision makers could not recall these trials at all) or imperfect labeling (decision makers did not link these trials to the term “PCT” because their practical features did not stand out). Segal

 explains why these 2 trials can be considered pragmatic. Payers’ impression that PCTs have been rare is consistent with a review that found only 22 PCTs out of the many clinical trials with reports published from January 1996 through September 2010.

Once comfortable with the PCT concept as we defined it, payer decision makers stated that PCTs would be part of the continuum of evidence they consider in assessing new drugs. As our interviewees explained, when gathering evidence about a new drug, their default mode is triangulation. Rather than focus on a single type of evidence, decision makers assemble several types—for example, RCTs, analyses of their own claims database, and observational studies of postlaunch data on populations other than their own enrollees. After weighing the different sources’ conclusions, limitations, and relevance to their own enrollees, decision makers triangulate (ie, draw inferences from multiple evidence sources about the relative effectiveness of a new drug). Accordingly, decision makers stated that PCTs would become one more link in the chain of evidence they consider.

Overall, among interviewees, the concept of triangulation across RCTs, PCTs, registry data, internal claims-based studies, and observational studies was widely shared. In an alternative analogy, evidence forms a multidimensional crossword puzzle: each dimension (type of evidence) provides a clue to the answer.

 Formulary decision makers find PCTs appealing because they improve the external validity of the evidence base. Payers generally view explanatory RCTs as exemplars of internal validity. Nonetheless, decision makers stated they want evidence of external validity: that the results apply not just to the RCT’s population but also to the payer’s own enrollees. Decision makers noted that patients excluded from many RCTs often resemble payers’ own patient populations more than the   RCTs’ restricted populations. Consequently, payer decision makers generally favor PCTs over RCTs because PCTs’ broad  population holds greater relevance to payers. Furthermore, payers recognize that more external validity (a wider array of subgroups studied) often entails less internal validity (small subgroup samples and less statistical power to detect a treatment effect). Among decision makers interviewed, the consensus was that external validity trumps internal validity—perhaps with caveats; for example, when a drug is the first in a therapeutic class or when evidence of effectiveness of comparators is weak.

Ranking Randomized Controlled Trials and Pragmatic Clinical Trials

. The decision makers interviewed differed in their views of where in their hierarchy of evidence PCTs would fit, especially when the evidence available included traditional RCTs. Some decision makers rejected the idea of ranking because RCTs and PCTs are incommensurate: “RCTs are the gold standard for demonstrating whether a drug works” while “PCTs provide evidence on the types of patients for which a drug works.” Other insurers expressed a related perspective: RCTs demonstrate whether a drug works under ideal conditions; in contrast, PCTs demonstrate whether a drug that works in an RCT also provides benefit under real-world conditions (imperfect patient adherence as well as physicians’ uneven conformity with practice guidelines). Most payers stated that traditional RCTs should be conducted to establish efficacy against a placebo in most (not all) therapeutic areas. In contrast, 1 payer wanted PCTs to supplant all RCTs. Nonetheless, most payers want traditional RCTs as well as PCTs and suggest that traditional RCTs should precede the PCTs.

Theme 2: Payers’ Views of Pragmatic Clinical Trial Design

Formulary decision makers emphasized that they hold nuanced views regarding PCT design. Five aspects stand out:

. As seen by decision makers generally, different medical conditions call for PCTs with different features. For example, condition X might call for a PCT with a broader patient population than most RCTs, while condition Y might call for a PCT with a head-to-head comparison or with particular outcomes measures.

 Although decision makers noted that diagnoses and clinical factors often affect which subgroups they want included in a trial, they singled out several subgroups frequently as key for inclusion in PCTs. Although designers of a PCT can select from an array of population subgroups, payers most often mentioned women and seniors. Most decision makers stated they want study populations to include more women. A smaller but sizable segment of interviewees wanted study populations to include seniors.

 Payers typically urge PCT designers to include “their” ethnic group in the study population. Ethnic groups that are most prominent among health plans’ enrollees differ by region. For example, it is possible that African Americans are the salient minority group in Northeast plans, Hispanics in Southwest plans, and Asians and Hispanics in West Coast plans.

Payers like key traits of pragmatic clinical trials, but are wary of pharmaceutical companies and plan to carefully scrutinize this new, appealing type of evidence.

Pragmatic Clinical Trials: US Payers' Views on Their Value

 To perform a systematic review of the current literature to assess the association between integrated healthcare delivery systems and changes in cost and quality.

: Medline, Embase, Cochrane Reviews, Academic Search Premier, and reference lists were used to retrieve peer-reviewed articles reporting outcomes (cost and quality) related to integrated delivery systems. A general Internet search and reference lists were used to retrieve non—peer reviewed publications meeting the same criteria. Included peer and non–peer reviewed publications were based in the United States and were published between the years 2000 and 2011.

 A total of 21 peer-reviewed articles and 4 non—peer reviewed manuscripts met the inclusion criteria. Twenty studies showed an association between increased integration in healthcare delivery and an increase in the quality of care. One study reported no changes in quality indicators associated with increased integration. None of these studies measured cost reduction directly, but used reduction in utilization of services instead. Four studies associated decreases in the utilization of services with increases in integration.

The vast majority of studies we reviewed have shown that integrated delivery systems have positive effects on quality of care. Few studies linked use of an integrated delivery system to lower health service utilization. Only 1 study reported some small cost savings.

Recent discussion of health reform has placed much emphasis on integrating healthcare delivery systems. Do integrated healthcare systems truly do a better job of providing highquality care while holding down costs? Very few studies have provided empirical evidence.

		We conducted a systematic literature review (2000-present) including peer-reviewed and non—peer reviewed studies, and analyzed 25 eligible studies that examined integrated delivery systems on the topics of cost and quality.

		The majority of these studies reported positive correlation between health system integration and quality of care.

		The evidence on reducing cost of care, however, is rather weak.

The US healthcare system has long been characterized as complex, fragmented, costly, and with significant variation in quality of care. During the health reform debate, many health policy experts have called for the country to reorganize healthcare providers and delivery systems through organizational or virtual integration. The concept of integrated healthcare delivery systems (IDSs) has gained considerable interest. Such systems have been viewed as a better approach to addressing the issues of quality and cost. Indeed, some IDSs are frequently portrayed not just in academic and medical circles but in the popular media as examples of the best, most effective healthcare in this country. However, the evidence used to support such assertions may not have been systematically evaluated. Now that healthcare reform has been enacted and accountable care organizations (ACOs), which have many organizational features similar to those of IDSs, are rapidly being developed and implemented, what do we know about how this approach to organizing healthcare impacts the key factors of quality and cost of care? Do IDSs truly do a better job of providing high-quality care while holding down costs? A comprehensive review of the existing literature that assesses the relationship between IDSs and cost/quality is critically needed.

Much of the recent innovation in US health policy has been based upon a fundamental belief that a higher level of integration will yield a more efficient healthcare delivery system. An IDS presumably provides higher quality and more patient-centric care at lower costs. However, there is no clear definition of what constitutes an IDS. Recently reorganized healthcare delivery systems (eg, ACOs) come in all sizes and shapes, and span a wide spectrum of types and levels of integration. Inconsistency in concept and vague definitions can pose significant challenges as policy makers strive to design policy tools to steer the current healthcare away from uncoordinated and fragmented health systems. Integrated healthcare is complex and has been categorized conceptually in 2 ways: (1) an organized structure that is managed by a financial entity (eg, a financial group that manages different facilities within a healthcare system) or (2) an organized healthcare delivery system that coordinates care and has synchronized functioning.

 An IDS may encompass providers belonging to multiple facilities who are responsible for providing quality patient care rather than providers from a single facility like a hospital or general practice.

Integrated healthcare has numerous definitions.

The World Health Organization defines integrated delivery as “the organization and management of health services so that people get the care they need, when they need it, in ways that are user-friendly, achieve the desired results and provide value for money.”

(p1) While this definition is focused on the patient experience of integrated delivery, other definitions focus on the responsibility of the system. One such definition states that organized delivery systems consist of “a network of organizations that provides or arranges to provide a coordinated continuum of services to a defined population and is willing to be held clinically and fiscally accountable for the outcomes and health status of the population served.”

(p7) Finally, other definitions focus on the specific services and features that describe an IDS as “one or more hospitals along with physicians, diagnostic centers, and other components of the supply side of the supply chain strive to share information, minimize duplication, and make treatment decisions based upon the institutional best practices.”

The preceding concepts and definitions illustrate the complexity and variation involved when discussing IDSs. However, they also help to determine the commonalities present. All of these concepts and definitions include increased communication and information sharing across the care continuum. There is also a common understanding that integrated care should coordinate patient care in a way that improves the patient experience and the quality of the care received.

Jonas and Kovner’s Health Care Delivery

 in the United States includes the various forms of IDSs under the umbrella of organized healthcare delivery. This book defines organized healthcare delivery as a situation where “Care providers have established relationships and mechanisms for communicating and working to coordinate patient care across health conditions, services, and care settings over time.”

(p206) This definition is broad enough to capture many different system structures, while still including the essential functions that an IDS must possess. Hence, for this systematic review, we include broader definitions of IDS that focus on care coordination and system integration.

Literature Search Strategy and Inclusion Criteria

This systematic literature review provides a comprehensive look at the current peer-reviewed literature on IDSs. It also includes a review of other non—peer reviewed publications on the subject, including white papers and reports from nonprofit and industry organizations. Although these documents have not been independently evaluated through peer-review processes, we believe that these sources can provide valuable insights. Due to the fact that these documents are not required to go through a lengthy review process required for publication in a peer-reviewed journal, they are often the first to report on emerging topics and issues.

For peer-reviewed articles, a systematic literature search was performed in 2012 using Medline, Embase, Cochrane Reviews, and Academic Search Premier, as well as the follow-up references in retrieved articles for outcomes (cost and quality) of IDSs from 2000 to 2011. Search terms included 

integrated delivery system, integrated care, integrated service network, continuity of care, chains of care, care coordination, coordinated care, provider system integration, and vertical integration

While the literature search for non—peer reviewed articles was not as systematic as that for the peer-reviewed literature, it was conducted through a similar process. A general Internet search engine was used to find white papers, case studies, and reports from various organizations. Several peer-reviewed publications cited such documents as well, leading to several more studies for consideration. The content of these documents was then assessed to determine the applicability of each paper to this review.

Articles were included if they evaluated outcomes (cost and quality) of IDSs. We excluded theory-based articles, articles not specific to the US health system, and articles published before 2000.

Numerous publications on IDSs were available. However, the majority of this material focused on the key elements required to successfully adapt current health systems to IDSs.

 In this review we were interested in articles and papers pertaining to the cost of care associated with IDSs and the quality of care they provide. Previous literature reviews have addressed the issue of how to define and measure integrated healthcare delivery

 but did not link integration with outcomes.

After the initial literature search, we identified 168 peerreviewed articles. We excluded 78 based on title and abstract review. A total of 90 full-text articles were assessed for eligibility, 69 of which were not eligible, for a final sample of 21 articles. All 21 articles evaluated some measures of quality of care and correlated these measures with health system integration; 6 of these articles also assessed the impacts on cost of care.

The initial search of the non—peer reviewed literature generated 27 white papers, reports, and case studies for consideration. Of these studies, 13 were excluded based on abstract review. The remaining 14 studies were read in their entirety to assess their eligibility for this review. Those that did not pertain to information on cost or quality outcomes were excluded. This resulted in a final sample of 4 non–peer reviewed publications (

To synthesize our literature review findings, we abstracted study settings and key findings from all selected papers into 3 domains: health system focus, outcome measures, and conclusion.

 Enhancing care continuity and coordination are 2 important components of IDSs. With more stringent guideline- or protocol-based care, integrated healthcare systems seemed to provide better quality of care in our systematic review. Out of 21 studies, 19 showed improvement in quality of care with respect to clinical effectiveness,

 Compared with non-IDSs, several studies have found more favorable outcomes for patients treated in IDSs for chronic diseases such as diabetes, hypertension, depression, congestive heart failure, and asthma.

 There is evidence of some improvement in the care delivery process as a result of IDSs, which when quantified ranged from 10 to 25 percentage points.

 For example, 4 studies found that IDSs improved patient medication adherence and decreased medication error rates.

Focusing on outpatient primary care, 1 study evaluated the association between organizational structure of physician groups and healthcare quality to determine whether IDSs provided higher quality care.

 These researchers found that integrated medical groups performed better on 4 Healthcare Effectiveness Data and Information Set (HEDIS) measures related to preventive health screenings; although on 2 measures related to management of chronic disease, there was no statistically significant improvement.

One study found that there was a correlation between integration and the presence of practice systems for chronic care management.

 Functional integration was much more significant than merely structural or financial integration as a determinant of chronic care systems. Similarly, another study found that large physician groups and those owned by a hospital or health plan were more likely than smaller groups to use evidence-based care management processes.

Four studies looked at the impact of electronic medical record/health information technology in IDSs and found that introducing health information technology creates operational efficiencies and patient-centered care

Few studies examined the impact of IDSs on cost of care. Only 5 peer-reviewed papers in our systematic review looked at healthcare cost.

 Of those, most studies provided evidence about the differences in utilization of services for patients treated in IDSs versus non-IDSs. In other words, the level of service utilization per patient was used as a proxy measure for cost of care. Using this measure, 4 out of 5 papers reported that IDSs were associated with lower cost of care. Two studies found that clinical service integration in various forms of ACOs and large multispecialty group practices would lower cost of care.

 One study examined the medication treatment management program within an IDS and concluded that the program generated cost savings related to medication use.

One study discussed the cost related to the level of service integration within the hospital for patients with heart failure, but found no significant relationship between integration of services and economic outcomes.

The studies selected for this review consist of 2 monographs

 Of the 4 selected non—peer reviewed publications, 3 reported that improvements in quality of care were associated with health systems’ clinical service integration. All 3 reports showed improvement in HEDIS measures.

 Among the 17 different health systems these reports examined, IDSs were associated with a lower hospital admission rate per patient, a shorter length of stay per hospital admission, and an overall lower rate of adverse health outcomes. With IDSs, there also were increases in the use of evidencebased practices by physicians in various fields.

 Many IDSs also saw improvements in preventive care (eg, increased vaccination rates).

 Although these studies appear to indicate a positive correlation between integration and quality of care, 1 report found that expected quality improvements were not present. They also found that patients participating in multiyear wellness and prevention programs in 1 IDS showed little to no health improvement38 (

Of the 4 non—peer reviewed publications selected for this systematic review, 3 examined cost of care. All 3 publications reported that cost savings were seen in health systems that had various features of clinical integration. Of these 3 reports, 2 found that there were substantial improvements in quality-of-care measures that the researchers assumed led to indirect cost savings for the system. These reports also noted that substantial cost savings were associated with electronic prescribing implementation due to a tandem generic medication prescription initiative within the IDS involved.

 While these reports indicate positive results, 1 report using data from several IDSs across the country found that, among the systems observed, there was very little cost difference, and that when cost differences were observed they were much smaller than what was expected.

A variety of emerging concepts describe IDSs. Most of the studies are theory based, yet the tenets of the theories have not been tested empirically to evaluate the relationship between the level of integration and either cost or quality outcomes. Earlier work by Shortell and colleagues

suggests that it is not size or structure per se, but rather an organizational commitment to, and culture of, continuous quality improvement that is most closely linked to better performance as measured by clinical quality, patient satisfaction, organizational learning, and financial performance. However, relatively few medical groups were high performers in all performance categories. In many cases, a medical group’s performance varied considerably within the domain of clinical quality; for example, a medical group that performed well in chronic disease management might be average or below in health promotion. Furthermore, the same study by Shortell and colleagues showed no correlation between high performance on quality measures and financial performance, calling into question the assumption that IDSs are inherently better at both improving quality and controlling costs. The researchers did find some positive correlation between high performance on quality measures and certain indicators of integration (eg, larger practice size, affiliation of the medical group with a hospital, health system, or health plan), yet affiliated practices were actually less likely to be high performers on financial measures.

The result of our systematic review provides partial evidence outcomes. The evidence on cost of care, however, is rather weak. No study in our review was able to directly measure the cost savings as a result of integration of a health system. Most studies reported a decrease in healthcare utilization as a result of health system integration but did not assess whether the decrease of service utilization was appropriate. While many of the IDSs involved in these studies had improvements in quality of care with lower service utilization, it is important to note these studies’ limitations. First, many of these reports did not include any indication that statistical tests were performed on the data. While the general data trends may have been positive in the areas of cost and quality, significance cannot be assumed. Also, most of the IDSs included in these studies were implementing multiple programs within and outside of their clinical integration initiatives. It is important to note that healthcare systems may operate under different mixes of reimbursement models (eg, managed care contracts). It was not possible for us to take this factor into consideration, as most of the studies included did not report on reimbursement models.

It is also difficult to distinguish the effect of integration per se from the effects of characteristics (eg, larger patient volume, greater use of electronic medical records, systematic quality improvement initiatives) that tend to be associated more frequently with integrated systems. Therefore, larger sample sizes that include various types of IDSs and more consistent outcome measures are needed to gauge the effects of IDSs on the cost and quality of care.

ACOs and their features involving integrated delivery of healthcare are currently at the forefront of healthcare policy. The hope for the ACO is that it will reduce healthcare costs and improve the quality of care. However, there are no data to support these assumptions, as these systems have only recently been created. Integrated healthcare delivery systems share with ACOs many of the same organizational features, foremost among them being high levels of organizational and clinical service integration. For this reason, the performance of existing IDSs with respect to quality and cost of care are being assessed to predict how well ACOs will perform in the future US healthcare system.

The vast majority of studies we reviewed showed that IDSs have positive effects on quality of care. A few studies we reviewed also linked IDSs to lower health service utilization. However, each study had a different way of describing and defining IDSs. In order to compare IDSs with respect to their effectiveness in controlling costs and improving quality of care, consistent definitions should be used and components of integration need to be well defined. Due to limited information provided in the literature, we were unable to identify specific features in each health system. Nor were we able to meaningfully group the health systems based on their commonalities into various types of IDSs. As a result, it was difficult to draw definitive conclusions. Moreover, to accurately capture the performance of a health system, established methods that consistently measure cost and quality of care across health systems are needed.

Our study has other limitations. The study lacks generalizability due to the heterogeneous nature of the IDS in the studies we reviewed, the unavailability of effect size calculations, and the absence of systematic working definitions for outcomes. There also may be publication bias since there is a higher probability of peer-reviewed journals publishing studies with favorable results rather than studies with inconclusive or unexpected results.

As integrated systems become more common, it also is becoming more apparent that despite certain commonalities, these systems vary tremendously in how they operate. The emergence of a growing number of IDSs should allow for more data-driven studies comparing organizational performance on quality, safety, access, and cost metrics, thereby adding more insight into exactly what characteristics of IDSs lead to improvements in each of these areas. The question, perhaps, is not so much whether IDSs are better than other models of delivery, but rather, how to identify specific system features that most effectively improve quality of care while controlling costs.

 From Department of Public Health Sciences (WH, JC, ML), Department of Medicine and Public Health Sciences (HP), Division of Health Services Research, Penn State University College of Medicine, Hershey, PA.

Integrated health systems are often portrayed as the most effective in providing high-quality care at lower costs. This systematic review assesses the empirical evidence.

Effects of Integrated Delivery System on Cost and Quality

 The goal of this pilot study is to demonstrate whether revisiting and focusing on simple and generally known primary care officemanagement practices has a meaningful impact on emergency department (ED) utilization for conditions that likely could have been treated in the primary care office setting (primary care physician [PCP] treatable).

Cohort study using health plan administrative data from 2007 to 2010 involving primary care physicians (PCPs) affiliated with both Blue Care Network of Michigan, a nonprofit health maintenance organization, and Oakland Southfield Physicians PC, a Metropolitan Detroit independent practice association. PCPs were assigned to cohorts according to pre-intervention increasing or decreasing temporal trends in annual ED visit rates for PCP-treatable conditions by 12-month continuously enrolled commercial members withthe same emergency care copay.

 A difference-in-difference approach measuring control and intervention PCPs for the same 4 months (September-December) during 3 years (2007-2009) pre-intervention, and the available same 4-month period post-intervention, to determine if the pilot was associated with decreased ED utilization for PCP-treatable conditions.

A substantive reversal of a worsening 2007 to 2009 trend (peak of 49.2 visits per 1000 in 2009 decreased to 7.3 visits/1000 in 2010) in ED use for PCP-treatable conditions at intervention sites, with the 2010 rate also lower than control sites (23.8 visits per 1000) during the same postintervention period.

Simple and effective practice management techniques, while generally known, require revisiting and focused attention by PCPs to limit rates of PCP-treatable ED visits.

		There exist simple and largely known primary care office management techniques that may limit emergency department visits for conditions generally treatable in the primary careoffice setting.

		These techniques are not necessarily well understood or implemented properly by all primary care physicians (PCPs), and should be revisited.

		Clear direction, support, and follow-up from a larger organization with which a PCP is affiliated (eg, independent practice association [IPA] or accountable care organization [ACO])may be necessary to achieve more consistent and effective use of these techniques within primary care practices.

Emergency department (ED) use as a substitute for primary care physician (PCP) acute minor episodic care has been an ongoing topic of interest in healthcare.

 One study that explored non-urgent visits to a pediatric emergency department demonstrated that 62.8% of visits were for parental convenience, and of the 45.4% of parents who did contact their PCP first, 72.6% were referred to the ED.

Additionally, perceptions of PCP unavailability appear to be a reason why some patients do not attempt to contact their PCP prior to an ED visit. Another study also focused on pediatric emergency care revealed that ED visits for non-urgent conditions were not perceived as a significant enough breach in continuity of care by either PCPs or parents to warrant any concern, and discrepancies exist between PCP and parental perceptions of adequate PCP communication and access.

Increased ED use is not solely due to the uninsured or those without a PCP, as is often perceived, and thus PCP office  access affects everyone.

 When a problem is identified, our society tends to look for solutions involving innovative complex systems or expensive technology. Awards are not given for revisiting past, less technology-based solutions.The goal of this study is to demonstrate whether revisiting and focusing on simple PCP office management practices, and performing them well, has a meaningful impact on ED utilization for conditions that could have been treated in the PCP office setting.

The Greater Detroit Area Health Council (GDAHC)

is a multistakeholder organization with the mission of driving collaborative improvements in healthcare quality, cost-effectiveness, and access to care across southeastern Michigan. GDAHC convened a multi-stakeholder team (payers, purchasers, consumers, and providers) to address ED utilization. An outcome of team deliberations was a goal of reducing ED visits for conditions when care likely could have been provided in the PCP office (PCP-treatable conditions) with interventions for improving PCP access.

Blue Care Network of Michigan (BCN) is a nonprofit, statewide health maintenance organization and wholly owned subsidiary of Blue Cross Blue Shield of Michigan (BCBSM). This study used BCN administrative data for commercial members only. The study time frame included the years 2007 to 2010, during which time total BCN commercial membership averaged about 500,000. BCN PCPs are charged with coordinatingoverall care for BCN members and are generally affiliated with a primary care group (PCG). PCGs are not physician practice groups, but rather each PCG is a business entity composed of 1 or more physician practices and functions essentially like an independent practice association (IPA). The PCP practices belonging to a PCG may consist of physicians who are salaried, independent, or a mixture. Generally, each PCP (as opposed to the practice the PCP may belong to) has an individual contract with the PCG with which he or she is affiliated. PCGs vary as to the comprehensiveness and sophistication of their administrative structures in support of the financial goals of the PCG, and in their risk-sharing arrangements both with BCN and with their PCPs. Oakland Southfield Physicians PC (OSP) is an IPA that  operates in Metropolitan Detroit and participates with BCN as a PCG. It consists of approximately 300 primary care physicians. OSP management (both physician and non-clinical) participated on the GDHAC multi-stakeholder team.

BCN and OSP collaborated on a pilot program to study implementation of the GDAHC team’s recommendations for improving PCP access. These recommendations include adopting telephone triage processes and recorded messages that direct patients to the appropriate venue for care; establishing a strategy on how patients can obtain acute minor episodic care when the PCP is unavailable and communicating that strategy to patients; and implementing a scheduling strategy (eg, open-access scheduling) to support same- or next-day appointments including evenings and weekends. Pilot program activities included education of all intervention PCP offices on the initiative and sharing of program materials. Materials were customizable to specific offices (ie, new patient welcome letter and current patient brochure on use of the ED) and also included office procedures for access to care (ie, telephone triage, appointment scheduling, and patient follow-up after a known ED visit). A sample after-hours telephone script was provided, as well as recommendations on how to use the OSP monthly ED visit reports (supplied to OSP by major payers in the area using paid claims data). The launch date of all program tools was tracked and OSP engaged in structured communication with the intervention practice sites at frequent, established intervals to support implementation of the program’s recommendations and tools.

A summary of the OSP intervention site communication program is contained in 

The sample materials, to be edited as appropriate for the actual circumstances of a specific practice, were provided to each intervention site and are contained within the 

Different methods for identifying PCP-treatable conditions were evaluated and consensus reached by the GDAHC team, which included representation from primary care,emergency physicians, a major hospital system, and health plans. The final method was also shared with the GDAHC Data Users Group and adopted as part of its future plans forregional surveillance of ED use in the GDAHC service area.

International Classification of Diseases, Ninth Revision

) diagnosis codes compiled by BCN and used for various analyses and reports. The diagnosis codes used represent the primary

 code on an emergency visit claim. Secondary, tertiary, etc, diagnosis codes that may (or may not) be present on a claim were not used to construct the list. The list contained elements of prior lists used by BCBSM and the list of codes contained within the Billings New York University algorithm, updated by using an Agency for Healthcare Research and Quality 

 diagnosis category grouper to aid in the process.

codes were excluded, as were any codes not at least 50% of the time also present in BCN PCP office encounter data from a 12-month sample of PCP and emergency claims.

 codes were also excluded. Emergency department presentation of behavioral health issues falls into 2 main categories: overt and covert. Overt presentations include diagnoses such as suicide attempt or ideation, psychosis, drug overdose, and alcohol intoxication. Covert presentations include potentially serious physical complaints with a psychological root cause that may or may not reveal itself during the clinical encounter (eg, chest pain in a depressed person to gain attention from family, anxiety presenting as chest pain). While the overt presentations might be considered by some to be PCP preventable (perhaps ED visit avoidable had the PCP better managed a patient’s substance abuse and/or psychological condition), such visits are not PCP office treatable, as the situation has escalated to a more severe manifestation making the emergency department the appropriate venue for evaluation. Patients with covert behavioral health presentations would be less amenable to the PCP access maneuvers described in this pilot, as seeing their PCP may deprive them of the desired psychosocial dynamic that is part of the spectacle of emergency care itself (eg, ambulance transport). Anecdotally, PCPs express that they have little influence in regard to such covert emergency visits and holding them accountable for such visits is unfair. Therefore, behavioral health diagnosis codes were originally excluded to increase provider acceptance of the code list in regard to BCN reporting in general, and remained excluded for the purposes of this pilot program to eliminate the possibility of introducing any bias or detracting from acceptance of this analysis.

The list was edited by BCN staff, which included general emergency medicine, pediatric emergency medicine, internal medicine, and family physicians. The final list consistsof

diagnosis codes considered more likely than not for conditions that could be treated in the PCP office setting. The purpose of this type of tool is not for denying claims or making declarations about the appropriateness of a specific ED visit, but rather to follow trends in a population (eg, health plan, PCG- or PCP-affiliated members).

OSP PCPs were identified who had more than 100 BCN 12-month continuously enrolled commercial members in each of the 3 years from 2007 to 2009 to establish PCP “engagement” with BCN and OSP in general, with this threshold based on prior published BCN research.

 Engagement is important to ensure that a PCP has a sufficient number of patients under a specific contractual arrangement to have interest in any associated  initiatives or programs. Next, to compare “apples to apples,” members with the same benefit design in regard to an emergency  care copay for all 12 months in any given year were identified. A member did not need to have the same benefit design or be a BCN member for all 3 years, but did need to be a BCN member with the same benefit design for all 12 months of any given year. A $50 copay was the most prevalent in 2007 and 2008 for OSP-affiliated BCN members, and the second-most prevalent in 2009. The $50 emergency copay members, of which 99.96% were also with the same PCP for 12 months of any given year, were linked to the OSP PCPs who met the overall engagement criterion. The engaged PCPs were required to have at least 30  such BCN members in each year (2007-2009) to be further considered for entry into either the intervention or control cohort. The count of BCN members who met criteria for study inclusion, by PCP, was used for rate calculations and the 30-member  minimum reduces extreme results due to small denominators while leaving sufficient PCPs to evaluate for inclusion in the pilot.

ED visits for PCP-treatable conditions, as defined by the BCN 

code list, were tallied for $50 copay members affiliated with BCN/OSP-engaged PCPs who also had at least 30 of the OSP-specific member subset in their BCN patient panel. These counts were then used to calculate ED visit rates, by year, for OSP PCPs that accounted for BCN/OSP engagement and member benefit design. The rates were then trended over 3 years to identify OSP PCPs with ED visit rates that were consistently worsening (yearly increase in utilization), consistently improving (yearly decrease in utilization), or that showed no consistent trend. In conjunction with OSP management, a demographically representative sample of PCPs was selected with a worsening trend for the intervention cohort, and PCPs with an improving or no trend as controls. GDAHC pilot program activities would not be implemented for an identified PCP alone if in a group practice, but rather for the entire office. Therefore, the initially identified intervention and control PCPs were labeled as index physicians and any other PCPs in an index physician’s practice wereidentified and included in the same cohort regardless of the ED utilization associated with the index PCP’s partner(s).

A total of 26 PCPs were included in the pilot, 15 at 6 practice sites as controls and 11 at 5 practice sites for program interventions. The specialty (as per BCN credential files) distribution of control PCPs was 8 family practice, 2 internal medicine, and 5 pediatric physicians. The intervention cohort consisted of 4 family practice, 1 internal medicine, and 6 pediatric  physicians. Self-reported practice environment for the intervention cohort was 3 in urban and 8 in suburban locations, and controls were 4 urban and 11 suburban. One control and 2 intervention PCPs were in solo practice. By design, all 11 intervention physicians had an index PCP association with a worsening trend in ED use for PCP-treatable conditions; among controls, 12 had an index PCP association with an improving trend and 3 with no consistent trend.

Prior to this pilot some PCP sites were designated as BCBSM Patient Centered Medical Homes (PCMHs) and participated in  the BCBSM Physician Group Incentive Program (PGIP). PCMH participation meant an office had a PCMH welcome letter with an  access to care statement and had written a PCMH access to care policy, as defined by the PCMH program. Participants had a PCMH “start date” and these practices would have been integrating PCMH operational procedures prior to that date. PCPs within OSP specifically did not receive extra PGIP financial incentives in regard to ED utilization; therefore no differing financial incentive for OSP PCMH sites versus OSP non-PCMH sites existed. Of the 15 control PCPs, 6 (40%) had a PCMH start date of July 2009 and 4 (27%) July 2010. In regard to intervention PCPs, 4 (36%) had a PCMH start date of July 2009 and 3 (27%) July  2010.

OSP PCP control and intervention practice identification, pre-implementation survey data collection, and intervention material creation were completed from June to July 2010. Survey data, which included PCP practice information, current office processes, and opinions, were collected by OSP management via telephonic interview. During August 2010 OSP introduced program materials to the intervention offices and tracked when specific program items were implemented. Intervention practices began utilizing program materials September 2010. Following 4 months (through December 31, 2010) and an additional 60 days for claims run-out (lag period to allow for submission of claims from providers), 2010 claims data were extracted and only the intervention practices resurveyed. PCP-treatable ED utilization from pre- and post-implementation was assessed for intervention PCPs, and compared with control PCPs, to determine if there were any substantive changes.

All study practices in control and intervention cohorts responded “yes” to the survey questions of: office staff utilizes a formal telephone triage process during office hours to evaluateif a patient  has an acute clinical problem and to evaluate call urgency; after-hours calls are screened by an answering service before allowing a connection to a physician; same- or next-day

appointments available; and practice offers “open-access” scheduling. Nearly all (93%) of control and a preponderance (64%) of intervention sites provided new patients with the name of a preferred urgent care center(s) for acute minor episodic care when immediate care was deemed necessary by the patient but not available from the PCP. A preponderance of both control (80%) and intervention (64%) practices also provided new patients with instructions on the appropriate use of the ED. Most control (87%) and intervention (73%) sites felt their patients were likely to use the ED for PCPtreatable conditions because the office was closed. None of the control sites felt their patients were likely to use the ED because they could not schedule an  office visit promptly, whilea minority (27%) of intervention sites were of that opinion. E-mail was used to communicate with  patients at 40% of control practices and at none of the intervention sites. Details are provided in 

Post-intervention Survey of Intervention Practices Only

All intervention sites made no change in regard to handling calls during regular business hours. A total of 27% changed how after-hours calls were handled, and 18% made changes in availability and scheduling of same- or next-day appointments. After the pilot, 73% of intervention PCPs supplied new patient information regarding ED utilization, a slight increase from the pre-intervention 64%. There was no change regarding the use of e-mail to interact with patients. All intervention sites felt the tools supplied by OSP made ED visits for PCP-treatable conditions less likely in regard to obtaining timely appointments and when the PCP office is closed. Most felt providing a follow-up letter or a telephone call after an ED visit helped reduce future use and also felt the pilot overall was a positive experience. Details are provided in 

The measurement period to determine any change in temporal trend was short, only 4 months (September to December 2010). While annual trends 2007-2009 were used for PCP cohort assignment, utilization outcomes for the last 4 months

of 2010 were evaluated against the same 4 months in 2007 to 2009 to consider seasonality in ED visit patterns. Counts of PCPs, affiliated BCN members, and ED use for PCP-treatable conditions are noted in 

. The prevalence of the $50 emergency care copay among OSP-affiliated members decreased over time, reducing eligible members to include in the 2010  assessment. The most prevalent copay in 2010 ($100) was less prevalent in prior years. Of the members in Table 4, 99.94% were with the same PCP for all 12 months of a given year.

Intervention sites had increasing rates of ED use for PCPtreatable conditions, which peaked at 49.2 visits per 1000 in 2009, and decreased to 7.3 visits per 1000 when measured for the same 4-month period after the pilot began in 2010. This represents a substantive reversal of the prior 3-year trend and is also lower than the 23.8 visits per 1000 for control sites during the same 4-month period (

The study design represents a difference-in-difference approach using 3 years of prior utilization to establish the temporal trend before the intervention. Cohorts of extremes (worst and best performing PCPs) were identified to make any change in trajectory as apparent as possible, as these data were too limited for formal statistical testing. The premise was that PCPs with an improving trend had, in the past, embarked upon a program of process improvement (specifics unknown) in regard to ED use for PCP-treatable conditions, while these data suggest that intervention sites were not addressing the issue effectively. Considering the small size of the study, control and intervention cohorts were reasonably similar in regard to the distribution of practices and PCPs by PCP specialty and self-reported practice environment. The percentage of PCPs with PCMH participation, and dates of engagement with the PCMH program, were also reasonably balanced. Intervention PCPs, in aggregate, had ED visit rates for PCP-treatable conditions increasing steadily for 3 years before the pilot in 2010, with clear improvement after interventions were implemented, while the ongoing aggregate improvement for control PCPs moderated. As the distribution of PCMH-participating PCPs was largely balanced between the 2 cohorts, the positive impact at intervention sites suggests the pilot provided benefits beyond the PCMH/PGIP program alone, as no additional financial incentives were attached to the GDAHC pilot, or the PCMH/PGIP program, for OSP PCPs in regard to ED use.

In the pre-implementation survey all control and intervention PCPs attested to already having a practice structure (telephone triage during office hours and appointment access processes) that should optimize PCP access and control ED use for PCP-treatable conditions. Both cohorts generally had new patient instructions regarding when to use an urgent care center as opposed to the ED, though this was more prevalent in the control group. The majority of PCP opinions in both cohorts felt patients went to the ED because the office was closed, yet most reported providing instructions to patients directing them to alternative sites of care (eg, urgent care). Hardly any PCPs felt their patients went to the ED for PCPtreatable conditions because of difficulty obtaining timely appointments. Pre-implementation survey responses indicated appropriate practice  management in support of PCP access, yet PCPs with worsening trends were identified for this pilot. While the reasons for these discrepancies are beyond the scope of this study, possible causes include poor understanding of the office management techniques stated to be in place, good understanding but a less than enthusiastic implementation, or simply answering what was considered an appropriate response when interviewed by the IPA with which they were affiliated.

The OSP implementation methodology (Table 1) included regular practice contact, encouragement, and follow-up to maximize PCP practice focus and attention in regard to the pilot materials. How the intervention was implemented by OSP, and its well-established relationship with these practices, is likely a vital factor. Concepts alone, such as open-access scheduling, do not guarantee results.10,11 All sites claim to have embraced such practices before the pilot and, according to the survey responses, very little changed as a result of the pilot. Thus, it is not sufficient to merely have knowledge of methods for improving PCP access, but rather it is necessary to have the desire, or requirement, to also implement them well. The motivation could be financial (eg, a PCP financial risk arrangement), or could be related to it being made a priority by a larger organization with which the PCP is affiliated.

Clear direction, support, and follow-up from a larger organization with which a PCP is affiliated (eg, IPA or accountable care organization [ACO]) may be necessary to achieve more consistent and effective use of these basic techniques within primary care practices. A focus on the basics can be lost, or perhaps never properly understood, due to the constant clarion call of “innovation” from one stakeholder to another. Large health plan customers want to see something done to produce “new” savings, and an IPA or ACO may want to impress a health plan to negotiate a better contractual arrangement. Policy makers,  almost by definition, must be advocating changing something to support their political viability. At the individual practice level, physicians can only absorb and implement so much. While innovation is meant to push out the old in favor of newer and ostensibly more effective solutions, assessing whether thorough and proper implementation of the older and established methods has occurred should not be overlooked. The concept of thorough comparative effectiveness evaluations in healthcare should apply not only to therapeutic options but to the administrative aspects as well.

Simple office management techniques should be revisited and given focused attention by primary care physicians to limit emergency department visits for primary care“treatable conditions.

May 2013

A Predictive Model of Hospitalization Risk Among Disabled Medicaid Enrollees

Pragmatic Clinical Trials: US Payers' Views on Their Value

Effects of Integrated Delivery System on Cost and Quality

Emergency Department Use Associated With Primary Care Office Management

Well-Child Care Visits and Risk of Ambulatory Care-Sensitive Hospitalizations

Medicare Prescription Drug Plans as Perceived by Public Health Providers

Variability in Resource Use: Diagnosing Colorectal Cancer

Patient Experience Over Time in Patient-Centered Medical Homes

Differences in the Clinical Recognition of Depression in Diabetes Patients: The Diabetes Study of Northern California (DISTANCE)

Can Cancer Patients Seeking a Second Opinion Get Better Care?

Medical Costs Associated With Type 2 Diabetes Complications and Comorbidities