Countywide Physician Organization Learning Collaborative and Changes in Hospitalization Rates
Brent D. Fulton, PhD, MBA; Susan L. Ivey, MD, MHSA; Hector P. Rodriguez, PhD, MPH; and Stephen M. Shortell, PhD, MPH, MBA
Cardiovascular disease is the leading cause of death in the United States, and its annual costs, combined with those of stroke, averaged $316.6 billion per year in 2011 and 2012, including $193.1 billion in direct healthcare costs and $123.5 billion in indirect costs.1-3
Achieving lower heart attack and stroke rates will not only save lives, but also generate substantial healthcare cost savings.
Quality improvement collaboratives have become a well-established approach for improving the quality of care in the United States.4,5
Several national and regional models have emerged to reduce cardiovascular and cerebrovascular diseases and their risk factors. Million Hearts, for instance, an initiative launched by HHS in 2011, aims to prevent 1 million heart attacks and strokes over 5 years by convening and educating communities, physician groups, federal agencies, and other partners.6
Regional models that aim to reduce cardiovascular and cerebrovascular risk factors include physician collaboratives focused on education, training, and data sharing. For example, South Carolina’s Hypertension Initiative, which spread to 4 states in the stroke belt, contributed to a 43% decline in coronary heart disease deaths and a 42% decline in stroke deaths between 1996 and 2006.7
Another collaborative in Ontario, Canada, promoted education in 11 primary care clinical practices, which resulted in reductions in patients’ systolic blood pressure and 10-year cardiovascular risk scores during the 3-year study period.8
Finally, a community-wide cardiovascular disease prevention collaboration in rural Franklin County, Maine, was associated with lower hospitalization and mortality rates over a 40-year period.9
University of Best Practices
The California Right Care Initiative (RCI) began in 2007, focusing on evidence-based and outcomes-improvement strategies to reduce preventable morbidity and mortality among Californians, particularly those with cardiovascular and/or cerebrovascular disease.10
One of RCI’s main strategies is to stimulate the sharing of best practices for managing risk factors (including diabetes) for these diseases among physician organizations, integrated delivery systems, and other stakeholders through regional inter-organizational learning communities known as University of Best Practices (UBP).11
The San Diego County UBP (now called “Be There San Diego”) is the longest running RCI learning collaborative. San Diego County’s efforts began with a National Institutes of Health Grand Opportunity grant in October 2009, in which the California Department of Managed Health Care and physician leaders began to plan for the learning collaborative. The UBP meetings began in February 2011, and the monthly learning collaborative is attended by medical, pharmacy, and quality improvement directors from all of the major physician organizations, integrated delivery systems, and community clinics in the county. The attendees represent organizations that serve approximately 80% of San Diego County patients, including those who are commercially insured, Medicare, Medi-Cal, safety net, US Navy, and US Department of Veterans Affairs. Each meeting is devoted to presentations and discussions of better ways to care for patients with cardiovascular and cerebrovascular disease risk factors (eg, high blood pressure, high lipid levels, high blood sugar levels, diabetes, obesity, and smoking) through patient activation methods, healthcare team coordination, and uptake of medication protocols. Each organization reports and shares its progress on these measures because controlling these intermediate outcomes prevents unnecessary heart attacks and strokes.12-15
The objective of this study is to examine whether the California RCI’s UBP in San Diego County was associated with fewer hospitalizations for acute myocardial infarctions (hereafter “heart attacks”) and cerebrovascular events (hereafter “strokes”). Learning collaboratives to improve coordination and quality of care are becoming more prevalent, as evidenced by recent initiatives including the CMS State Innovation Model Initiative, CMS’s Transforming Clinical Practice Initiative, the Medicaid program, and the Children’s Health Insurance Program.16-20
In spite of large-scale initiatives using regional learning collaboratives, no study has examined the impact of a physician organization learning collaborative in the context of a highly competitive managed care market that is attempting to collaborate on shared community-wide goals to better manage cardiovascular and cerebrovascular risk factors.
Inpatient hospitalization data are from California Office of Statewide Health Planning and Development’s (OSHPD) nonpublic Patient Discharge Data for 2007 to 2014 for adults 18 years or older. OSHPD collects these data from all hospitals in California. To identify relevant hospitalizations, we examined the principal discharge diagnosis code, which was based on the International Classification of Diseases, Ninth Revision, Clinical Modification
). We used code 410 to capture acute myocardial infarction (heart attack)21,22
and codes 430 to 438 to capture strokes.23
Hospitalizations were converted to age-adjusted rates, based on California’s 2014 population,24,25
using the direct standardization method for those aged 18 to 64 years and those 65 years or older.26
During the 2007 to 2014 study period, there were 25,287,552 hospitalizations in California, including 437,774 for heart attacks and 667,776 for strokes in acute care hospitals. To avoid counting multiple hospitalizations for the same event, we excluded hospitalizations when a patient was discharged to the admitting or another hospital for acute inpatient care. These transfers likely occurred because the patient needed care at a higher-acuity hospital. This reduced the number of hospitalizations to 372,205 (–15.0%) for heart attacks and 642,455 (–3.8%) for strokes.
To estimate whether the California RCI’s UBP in San Diego County was associated with changes in hospitalization rates for heart attacks and strokes, we estimated difference-in-differences (DID) models, which are used in quasi-experimental research designs.27,28
A DID model reduces the potential for bias by controlling for baseline differences in hospitalization rates between San Diego County and the rest of California, and by controlling for hospitalization rate reductions that occurred in the whole state during this entire period. Equation 1:
|| ) = β0+β1SDc+β2year11_14t+β3SDc×year11_14t+β4Xc,t
shows the logistic regression model used to estimate the DID models, where Hosp
is 1 for a hospitalization for individual (i
) in county (c
) in year (t
) and is 0 for no hospitalization; SD
is 1 for a hospitalization in San Diego County and is 0 for a hospitalization in other counties; year11_14
is 1 for years 2011 to 2014 and is 0 for years 2007 to 2010; SD
is the DID interaction term; and Xc,t
are time-varying, county-level control variables from the California Health Interview Survey,29
including the percentage of the adult population that did not have health insurance, the percentage that did not have a usual place to go when sick or needing health advice, and the percentage that had an income from 0% to 199% of the federal poverty level because these factors are associated with the ability to pay for healthcare services and access to care. The DID interaction term compares how the odds of a hospitalization changed in the UBP period (2011-2014) versus the pre-UBP period (2007-2010) for San Diego County residents versus the residents in the rest of California. Separate models were estimated for heart attack and stroke hospitalizations.
In addition to the UBP in San Diego County, Sacramento and Los Angeles counties began UBPs in July 2012 and April 2013, respectively; however, they have far less physician organization participation and lower fidelity of implementation of learning collaborative models.4
As a sensitivity analysis, we re-estimated the DID models by excluding these counties.
We also analyzed pre-UBP hospitalization trends for San Diego County and the rest of California, because a DID model assumes parallel pretreatment trends, which means that absent the UBP intervention, the hospitalization rates in San Diego County and the rest of California are expected to change at the same rate.28
As a supplemental analysis, we extended the pre-UBP analysis by examining pre-UBP versus UBP period hospitalization trends in San Diego County and the rest of California using a piecewise linear regression difference-in-differences-in-differences (DIDID) model. The eAppendix
describes these methods in detail (eAppendix available at ajmc.com
Our data only include individuals who were hospitalized. Therefore, we estimated the logistic regression models using Stata version 12 (StataCorp; College Station, Texas) blogit
command that implicitly includes individuals who were not hospitalized, because one variable within the blogit
command is the population, which we obtained from the California Department of Finance.24,25
The number of individuals who were not hospitalized equals the population minus the number of hospitalizations from individuals who were hospitalized, which allows for more than 1 hospitalization in a year for a separate medical event for an individual (eg, ICD-9-CM
code 412 is for previous heart attacks, which we did not include). The other key variable within the blogit
command is the age-adjusted number of hospitalizations, which we calculated by multiplying the aged-adjusted rate by the population for each county(s)-year. The blogit
command produces the same results as if we had estimated our models as logistic regressions with individual-level data that include both hospitalized and nonhospitalized individuals.
Our blogit model estimation approach is similar to 2 studies that analyzed emergency department (ED) visits, which explicitly included individuals who were not present in the ED visit data.30,31
We did not explicitly include individuals who were not present in our hospitalization data because we age-adjusted our hospitalization rates. Moreover, San Diego County did not experience gender and age demographic changes between the 2007-to-2010 period and the 2011-to-2014 period that substantively differed from changes in the rest of California.
To quantify the magnitude of our heart attack results from equation 1, we calculated how many additional hospitalizations would have occurred in San Diego County if there was no association between UBP and age-adjusted hospitalization rates. Furthermore, we calculated how many fewer hospitalizations would have needed to not occur in the rest of California if the same association between UBP and San Diego County’s age-adjusted hospitalization rates was also found in the rest of California. To translate hospitalizations into dollars, we assumed a hospitalization cost a payer $22,427 in 2014 inflation-adjusted dollars.32
Our study was approved by the California Health and Human Services Agency’s and University of California, Berkeley’s institutional review boards.
shows the age-adjusted number of hospitalizations per 100,000 adult population for heart attacks and strokes in San Diego County versus the rest of California between 2007 and 2014 (see eAppendix Table 1
for the underlying data). During this period, the rate for heart attacks decreased sharply in San Diego County from 180.8 to 137.4 (24.0%), but the decrease was more moderate in the rest of California, from 185.5 to 160.8 (13.3%). For strokes, the rate decreased in San Diego County from 310.1 to 277.6 (10.5%), but the percentage decrease was larger in the rest of California, from 323.6 to 269.5 (16.7%). During the UBP period, the rate for heart attacks decreased more in San Diego County (23.5%) than in the rest of California (7.6%), but decreases for strokes were similar in San Diego County (13.8%) and the rest of California (12.3%).
shows the age-adjusted number of hospitalizations per 100,000 adult population for heart attacks by gender, which shows the rate for males is approximately twice the rate of females, but both genders experienced similar decreases during the UBP period.
shows the DID logistic regression results for heart attacks (models A1-A3) and strokes (models A4-A6). The top portion of the heart attack and stroke sections show the summary DID results based on the logistic regression models: they report each odds ratio (OR) that makes up the DID parameter, which is a ratio of 2 ORs. The summary result from equation 1 for San Diego County is β2
, for the rest of California is β2
, and for the DID ratio of the ORs is β3
. Below the summary DID results, the table reports the regression parameter estimates for each model.
In the UBP versus pre-UBP period, the odds of adults being hospitalized for a heart attack in San Diego County decreased (OR, 0.84), while the odds stayed the same for adults in the rest of California (OR, 1.00): DID ratio of OR, 0.84 (P
<.001) (model 1). The results were substantively the same for males and females (models 2 and 3, respectively). In the UBP period versus pre-UBP period, the odds of adults being hospitalized for a stroke in San Diego County decreased (OR, 0.95) slightly more than for adults in the rest of California (OR, 0.97), but the ratio of the ORs was not significant at the 0.05 level (OR, 0.98; P
= .08) (model 4). The stroke results were sensitive to inclusion of the control variables, whereas the heart attack results were not. For example, if the percentage of the adult population that reports no usual place of care is removed from the stroke model 4, then the DID ratio of OR increases to above 1.00 (DID ratio of OR, 1.05; P
<.001). It is important to control for this covariate, but the significant change from its exclusion makes the original result less robust.
Recall that a DID model assumes parallel pretreatment trends, which means that absent the UBP intervention, the hospitalization rates in San Diego County and the rest of California are expected to change at the same rate.28
Compared with the rest of California during the pre-UBP period, the San Diego County hospitalization trend approached being higher for heart attacks and was higher for strokes (see eAppendix Table 3
). Assuming the phenomenon driving the nonparallel trends continued into the UBP period and assuming UBP was associated with lower hospitalization rates, then San Diego County’s pre-UBP higher hospitalization trend would cause the DID results to be understated. This may be the case for heart attacks; however, for strokes, the pre-UBP trend differences are primarily because of San Diego County’s age-adjusted hospitalization rate in 2010 being an outlier. Its rate increased by 5.1% from 2009 to 2010, while the rest of California’s rate decreased by 0.3%. Therefore, it is difficult to know whether San Diego County’s 3.7% rate decrease in 2011 was because of UBP (ie, UBP reversed the pre-UBP trend difference) or if the 2011 decrease represented a regression to the mean. However, the DIDID results in eAppendix Table 4
do not provide evidence to support that UBP was associated with a lower hospitalization rate trend for strokes during the UBP period: San Diego County’s UBP hospitalization trend (OR, 0.954; P
<.001) was similar to the rest of California’s (OR, 0.959; P
<.001), resulting in a UBP DID trend of 0.995 (P
= .51) (see Model A10 in eAppendix Table 4).
Because of the difficulty of interpreting the DID parameter, a ratio of ORs, Table 2
presents the magnitude of the heart attack results from Table 1. From 2011 to 2014, there were 13,849 hospitalizations for heart attacks in San Diego County. Based on the logistic regression results in model 1, the difference between San Diego County’s and the rest of California’s age-adjusted hospitalization rate for heart attacks from 2007 to 2010 was 1.5 hospitalizations per 100,000 adults on average. If this difference had continued from 2011 to 2014, then there would have been 16,584 hospitalizations in San Diego County from 2011 to 2014. Therefore, this translates into the UBP being associated with 2735 hospitalizations being avoided in San Diego County (or 16.5% of the 16,584 potential) that would have cost payers $61 million (2014 dollars) in hospitalization costs from 2011 to 2014.
From 2011 to 2014, there were 174,195 hospitalizations for heart attacks in the rest of California (Table 2). Again, if the above rate difference of 1.5 had continued from 2011 to 2014, meaning that the rest of California experienced the UBP association with fewer hospitalizations for heart attacks, then there would have been only 153,994 hospitalizations in the rest of California from 2011 to 2014, a difference of 20,201 (11.6%) fewer hospitalizations at a cost of $453 million (2014 dollars).
Sacramento and Los Angeles counties began less-robust UBPs in July 2012 and April 2013, respectively. If these UBPs were associated with fewer hospitalizations, then they would have attenuated our original results because they were included in the control group. When the DID regression model in equation 1 was re-estimated excluding these counties, the heart attack result did not substantively change (model 1 re-estimate: DID ratio of OR, 0.85; P
<.001), but the stroke result actually became weaker (model 4 re-estimate: DID ratio of OR, 1.00; P
= .79), contributing to the original stroke result’s lack of robustness.
Furthermore, we conducted a falsification test and re-estimated our DID regression model in equation 1 for heart attacks (model 1 in Table 1) by treating the 8 counties with populations of more than 750,000 adults in 2014 as though they received the UBP treatment beginning in 2011. In summary, we did not find consistent evidence that any of these 8 counties experienced what San Diego County did. Although Alameda and Orange counties’ DID results were lower or similar to San Diego County’s, those results were partially caused by their pre-2011 lower hospitalization rate trend compared with the rest of California (see eAppendix for details).
The California RCI’s UBP in San Diego County, a countywide physician organization learning collaborative, was associated with a lower level of age-adjusted adult hospitalization rates for heart attacks compared with the rest of California. The UBP was associated with 2735 (16.5%) fewer hospitalizations for heart attacks, totaling $61 million (2014 dollars). If the rest of California had experienced the same reduction in hospitalizations for heart attacks associated with UBP, then there would have been 20,201 (11.6%) fewer hospitalizations at a cost of $453 million (2014 dollars). This result is consistent with other regional collaborative efforts to improve hypertension care7-9
and inter-organizational learning activities being associated with performance improvement.33
However, a similar robust association between UBP and hospitalization rates for strokes was not found.
Our findings extend the current literature by providing evidence that physician organizations operating in the context of a highly competitive managed care market can collaborate and exchange best practices to achieve shared community-wide goals to decrease hospitalizations for heart attacks. Although there is limited evidence about the relative impact of various learning collaborative components, recent research has identified the core components of learning collaboratives that participants value most.4
Collaborative faculty, solicitation of staff ideas, change package, Plan-Do-Study-Act cycles, facilitated learning sessions, and a collaborative extranet have been identified as collaborative components that provide participants with motivation, social support, and project management skills.4
These components leverage intrinsic motivation among physicians for quality improvement.34
The UBP collaborative incorporated these core components directly or with modification. Our findings suggest that a combination of components may be needed in collaboratives to achieve desired performance outcomes as opposed to any single component. Future research should attempt to identify the key combinations of components that have the greatest impact for a particular context. For example, the UBP’s emphasis on the use of collaborative faculty (including a cardiology expert) and facilitated learning sessions warrant further study to determine if these 2 components are among the most important for achieving successful outcomes.4
Alternative Explanations and Limitations
Although a quasi-experimental research design using DID models is a strong design, results could be biased if another intervention or phenomenon occurred contemporaneously with the UBP that was also associated with hospitalization rates in San Diego County and/or the rest of California. Kaiser Permanente has started a number of statewide initiatives in California to improve cardiovascular and cerebrovascular care, such as the Prevent Heart Attacks and Strokes Everyday (“PHASE”) program and the Aspirin, Lisinopril and Lipid-Lowering Medication (“ALL”) initiative; however, we do not think these would bias our results because they began well before UBP’s start in 2011.35-38
Changes in the share of patients who experienced a heart attack, but did not survive to be admitted into the hospital, could also bias our results, but there is no reason to think this change would have occurred disproportionately in San Diego County. Also, stroke hospitalizations were more difficult to analyze because San Diego County’s 2010 age-adjusted hospitalization rate for strokes was a high outlier: its rate increased by 5.1% that year, just before the start of UBP, whereas the rest of California’s rate decreased by 0.3%. This could be one reason why the stroke findings were not significant and sensitive to different model specifications. Finally, we do not think hospital closures significantly contributed to lower hospitalization rates in San Diego County, because among the 16 hospitals in the county in 2007, only Fallbrook Hospital, a small, 47-bed hospital, closed during the study period, but not until November 2014. On the other hand, Palomar Medical Center, a 288-bed hospital, opened in August 2012.
Our study results suggest California RCI’s UBP physician organization learning collaborative in San Diego County was significantly associated with—and likely at least a partial cause of—a marked decline in the number of hospitalizations for heart attacks. No robust relationship was found for hospitalizations for strokes. Although our findings are not explicitly generalizable to other physician organization learning collaboratives, they could inform similar existing or new learning collaboratives, whose numbers are increasing.16-20
As the Medicare Access and CHIP Reauthorization Act expands value-based payment arrangements, physicians and their affiliated hospitals will have greater incentives to test different learning collaborative models to reduce hospitalizations via prevention and disease management strategies. Thus, efforts to extend learning collaborative models to other counties in California and elsewhere in the United States become especially important.