David C. Miller, MD, MPH

Although hospitals face increasing pressure from payers to improve the efficiency of healthcare delivery beyond the index hospitalization, they often lack information on postdischarge events. The Michigan Value Collaborative (MVC) developed a claims-based algorithm to provide hospitals with data on events that occur to patients beyond the hospitalization. Herein, we discuss the validation of MVC’s claims-based algorithm. 

Retrospective analysis of a claims-based algorithm’s ability to identify specific medical events, such as index hospitalizations, 30-day readmissions, emergency department visits, skilled nursing facility admissions, home health visits, and rehabilitation services. The claims-based events were validated using a primary review at 63 hospitals. 

 We selected 1830 Blue Cross Blue Shield of Michigan episodes from MVC data and asked 63 Michigan hospitals to query their medical records for the presence or absence of specific events. We then calculated agreement statistics and improved our algorithm using feedback from hospitals. 

 All 63 hospitals participated in the validation process and successfully identified 99% of episodes in their medical records. The initial agreement between our algorithm and medical records was moderate for 4 postdischarge events (kappa ranging from 0.62-0.78) and poor for rehabilitation services (0.16). Much of the disagreements occurred because hospitals could not identify postdischarge events occurring outside of their hospital systems. Other disagreements occurred because of hospital coding practices. Through this analysis, the claims-based algorithm was improved to better reflect real-world coding practice.

 Our findings suggest that the MVC claims-based algorithm identifies and classifies claims with high fidelity and outperforms medical records in the identification of postdischarge events. These findings provide important insight to policy makers, payers, and hospital administrators about the value of claims-based data for the implementation of episode-based programs.

The Michigan Value Collaborative (MVC)’s claims-based algorithm identifies postdischarge events with high fidelity, often outperforming medical records.

The creation of this algorithm and its subsequent validation demonstrates that deriving episode-level utilization from administrative claims is achievable. 

With an increasing focus on improving efficiency of care after the index hospitalization, hospitals need accurate information. To this end, MVC’s claims-based algorithm provides precise episode-level data on postdischarge care. 

Given that administrative claims data outperformed medical records, hospitals should seek to collect such data, possibly by participating in a statewide, regional, or health-system value improvement collaborative or asking payers for claims-based analytic data.

Hospitals are increasingly being held accountable for services and expenditures that occur beyond the hospitalization through episode-based performance measures.

 Postdischarge expenditures, such as postacute care and readmissions, have been cited as the fastest growing spending categories over the last 2 decades and have been the target of many national programs focused on reducing healthcare costs.

 For example, CMS recently implemented the Comprehensive Care for Joint Replacement bundled payment program, which will hold hospitals financially accountable for expenditures occurring from admission through 90 days post discharge.

 In addition, accountable care organizations were developed to reduce costs that occur both outside of and during hospitalizations.

 Based on the prevalence and growth of episode-based payment programs, it is evident that many payers believe the key to reducing healthcare expenditures is to hold hospitals responsible for efficiency along the entire patient care episode. 

Despite enthusiasm for increased episode efficiency, identifying specific high-cost events, such as readmissions, can be challenging for hospitals for several reasons.

First, it is difficult for hospitals to track events outside of the initial hospitalization. In fact, hospitals are often not even aware of postdischarge events that occur at outside facilities. Secondly, unless directly affiliated with the hospital, postdischarge providers are not incentivized to report utilization patterns to hospitals. Furthermore, many small hospitals may not have internal resources to monitor and track postdischarge events and spending. 

In Michigan, one response to these challenges was the development of the Michigan Value Collaborative (MVC).

 Established in 2012 and funded by Blue Cross Blue Shield of Michigan (BCBSM), MVC’s mission is to provide hospitals with episode-level data and promote high-quality care at the lowest reasonable costs.

 One particular area of interest has been postacute care, specifically rehabilitation. MVC hospitals have begun to use episode-level data to monitor rehabilitation expenditures, especially for conditions such as acute myocardial infarction (AMI) and hip replacement, where more than one-third of all patients are discharged to rehabilitation facilities (

However, a necessary step in this process is to ensure that MVC’s claims-based identification of postdischarge services is accurate. 

In this context, we describe a large-scale medical records—based validation of the algorithm used by MVC to define clinical episodes of care in commercial claims. We believe that the MVC validation experience will provide useful insight to hospitals and payers about the advantages and limitations of using claims to track events that occur after hospitalization.

The MVC collects claims data for BCBSM beneficiaries admitted to 1 of 63 Michigan hospitals for 21 medical and surgical conditions. We consulted clinical experts who used 

International Classification of Diseases, Ninth Revision

) diagnosis/procedure and Current Procedural Terminology (CPT) codes to define the conditions. Then, we utilized variables such as revenue codes, diagnosis-related groups (DRGs), facility identification, and CPT codes to classify individual claims into: inpatient, skilled nursing facility (SNF), home health, emergency department (ED), inpatient rehabilitation, outpatient rehabilitation, and general outpatient claims. Cases with readmissions to hospitals other than the index facility were excluded from the readmissions validation step. 

) further details the entire attribution process. Table 1 presents the characteristics of episodes identified from claims data by condition type, including use of postdischarge services. This study was deemed exempt from review by the Michigan Institutional Review Board. 

Our validation process occurred in 2 phases (pilot and full validation). During the pilot, 6 hospitals were asked to review 10 to 20 BCBSM preferred provider organization (PPO) cases from between January 1, 2013, and October 31, 2014. For matching purposes, the clinical condition, national provider identifier, date of birth, gender, admission date, and discharge date for each patient was provided. Specifically, participants were instructed to indicate if the patient listed had records demonstrating that a specified event occurred within 90 days of their discharge date.

These events included 30-day readmissions, ED visits, SNF admissions, home health visits, and rehabilitation services (inpatient and outpatient).

The lessons learned from the pilot were used to inform the full validation. Here, we distributed the same key variables to all 63 MVC participants. Each hospital was provided 30 cases to review, with the exception of hospitals that participated in the pilot (which were asked to review 20 cases). We selected conditions based on volume, prevalence of associated postdischarge services, and suggestions from clinical experts. The conditions included colectomy, coronary artery bypass graft, AMI, pneumonia, congestive heart failure, hip replacement, knee replacement, cesarean delivery, vaginal delivery, trauma, and spine surgery. In total, 1830 BCBSM PPO cases were selected for data validation from these 11 conditions. 

We identified areas of agreement and disagreement between MVC’s claims-based algorithm and medical records. First, we looked for agreement that these episodes occurred and were attributed to the correct hospital and to the correct condition. Next, we used a kappa statistic to assess agreement for postdischarge services (

). For each disagreement in a case, 2 members of the MVC team, a clinician and an analyst, reviewed the specific claims. After determining the cause of the discordance, we adjusted our algorithm to better capture hospital events and re-evaluated the level of agreement. 

In the cases where MVC reported a postdischarge event that did not match a hospital’s medical records, we used confirmatory evidence from the claims to improve our confidence that the event occurred. Specifically, we examined the following items: 1) Did the postdischarge event have more than 2 claim line items with dates following the index admission?; 2) Did the place of service designation on the claims support the assignment of the claim to the postdischarge service?; and 3) Did the revenue center code on the claims support the assignment of the claim to the postdischarge service? 

One hundred percent of hospitals (n = 63) participated in the validation process. Hospitals matched 1812 of the 1830 (99%) MVC episodes to records in the hospital’s medical charts. 

The agreement for the occurrence of postdischarge services ranged from 0.16 to 0.78, with rehabilitation services having the poorest agreement. eAppendix B contains details of the validation process. 

Using administrative claims, we were able to identify 183 readmissions for 1812 episodes (10% readmission rate). Of these, there were 15 cases in which MVC observed a 30-day readmission that was not evident in the hospital data (

These were due to readmissions to a hospital different than the index facility and, per the methods section, were excluded. There were 24 discordant readmissions which MVC did not identify and a hospital did; these cases were classified as observation unit stays in MVC data (eAppendix B). 

We identified 452 ED visits (25% of episodes). Of these visits, 292 (65%) were identified by hospitals. Importantly, there were 160 out of 452 cases (35%) in which MVC observed an ED visit that was not reported in the hospital clinical data (Table 2). Of the 160 discordant cases, 122 (76%) of these ED visits occurred at a different facility than the index admission. All 160 cases had confirmatory evidence of a valid ED visit. 

There were 85 episodes (5%) in which a hospital reported an ED visit that was not evident in the MVC data. Upon review, many of these were classified elsewhere. For example, 38 preceded an index admission or a readmission and the ED services are grouped with this hospitalization in the MVC episode (eAppendix B). 

We identified 64 SNF admissions (4% of episodes). Of these, 43 (67%) admissions were identified by hospitals. There were 21 cases (33%) in which MVC identified a SNF admission but the admissions were not reported by hospitals (Table 2). We found evidence in the claims data to confirm that all 21 visits occurred. There were only 9 cases in which hospitals reported care in a SNF, and we found no evidence of a SNF admission in the MVC claims data (eAppendix B). 

We identified 949 home health visits (52% of episodes). Of these visits, 781 (82%) were identified by hospitals (Table 2). Of those reported by hospitals, there were 168 cases (21%) in which MVC observed a home health visit that was not evident in the hospital clinical data. There were 99 cases (10% of episodes) in which a hospital observed a home health visit that was not evident in the MVC data (eAppendix B). 

The algorithm identified 1223 rehabilitation visits (67% of episodes). Of these visits, 350  (29%) were reported by hospitals. There were 873 cases (71%) in which MVC observed a rehab visit that was not evident in the hospital clinical data (Table 2). Of the 873 discordant cases, 851 (97%) had confirmatory evidence in the MVC claims for utilization of rehabilitation services. There were only 42 cases (3% of episodes) in which a hospital observed a rehab visit that was not evident in the MVC data (eAppendix B). 

Improvements to the Claims-Based Algorithm 

After reviewing all cases of discordance between the claims-based algorithm and the medical records, we identified several areas for improvement, the most important being to update the 

 codes used to identify postdischarge claims to better reflect the coding practices used by hospitals. For example, 

 code V57.8 (care involving other specified rehabilitation procedure) was initially not considered by MVC as a related diagnosis code for patients after joint replacement. However, this code was used for a large number of rehabilitation services after joint replacement. After making this and other improvements, we re-evaluated the level of agreement between our algorithm and the medical records and found improvement in agreement for all services (eAppendix B). 

In this study, we validated the MVC’s claims-based algorithm for the identification and classification of postdischarge events. During the process, we found a high episode match rate between MVC data and medical records. Much of the disagreement was due to the inability of hospitals to identify readmissions, rehabilitation services, and other postdischarge events. Collectively, these findings suggest that this claims-based algorithm outperforms medical records in identifying postdischarge events. 

Previous investigators have convincingly demonstrated that variation in episode spending is largely due to postdischarge events.

 Preventable readmissions and variations in postacute care could indicate areas to improve hospital efficiency and outcomes. Others have also validated and demonstrated on a health-system level the success of claims-based algorithms in identifying hospital events.

 Existing literature has demonstrated the effectiveness of utilizing these tools to identify high-cost inpatient events and improve the value of care.

 It is reasonable to believe that by using this same approach to identify postdischarge events, institutions could potentially achieve similar results outside the hospital setting. The current study findings demonstrate, on a large statewide scale, that accurately identifying and measuring postdischarge utilization may be difficult for hospitals using medical records alone.

A claims-based algorithm could better identify postdischarge events, especially those that occur outside hospitals’ networks. With the current national focus on episode efficiency, identification of these events is imperative to driving high-value care. 

Our study has several limitations. First, we only included BCBSM patients in our validation process. Due to CMS privacy restrictions, we were unable to validate our algorithm with Medicare beneficiaries. Second, the MVC algorithm may not be generalizable for other commercial payers, although BCBSM is the largest commercial payer in Michigan. Third, we did not have hospitals look for readmissions that occurred outside of the hospital where the index event occurred. This decision was primarily made to reduce the chart review burden to hospitals; we received early feedback that hospitals could not identify these particular events. Finally, we did not validate our classification algorithm for other postdischarge events (eg, outpatient procedures) and intensity of services (eg, SNF length of stay). However, this study was focused on validating the occurrence of major postdischarge services. Although we did not end with perfect agreement between the MVC data and the medical records, there were only a few events identified by hospitals not seen in MVC claims (0.5%-4%). 

Our findings will help stakeholders understand the opportunities and challenges of using a claims-based algorithm to measure episode spending. Relevant to hospital administrators, the finding that the claims-based algorithm used in this study outperformed medical records suggests that such data provide more complete intelligence about the postdischarge period. This finding should encourage hospital administrators to obtain additional claims data by participating in a statewide, regional, or health-system collaboration and by asking payers to share these data. Without these claims data, hospitals will be limited in their ability to measure and optimize services provided outside of their facilities. This is particularly important as CMS and commercial payers are increasingly using episode-based performance measurement and payment bundling. 

Moving forward, research in this area should focus on how these data can be refined to provide more granular information to hospitals. For instance, providing hospitals with data on the average length of stay and intensity of services provided at SNFs may help providers understand the efficiency of facilities where patients are sent after discharge. Ultimately, the value of episode-based performance measurement and bundled payment programs as mechanisms to drive high-value care will strongly depend on the accurate measurement of episode-level payments and utilization.&ensp;

The authors thank Dr Vinita Bahl (University of Michigan), Kelly Rice (Beaumont Health System), Steve Lewis (St. Joseph Mercy Health System), Meghan Coughlin (MidMichigan Medical Center), and John Robertson (Hillsdale Community Health Center) for their comments and insight during the course of this research. They would also like to thank Dr David Share, Ellen Ward, Tom Leyden, and the Value Partnerships at BCBSM for their ongoing support of the Michigan Value Collaborative and this manuscript.

 Institute for Healthcare Policy and Innovation (CE, DCM, JMD), and Dow Division of Health Services Research, Department of Urology (CE, JDS, DCM, JMD), and Michigan Value Collaborative (CE, JDS, BV, VG, DCM, JMD), University of Michigan, Ann Arbor, MI.

 This research was supported by the Agency for Healthcare Research and Quality (1F32HS024193-01 to Dr Ellimoottil). Dr Miller receives salary support from Blue Cross Blue Shield of Michigan for his role as the director of the Michigan Urological Surgery Improvement Collaborative and the Michigan Value Collaborative. Dr Dupree receives salary support from Blue Cross Blue Shield of Michigan for his role as the co-director of the Michigan Value Collaborative and his involvement in the Michigan Urological Surgery Improvement Collaborative. Mr Voit receives salary support from the Michigan Value Collaborative as an account manager and is an employee of ArborMetrix.

 The authors are employed by University of Michigan, which has a contract from Blue Cross Blue Shield of Michigan to operate the Michigan Value Collaborative. 

Articles

The Michigan Value Collaborative has created a claims-based algorithm that categorizes claims into episode components. This manuscript describes the validation of this algorithm.

Validation of a Claims-Based Algorithm to Characterize Episodes of Care

 Prostate cancer treatment is a significant source of morbidity and healthcare spending. Evolving clinical data have supported expanding surveillance as a means to “right-size” treatment. Integrated delivery systems afford the possibility of hastening this objective.

 Retrospective cohort study of Medicare beneficiaries.

 We used a 20% sample of national Medicare claims to assess the impact of healthcare integration on rates of treatment and potential overtreatment in men newly diagnosed with prostate cancer between 2007 and 2011. Rates were measured according to the extent of integration within a market (ie, none, low, intermediate, and high). Generalized estimating equations were used to assess the relationship between integration and utilization, adjusting for confounders.

Rates of treatment declined across all markets (

 <.01 for overall time trend), but the rate of decline was similar for the 4 market types (

 = .27). In the most integrated markets, the rate decreased by 28.8%, or from 55.5 per 10,000 population in 2007 to 39.5 per 10,000 in 2011. After adjusting for confounders, men residing in the most integrated markets were 2.1% less likely to be treated with curative intent compared with those living in areas without integrated delivery systems (

 = .04). However, rates of potential overtreatment were similar across all markets regardless of the level of integration (

Healthcare integration was associated with small declines in prostate cancer treatment in newly diagnosed men, but not with potential overtreatment. Integrated care alone may be insufficient to curtail potential overtreatment of prostate cancer.

		Integration was associated with a small decrease in the treatment of prostate cancer. 

		Integration was not associated with treatment among men least likely to benefit (ie, those with a high risk of noncancer mortality). 

		Integration alone may be insufficient to promote optimal care for prostate cancer. 

		Financial risk, implied by evolving delivery system reforms, may help to promote better stewardship of preference-sensitive diseases, such as prostate cancer.

Prostate cancer is among the most common malignancies in men in the United States.

 Ongoing uncertainties about how best to treat the disease, coupled with the availability of multiple options, have led to wide variations in both the quantity and quality of care.

 Prostate cancer spending has increased by 11% annually over the last decade, outpacing rates for other common conditions (such as cardiovascular and pulmonary diseases) and resulting in $12 billion in yearly expenditures.

 Although the merits of screening are a subject of ongoing debate in the field, consensus is growing that some newly diagnosed men with prostate cancer stand to gain little from treatment.

Improving the efficiency of the delivery system and eliminating wasteful spending have long been priorities for payers and policy makers, and many hope that accountable care organizations (ACOs) and related components of healthcare reform will do just that. By encouraging closer alignment between hospitals and caregivers, ACOs aim to focus on improving quality and cutting costs—both of which may affect prostate cancer care. To a large extent, ACOs are extensions of integrated delivery systems that, due to their emphasis on evidence-based medicine and minimizing unnecessary healthcare, are associated with providing higher quality.

 Thus, understanding the implications of integrated delivery systems for prostate cancer care will help us to anticipate the likely effect of evolving reforms of the Affordable Care Act.

For this reason, we performed a national study to determine the impact of healthcare integration on the management of prostate cancer. We hypothesized that the most integrated markets would be more selective in the use of curative treatment for prostate cancer, particularly among those unlikely to benefit from intervention.

Using a 20% sample of Medicare claims, we performed a retrospective cohort study of fee-for-service beneficiaries newly diagnosed with prostate cancer between January 1, 2007, and December 31, 2011. We limited our study to men continuously enrolled in Parts A and B for at least the 12-month periods prior to and after the prostate cancer diagnosis. To ensure that we had complete claims on all patients, we excluded patients in risk-bearing Medicare managed care plans. Patients were followed through December 31, 2012.

To identify incident prostate cancer cases in national Medicare claims, we developed an algorithm and validated it using Surveillance Epidemiology and End Results (SEER) cancer registry data. Briefly, we used a 5% sample of Medicare beneficiaries residing in a catchment area of a SEER registry in the years 2003 to 2005. We selected men with at least 2 “Evaluation and Management” visit codes in which the line diagnosis 

International Classification of Diseases, Ninth Revision, Clinical Modification 

) code was 185 for prostate cancer. We further required that all incident cases underwent prostate biopsy within 180 days of the first visit code associated with a prostate cancer diagnosis. Men with any claim in the preceding 12-month period that was associated with an 

 diagnosis code of 185 (prostate cancer) or V10.46 (personal history of malignant neoplasm of prostate) were excluded. Finally, we validated this approach against the Patient Entitlement Denominator Summary File, which identifies all incident cases in SEER regions, and found our algorithm to have specificity and positive predictive values of 99.8% and 88.7%, respectively. We then implemented this algorithm in our 20% national sample of Medicare claims to identify incident cases, which compose our study population.

We used hospital referral regions (HRRs), as described by the Dartmouth Atlas,

 to reflect distinct healthcare markets. There are 306 HRRs in the United States, each of which represents a collection of zip codes in which Medicare patients receive the bulk of their healthcare. We determined each market’s level of integration by measuring the proportion of hospital discharges occurring from an integrated delivery system, which were identified from public reports based on data from IMS Health

 and have been used in a similar context.

 These data provide information on delivery system relationships, including affiliations between hospitals and physician practices. Each health system is rated for 33 attributes in 8 domains: overall integration, integrated technology, hospital utilization, financial stability, services, access, contract capabilities, and physicians. Domain-specific scores are summed to yield an overall score for the delivery system, with higher scores reflecting a greater degree of integration.

All HRRs were characterized by the proportion of hospital discharges occurring from a top-100 integrated delivery system.

 Of these, 127 HRRs had no discharges from a top integrated delivery system. The remaining 179 HRRs were sorted into 3 equal groups (ie, terciles) ranging from least integrated (a mean of 14% of discharges from an integrated delivery system) to most (a mean of 71% of discharges from an integrated delivery system). For the purpose of analysis, our market level of integration exposure was treated as a categorical variable with 4 levels: none, low, intermediate, and high.

Because our measure of integration relies on public reports of integrated delivery systems and does not necessarily distinguish between clinical and/or financial integration, it is possible that it may be insensitive to real differences in quality. For this reason, we assessed whether this measure was able to identify differences in publicly available measures

 of quality—including patients with diabetes undergoing glycated hemoglobin (A1C) testing, eye examinations, and blood lipid testing&mdash;between 2008 and 2011. Generally, we observed a monotonic relationship between the level of integration and adherence to each measure. In all instances, the most integrated markets performed better than those without integration. For instance, beneficiaries in highly integrated markets were 21% more likely to undergo A1C testing than were those residing in markets without a top-100 integrated delivery system. These data support that our exposure can detect measurable differences in quality in circumstances in which clinical integration is important.

Our primary outcome was the rate of curative treatment (ie, surgery, external beam radiation therapy, brachytherapy, and cryotherapy) within 12 months of diagnosis, measured at the HRR level. For this calculation, the numerator was determined by the annual count of newly diagnosed patients undergoing any of the aforementioned treatments in a given HRR; the denominator was determined by the eligible male Medicare population residing in an HRR in a given year.

We also measured potential overtreatment. Because of their emphasis on quality and cost containment, integrated delivery systems and, presumably, the markets in which they dominate may be more selective in the services they provide. In accordance with clinical guidelines, we would expect more-integrated markets to have lower rates of treatment among those patients with a high risk of noncancer mortality within 10 years of diagnosis. These patients typically have a low probability of dying from the disease, even absent treatment.

 We assessed treatment among this population of patients (ie, the quartile of men with the highest risk of dying from noncancer causes within 10 years) by implementing methods developed by Gross and colleagues.

 Using a 5% sample of Medicare beneficiaries without cancer, we built a robust patient-level model to predict mortality (C-index = 0.91). This enabled us to estimate the 10-year mortality risk of patients in our prostate cancer cohort absent their cancer diagnosis. Those patients in the top quartile had a 78% risk of non—prostate cancer mortality within 10 years of their diagnosis. Potential overtreatment was assessed at the patient level, and population-based rates were calculated as described above (ie, the numerator was the number of potentially overtreated newly diagnosed patients with prostate cancer, and the denominator was the number of eligible Medicare beneficiaries).

We first contrasted patient and regional characteristics according to the healthcare integration exposure (ie, none, low, intermediate, and high). Statistical inference was made using the 

 tests for continuous data. We then fit a Poisson model with an offset term for the population denominator to assess a trend in population-based rates of curative treatment over time across the integration groups.

To test the independent effect of healthcare integration on rates of prostate cancer treatment, we fit a multivariable logistic regression model using patient-level treatment as the outcome. Our healthcare integration exposure was incorporated into the model at the HRR level. To account for the nested nature of the data (ie, patients within HRRs), generalized estimating equations were used. We adjusted the model for patient-level differences, including age, race, comorbidity, and socioeconomic class. Comorbidity was calculated with patient claims for the 12-month window prior to diagnosis using established methods.

 Socioeconomic class was estimated using a composite measure developed at the zip code level, as described by Diez-Roux and colleagues.

 We used the Area Resource File to derive several market-level variables to include in the model, including measures of social capital (% female head-of-household) and education (% with a high school degree or more), and 2 supply side variables (urologists and hospital beds per capita). We computed adjusted percentages for the use of treatment for each level of our market integration exposure by back-transforming the predicted use from the model. Using similar methods, a separate model was then fit to derive adjusted percentages of treatment among those least likely to benefit. Further, we assessed trends in rates by both curative (ie, surgery, external beam radiation therapy, brachytherapy, cryotherapy) and noncurative (ie, hormone therapy, no treatment) modalities in the population least likely to benefit from treatment.

All analyses were carried out using computerized software SAS version 9.4 (SAS Institute, Cary, North Carolina). All tests were 2-tailed and the probability of type 1 error was set at 0.05. The study protocol was judged to be exempt by the Institutional Review Board at the University of Michigan.

Patient and regional characteristics were contrasted according to the market level of integration (

). Although statistically significant differences among market types were evident for some variables (eg, age), the magnitudes of the absolute differences were small, with one exception. Among markets with at least some discharges from integrated delivery systems, patients in markets with the lowest level of integration were more affluent than those in more integrated ones. For instance, 28.8% of patients in the least integrated markets resided in the highest quartile of socioeconomic class compared with 21.9% of patients in the most integrated ones (

Between 2007 and 2011, population-based rates of curative treatment among men newly diagnosed with prostate cancer declined across all markets (

), regardless of the level of integration (

 <.01 for overall time trend). For example, in the most integrated markets, rates of treatment decreased from 55.5 per 10,000 male beneficiaries in 2007 to 39.5 per 10,000 in 2011, a relative decrease of 28.8%. In markets without integrated delivery systems (ie, no discharges from these facilities), rates of treatment declined by 30.1%, or from 57.2 per 10,000 to 40.0 per 10,000 over the same period. The rate of decline in treatment over time was similar for the 4 market types (

, rates of potential overtreatment—treatment of those patients with the highest probability of death from noncancer causes&mdash;also decreased, albeit to a lesser extent (

 <.01 for overall time trend). In the most integrated markets, curative treatment in this population decreased by 19%, from 10.0 per 10,000 to 8.1 per 10,000. Rates of decline were similar for the less integrated markets (

 = .98). We next explored changes in treatment, by modality, among those least likely to benefit from treatment (

). Rates of radical prostatectomy in these patients were low but remained stable over time (

 = .99). Conversely, rates of brachytherapy, cryotherapy, and external beam radiation therapy decreased during the course of the study (each 

 <.01 for overall time trend), but these declines did not vary with respect to the market level of integration (each 

 >.20). Similar trends were observed for both noncurative modalities.

Finally, we used multivariable modeling to account for the observed subtle differences between patients and healthcare markets (

). Among beneficiaries with newly diagnosed prostate cancer, we found that men residing in the most integrated markets were 2.1% less likely to be treated with curative intent compared with those living in areas without integrated delivery systems (

 = .04). However, rates of potential overtreatment (ie, treatment of men with a high probability of death from noncancer causes) were similar across all markets regardless of the level of integration (

Rates of treatment for prostate cancer among Medicare beneficiaries declined significantly between 2007 and 2011. Similarly, treatment among men with a high risk of noncancer mortality (ie, potential overtreatment) decreased over the same period, albeit to a lesser extent. The use of nonsurgical approaches decreased significantly in men with the highest probability of noncancer death within 10 years, while rates of surgery remained stable in this population. Rates of observation (ie, no treatment) and hormone therapy in this population declined similarly. This supports the theory that secular declines in screening and diagnosis, which are well established,

 underlie the observed trends, as opposed to more selective curative treatment by physicians. Although patients residing in markets with the highest level of integration were less likely to undergo curative treatment for their cancer, the difference in magnitude relative to less integrated markets was small and likely of limited clinical significance. Notwithstanding the advantages of highly integrated markets to deliver better evidence-driven healthcare, we found that the use of treatment among those least likely to benefit did not vary significantly according to the degree of integration in a market.

Although prostate cancer remains a common cause of cancer-related death in the United States, consensus is growing about the potential pitfalls of screening and the indolent nature of some cancers.

 Approaches to management of prostate cancer have evolved considerably over the last decade, and surveillance has increasingly been recognized as a strategy in some men to prevent overtreatment. Despite this recognition, there is little evidence that the use of treatment in men unlikely to benefit is decreasing; rather, population-based data suggest that it is increasing.

 In our study of national Medicare beneficiaries, we noted a significant decline in potential overtreatment across all markets regardless of their level of integration. However, these declines were relatively modest given the nature of the population (ie, having a very high risk of noncancer mortality within 10 years of diagnosis).

That healthcare integration was associated with small declines in treatment, but not with potential overtreatment, is surprising. Indeed, integrated delivery systems are more apt to follow evidence-based practice guidelines, adopt electronic health information systems, and implement strategies for performance improvement; ultimately, they are associated with providing higher-quality healthcare.

 Because of these qualities, we would expect healthcare markets dominated by these systems to be more selective in whom they treat for prostate cancer, particularly among those who are unlikely to benefit.

With recent health reforms toward more accountable care, collections of providers and, in some cases, hospitals, are evolving to become better stewards of population health. To some extent, ACOs are the result of the natural evolution of high-performing integrated delivery systems. However, in addition to sharing a focus on care coordination and other aspects of integrated care, ACOs imply a level of financial risk likely surpassing that which was assumed by integrated delivery systems in the era prior to health reform. Perhaps it is this risk, added to the pressure to realize savings at the beneficiary level, that will promote better stewardship of preference-sensitive conditions in which there are clear tradeoffs with treatment, such as prostate cancer.

One potential limitation of our findings is the absence of measures of disease severity (ie, cancer grade, stage, and prostate-specific antigen levels) in national Medicare claims. Although we classify potential overtreatment using the probability of noncancer death, our findings may underestimate the scope of treatment in the population, as patients with low-risk prostate cancer, regardless of life expectancy, are well-accepted candidates for surveillance.

 Further, the absence of disease severity measures has little implication for our comparison of treatment rates across healthcare markets, as the distribution of cancer grade and stage tends to be similar across geographic regions.

Our findings have important implications for evolving reforms aimed at improving the efficiency of healthcare delivery. We observed a small, albeit significant, association between higher levels of integration and more constrained use of prostate cancer treatment. However, rates of potential overtreatment of men with prostate cancer were similar, regardless of the extent of market integration. Collectively, these findings suggest that integration alone may be insufficient for optimizing the management of conditions such as prostate cancer, in which considerable uncertainty exists about the tradeoffs between treatment and observation. Future research should explore how the added financial risk associated with ACOs is able to modulate the management of preference-sensitive diseases such as prostate cancer.

 Dow Division of Health Services Research, Department of Urology (BKH, MJB, SRK, LH, TAS, DCM) and the Kidney Epidemiology Cost Center (VBS), University of Michigan, Ann Arbor, MI.

 This work was supported by a Research Scholar Grant RSGI-13-323-01-CPHPS to BKH from the American Cancer Society. VBS is supported by funding from the National Cancer Institute (R01 CA168691). DCM is supported by funding from the National Cancer Institute (R01 CA174768).

 Dr Hollenbeck received a grant from the American Cancer Society. Dr Miller received a grant from the National Cancer Institute. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (BKH, DCM, VBS); acquisition of data (BKH); analysis and interpretation of data (MJB, BKH, LH, SRK, TAS, VBS); drafting of the manuscript (BKH, LH, DCM, TAS); critical revision of the manuscript for important intellectual content (MJB, BKH, LH, SRK, DCM, TAS, VBS); statistical analysis (MJB, SRK); obtaining funding (BKH, DCM, VBS); and supervision (SRK).

 Brent K. Hollenbeck, MD, MS, Dow Division of Health Services Research, Department of Urology, University of Michigan, 2800 Plymouth Rd, NCRC Bldg 16, Ann Arbor, MI 48109-2800. E-mail:

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. 

. 2015;65(1):5-29. doi: 10.3322/caac.21254.

2. Cooperberg MR, Broering JM, Carroll PR. Time trends and local variation in primary treatment of localized prostate cancer. 

. 2010;28(7):1117-1123. doi: 10.1200/JCO.2009.26.0133.

3. Spencer BA, Miller DC, Litwin MS, et al. Variations in quality of care for men with early-stage prostate cancer. 

. 2008;26(22):3735-3742. doi: 10.1200/JCO.2007.13.2555.

4. Roehrig C, Miller G, Lake C, Bryant J. National health spending by medical condition, 1996-2005. 

. 2009;28(2):w358-w367. doi: 10.1377/hlthaff.28.2.w358.

5. Mariotto AB, Yabroff KR, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010-2020. 

. 2011;103(2):117-128. doi: 10.1093/jnci/djq495.

6. Klotz L, Vesprini D, Sethukavalan P, et al. Long-term follow-up of a large active surveillance cohort of patients with prostate cancer. 

. 2015;33(3):272-277. doi: 10.1200/JCO.2014.55.1192.

Healthcare integration was associated with small declines in treatment, but no change in overtreatment of prostate cancer. Integrated care delivery alone may be insufficient to curtail overtreatment.

Implications of Evolving Delivery System Reforms for Prostate Cancer Care

Despite broad support for the patient-centered medical home (PCMH), the implications of PCMH implementation efforts that require that participants have some degree of PCMH readiness at baseline are unclear. Therefore, we sought to examine the association among PCMH readiness, quality, and the care of vulnerable patients.

 We conducted a cross-sectional study of adult visits to a nationally representative sample of US office-based primary care physicians in 2007 and 2008.

Using National Committee for Quality Assurance criteria, we determined whether or not a visit occurred at a PCMH-ready practice. We used 

 tests and multiple linear regression to measure the association between PCMH readiness and performance on 9 validated outpatient quality indicators.

Among 12,235 visits to general practitioners and 5123 visits to general internists, 73% occurred at practices that were PCMH-ready. Visits by patients with 3 or more chronic medical conditions were more likely to occur at ready practices (

= .001). Visits by patients that were poor or minority were equally likely to occur at ready and unready practices. Performance at ready practices was higher for 3 of 9 quality indicators related to chronic disease management and preventive counseling (

 = .031 [beta-blocker or diuretic prescribed for hypertension]; 

Implementation efforts that encourage the enrollment of practices most ready for the PCMH could improve the quality of care for complex patients without exacerbating socioeconomic disparities in access to care.

	Despite substantial support for the patient-centered medical home (PCMH), the implications of PCMH implementation efforts that require that participants have some degree of PCMH readiness at baseline are unclear. Our data suggest that PCMH program design that encourages the enrollment of practices most ready for the PCMH could improve the quality of care for complex patients without exacerbating socioeconomic disparities in access to care.

Empirical work provides mixed evidence that implementation of the patient-centered medical home (PCMH) will improve the quality and costs of healthcare. Some demonstration projects have shown that the PCMH is associated with increased use of preventive services,

 and reductions in emergency department visits and/or hospital admissions.

 Evidence that the PCMH positively impacts clinical processes of care and patient outcomes is weaker and more inconsistent.

 However, there is little to suggest that the PCMH will harm patients, and many policy makers hope that the PCMH will eventually improve care. In this context, efforts are under way to disseminate the PCMH,

 including a multi-state pilot initiative from CMS.

While many PCMH programs are voluntary, a substantial number require that participants have a baseline level of medical home readiness.

 Other programs require that practices attain a minimum level of medical home readiness within 12 to 18 months of program entry.

 Such choices about PCMH program design would seem to favor the enrollment of practices that are PCMH-ready (or nearly ready). If there is no differential access to care between PCMH-ready and PCMH-unready practices, then current program design makes sense, as PCMH-ready practices also tend to be multi-specialty practices with substantial visit volume.

 However, if practices that are least ready for the PCMH also disproportionately treat vulnerable populations, then overlooking such practices could initially exacerbate existing disparities. In this case, a purely voluntary approach without any prerequisites for PCMH program enrollment, combined with improvement metrics tailored to a practice’s baseline PCMH readiness, might be preferred.

Since no prior research has investigated the strengths and weaknesses of either strategy, we used data from a nationally representative sample of visits by adults to general practitioners and internists to examine the characteristics of visits to practices stratified by their medical home readiness. We addressed 2 questions: 1) What types of patients are more likely to be seen in visits to practices that are more PCMH-ready? and 2) Is there an association between PCMH readiness and quality of care delivered?

For our study, we used data from the 2007 and 2008 National Ambulatory Medical Care Survey (NAMCS), which is an annual multistage probability sample of outpatient visits to randomly selected, nonfederally employed, office-based physicians in the United States.

 Practices are sampled during a 1-week time frame, so the sample is unlikely to include 2 visits by the same patient. In 2007 and 2008, data were collected from 32,778 and 28,741 office-based visits, respectively. The data files contain practice-, physician-, patient-, and visit-level characteristics. Weighted estimates from them are considered representative of all US outpatient visits. We focused on adult (ie, aged ≥18 years) ambulatory visits occurring at general and internal medicine practices.

We assessed the medical home readiness of each practice where a visit occurred using previously described methods.

 In brief, we mapped practice-level characteristics reported by physicians in the NAMCS to the 2011 National Committee for Quality Assurance (NCQA) PCMH certification standards (see 

 The NCQA assigns a point value to each of the 30 elements (including 6 “must-pass” elements) outlined in the PCMH standards. We mapped 14 elements (including 3 “must-pass” elements), representing 6 of the medical home standards, to items collected in the NAMCS. The 6 PCMH standards were: 1) enhanced access and continuity (8 points); 2) identifying and managing patient populations (16 points); 3) planning and managing care (7 points); 4) providing self-care support and community resources (6 points); 5) tracking and coordinating care (6 points); and 6) measuring and improving performance (13 points).

To calculate a practice’s PCMH readiness score, we used the NCQA scoring system

 to derive a cumulative point total for each practice by summing across all of the passed elements. Based on the 14 measured elements, the maximum point total was 56 when no missing data were present. In the case of missing data, the maximum point total was based on all non-missing data (see 

). The practice-level readiness score was then calculated by dividing the cumulative point total by the total number of available points. The resulting readiness score is expressed as a percentage and can be interpreted as the percentage of measureable NCQA elements that the practice (where the visit occurred) possesses. Based on the NCQA’s levels of medical home recognition, we categorized practices into 2 groups: unready (<35 %) and ready (≥35 %) for medical home implementation.

To measure the quality of care provided at each visit, we first determined whether or not the visit met any of 6 validated medication quality indicators.

 These indicators were: 1) angiotensin-converting-enzyme inhibitor/angiotensin receptor blocker for congestive heart failure (CHF); 2) beta-blocker for CHF; 3) diuretic or beta-blocker for hypertension; 4) beta-blocker for coronary artery disease; 5) statin for hyperlipidemia; and 6) the prescribing of no inappropriate medications during the visit. We coded these indicators using diagnosis and medication codes in NAMCS, as well as responses to questions about the presence or absence of specific chronic diseases. For each visit, the treating physician can list up to 3 diagnoses (primary, secondary, or tertiary) on the patient record form. In addition to diagnoses, the patient record form lists up to 8 original and 8 generic medications prescribed by the treating physician.

In addition to the 6 medication quality indicators, we examined 3 previously validated prevention and counseling quality indicators: diet and exercise counseling (yes/no responses in NAMCS), and blood pressure monitoring at any general medical exam visit.

 The NAMCS asks respondents to enter blood pressure readings (if blood pressure was measured at the visit). In defining a general medical exam, a visit for preventive care was defined as a general medical exam, but a follow-up visit was not considered a general medical exam.

We first described the characteristics of visits occurring at practices that were PCMH-ready versus unready. We used Student’s 

 tests to test for differences in visit characteristics. We also used 

 tests to determine whether the percentage of visits meeting each of the 9 quality-of-care indicators differed between ready and unready practices.

We applied appropriate sampling weights, clusters, and stratification to correct our standard error estimates for the complex survey design. We performed all analyses using Stata version 11.0 (StataCorp LP, College Station, Texas). The University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board determined that this study was exempt from its oversight.

We examined 17,358 visits, representing 341 million adult outpatient visits to general practitioners and internists during the study interval. Nearly three-fourths occurred at practices ready for the PCMH (

). Compared with patients seen at visits to unready practices, patients seen at visits to ready practices were more likely to have 3 or more comorbid conditions (25% of visits to PCMH-ready practices vs 17% of visits to other practices; 

= .001). However, visits to both types of practices were similar with respect to race/ethnicity and socioeconomic status. Practice characteristics of both types of visits were also similar except for physician ownership and rural location, although this latter finding was not statistically significant (

 = .058). In sensitivity analyses with broader categories, 2 findings became statistically significant. When we compared visits in the South with all other practices, visits in the South were less likely to occur at PCMH-ready compared with unready practices (36% vs 52%; 

 = .049). Similarly, when we compared privately insured visits with all other visits, privately insured visits were more likely to occur at PCMH-ready compared with unready practices (57% vs 51%; 

Visits to PCMH-ready practices were generally more likely to meet the medication and counseling/screening quality indicators, although this difference reached statistical significance for only 3 indicators (

). For example, 18% of visits to ready practices included diet counseling, compared with 12% of visits to unready practices (

 = .018). Similarly, exercise counseling occurred at 14% of visits to ready practices, but at only 5% of visits to unready practices (

 <.001). For medication quality indicators, at visits to ready practices (vs unready ones), 41% (vs 32%) of patients were prescribed a diuretic or beta-blocker for hypertension (

Our study had 2 principal findings. First, while visit rates among racial minorities and patients from poorer neighborhoods did not differ based on a practice’s PCMH readiness, those most ready for this new care model have a disproportionate share of patients with multiple comorbid conditions. Second, ready practices, at baseline, delivered higher or equivalent quality care in all dimensions.

Our study must be considered in the context of prior work on the value of health information technology (IT) tools, which the NCQA PCMH standards emphasize heavily. Many of the NAMCS items that we evaluated mapped to the NCQA elements that capture health IT tools. To the extent that the use of health IT improves chronic disease management,

 our finding that visits by patients with multiple chronic conditions are more likely to occur at ready practices is reassuring. Second, evidence is still being gathered to assess whether medical homes deliver better-quality care across multiple domains (eg, patient satisfaction, clinical processes of care, patient outcomes).

 While a fully developed medical home is clearly more than the sum of its parts, our observation of modestly higher quality among visits to ready practices provides 1 piece of evidence to suggest that adoption of PCMH components may be beneficial to care quality.

Our study has several important limitations. First, we ascertained PCMH readiness by matching survey responses about a practice’s infrastructure and processes of care to 1 standard of medical home readiness: the NCQA PCMH standard. However, prior studies have utilized this method,

 and the NCQA standard has been widely adopted and was developed with input from organizations such as the American College of Physicians.

 Second, because the NAMCS questions are not based on the NCQA PCMH standard, we could not perfectly match NCQA elements to NAMCS questions, nor could we match all NCQA elements to an NAMCS question. This may have resulted in some misclassification of ready practices as unready or vice versa, but the directionality of this bias is difficult to ascertain. Prior work has found the NAMCS-NCQA matching approach robust to missing data (see eAppendix 2). Third, our measures of quality were limited by the cross-sectional nature of the NAMCS survey. However, the quality measures have been validated,

 and one of the criteria for constructing these measures with NAMCS data was that the measures have a high likelihood of correlating with improved patient outcomes.

 Nevertheless, we cannot make conclusions about causality between medical home readiness and quality of care. Fourth, it is possible that better scores on exercise and diet counseling measures for PCMH-ready versus unready practices reflect better documentation at PCMH-ready practices, since many of the NCQA measures that we mapped to NAMCS rely on health IT tools.

The characteristics of patients who visit practices that are ready versus unready for the patient-centered medical home differ in important ways.

David C. Miller, MD, MPH

Articles

Validation of a Claims-Based Algorithm to Characterize Episodes of Care

Implications of Evolving Delivery System Reforms for Prostate Cancer Care

The Association Among Medical Home Readiness, Quality, and Care of Vulnerable Patients