Ted A. Skolarus, MD, MPH

 To formally assess the appropriateness of different timings of urethral catheter removal after transurethral prostate resection or ablation. Although urethral catheter placement is routine after this common treatment for benign prostatic hyperplasia (BPH), no guidelines inform duration of catheter use.

Using a standardized, multiround rating process (ie, the RAND/UCLA Appropriateness Methodology), an 11-member multidisciplinary panel reviewed a literature summary and rated clinical scenarios for urethral catheter duration after transurethral prostate surgery for BPH as appropriate (ie, benefits outweigh risks), inappropriate, or of uncertain appropriateness. We examined appropriateness across 4 clinical scenarios (no preexisting catheter, preexisting catheter [including intermittent], difficult catheter placement, significant perforation) and 5 durations (postoperative day [POD] 0, 1, 2, 3-6, or ≥7).

 Urethral catheter removal and first trial of void on POD 1 was rated appropriate for all scenarios except clinically significant perforations. In this case, waiting until POD 3 was deemed the earliest appropriate timing. Waiting 3 or more days to remove the catheter for patients with or without preexisting catheter needs, or for those with difficult catheter placement in the operating room, was rated as inappropriate.

 We defined clinically relevant guidance statements for the appropriateness of urethral catheter duration after transurethral prostate surgery. Given the lack of guidelines and this robust expert panel approach, these ratings may help clinicians and healthcare systems improve the consistency and quality of care for patients undergoing transurethral surgery for BPH.

Given increasing focus on appropriateness of care for quality, payment, and policy, clarifying appropriate urinary catheter duration after transurethral prostate surgery could help improve consistency and quality of care for healthcare organizations. In particular:

Urethral catheter removal and trial of void on postoperative day 1 after the procedure was rated appropriate for all scenarios except clinically significant perforations.

Waiting 3 or more days to remove a catheter for a first voiding trial after these common procedures was inappropriate for the majority of patients.

Both indwelling catheter placement and intermittent catheterization were acceptable approaches to a failed trial of void.

Benign prostatic hyperplasia (BPH) is a leading diagnosis among male Medicare beneficiaries. Approximately 100,000 men are treated with transurethral prostate surgery each year, making it one of the most common surgical procedures in the United States.

 The procedure is performed using various approaches and routinely involves urinary catheter placement. Given an increasing focus on appropriateness of care for policy,

 clarifying appropriate urinary catheter duration after this common surgery could help improve consistency and quality of care for healthcare organizations and their patients treated surgically for BPH.

However, there are no guidelines for the duration of urinary catheter use after transurethral prostate surgery.

 Some providers recommend overnight urinary catheter placement, whereas others recommend leaving the catheter in place for days afterward. Observational studies indicate that catheter removal and trial of void the day after surgery is safe for most patients,

 relieving them of their 1-point restraint

 and associated discomfort sooner rather than later. Decreasing indwelling urinary catheter duration not only reduces patient discomfort and nursing care during the hospitalization and after discharge, but it also lowers the risks of complications, including catheter-associated urinary tract infections (UTIs).

 Although the former might affect patient satisfaction and postsurgical care utilization, the latter is an important quality-of-care metric, especially when catheter use might be scrutinized as inappropriate by national institutions such as the CDC.

In the absence of evidence-based guidelines, defining the most appropriate duration of urinary catheter use after this procedure may help decrease practice variation, reduce postoperative complication risk, and improve consistency and quality of care for patients with BPH and lower urinary tract symptoms.

For these reasons, we assessed the appropriateness of different timings of urinary catheter removal among patients treated with transurethral resection or ablation of the prostate. Following the RAND/UCLA Appropriateness Method,

 we asked a multidisciplinary panel of experts and practicing urologists to review the studies included in our literature search and use their clinical expertise to rate the appropriateness of different options for urinary catheter removal and trial of void after transurethral prostate surgery. This manuscript details and synthesizes findings from this approach in order to provide guidance for and promote standardization of urinary catheter use after this common BPH surgery within and across healthcare organizations.

We used the RAND/UCLA Appropriateness Method to develop these appropriateness criteria.

 We previously used this multidisciplinary, stepped approach to define appropriateness of urinary catheter use in hospitalized medical patients and perioperatively for general and orthopedic surgery patients.

 The methodology couples scientific evidence for a given practice—in this case, urinary catheterization after transurethral prostate surgery&mdash;with clinical judgment to produce clinically relevant guidance statements regarding a procedure’s appropriateness in light of a patient’s symptoms, test results, and medical/surgical history. This robust approach has been used to define appropriate care, and even develop quality indicators, across many clinical scenarios, including coronary revascularization, endoscopic sinus surgery, and active surveillance for prostate cancer.

 This approach has been shown to be useful to provide guidance when more definitive studies are lacking and has been predictive of future randomized controlled study results.

 Finally, managed care and accountable care organizations are increasingly adding metrics involving appropriateness of care (eg, appropriate diagnostic imaging services)

 for quality assessment, and even value-based purchasing programs for their beneficiaries’ providers, making this approach and its findings relevant and timely.

The first step of the RAND/UCLA Appropriateness Method is to conduct a literature review to identify the most relevant articles for a given practice. The literature is divided into categories based on relevance and level of evidence, and common clinical scenarios for appropriateness rating are identified. Similar to prior appropriateness research projects, we began our literature search with a systematic review of databases (Web of Science, CINAHL, Embase, Cochrane, and PubMed/MEDLINE). We searched available literature for studies assessing outcomes for patients undergoing transurethral resection (using monopolar or bipolar technique) or ablation (using plasma vaporization “button procedure” or photoselective vaporization for BPH, including enucleation of the prostate). We searched each database using Boolean logic (eg, AND, OR) for our various combinations of transurethral prostate surgery types. The MeSH system was also searched separately. The literature search and scenario development occurred between September 2014 and February 2015 and included 4428 articles before excluding duplicates across all databases (

). Our study team urologist (T.A.S.) reviewed 472 articles meeting subsequent criteria by abstract, title, keyword, and full text to select the final articles. Forty-four articles met inclusion criteria for our study.

We categorized these articles into 3 groups (A, B, and C) based on their relevance to urinary catheter strategies after transurethral prostate surgery and patient outcomes. Group A (n = 15) articles assessed a particular urinary catheter strategy and its impact on patient outcomes. We expected these articles to be of highest relevance for describing the evidence available to inform appropriateness ratings. Group B (n = 15) studies reported relevant patient outcomes without assessing a particular type of urinary catheter strategy. Group C (n = 14) included supplementary articles (eg, review articles). We provided copies of all articles and generated summary tables for articles in groups A and B (

]), highlighting outcomes of interest. We also provided the team with an overview of transurethral surgical procedures for treating BPH from UpToDate

 as a reference to give a general overview of BPH and its surgical treatments, particularly for the nonurologist members of the panel (eg, nurses, infectious disease physicians).

Next, we recruited experienced practicing clinicians, including urologists who had performed transurethral prostate surgery across a variety of practice types (academic, private, government) and US regions (West, Midwest, Northeast), and subject matter experts (eg, BPH, infectious disease, neurogenic bladder) to participate in this panel by sending an introductory email describing the panel and process. Panelists (N = 11) (

) included urologists (n = 8), nurses (n = 2) who care for urologic patients, and an infectious diseases physician with expertise in UTIs.

Participating panelists were sent materials to complete a round 1 independent rating of the clinical scenarios (March-April 2015). They were sent instructions, literature summary tables, relevant articles, and a scoring document (see 

 for example section of the round 1 scoring document) that included the clinical scenarios and asked when the first trial of void should occur after surgery. They were asked to examine appropriateness across 4 clinical scenarios (no preexisting catheter, preexisting catheter [including intermittent], difficult catheter placement, significant perforation) and 5 durations (postoperative day [POD] 0, 1, 2, 3-6, or ≥7). All scenarios were for adult male patients in acute care inpatient or ambulatory surgery settings undergoing routine surgery. Panelists were instructed to use their best clinical judgment in combination with evidence from the literature review and to assume no other relevant patient characteristics (eg, comorbidities, catheter trauma). They also rated the appropriateness of intermittent straight catheterization (ISC) and indwelling urinary catheter placement after a failed first trial of void, including time to a second trial of void.

Catheter removal was considered appropriate if “the expected health benefit (eg, relief of pain, reduction in anxiety, improved functional capacity) exceeds the expected negative consequences (eg, mortality, morbidity, anxiety, pain, time lost from work) by a sufficiently wide margin that the procedure is worth doing, exclusive of cost.”

 For each scenario, we asked the panelist to rate the appropriateness of the duration of urethral catheterization by circling a number on a scale from 1 to 9. A rating of 1 indicated that the harms significantly outweigh the benefits (ie, inappropriate), whereas a rating of 9 indicated that the benefits significantly outweigh the harms (ie, appropriate). A central rating of 5 indicated that the benefits or harms were considered equal or that the participant was unable to make an informed rating of the clinical scenario.

After the initial round 1 ratings were complete, we conducted a 1-hour conference call (April 17, 2015) to clarify the clinical scenarios and reduce disagreement or uncertainty in panelist ratings. Gross hematuria requiring bladder irrigation was not included in the final scenarios because catheter removal would not be clinically appropriate per the panel. The panel was then brought together for a face-to-face meeting in which each scenario, preliminary scores, and rating differences were discussed to determine a final appropriateness score for each clinical scenario. This round 2 meeting was conducted on May 4, 2015. Panelists rerated each clinical scenario after the in-person discussion. The median round 2 scores were used to classify each scenario as appropriate (panel median score of 7-9), uncertain or neutral (panel median score of 4-6), or inappropriate (panel median score of 1-3). In addition, if 4 or more panelists rated a scenario as appropriate (median score of 7-9) and 4 or more rated it as inappropriate (median score of 1-3), the scenario was rated as uncertain or neutral due to disagreement.

As detailed in eAppendix Table 1, 15 studies

 published from 1991 to 2014 were identified for transurethral prostate and/or ablation procedures reporting at least 1 outcome of interest (catheter use, urinary retention, urinary tract infection, or other complications) for patients with respect to a specific postoperative urinary catheter removal protocol. Five studies were randomized controlled trials,

 5 were quasi-experimental studies with controls,

 and the remaining 5 were cohort studies without a comparison group.

 Variations in early catheter removal protocols included removal on same day of surgery; removal within 24 hours or on POD 1, including removal at midnight on day of surgery versus 6 am on POD 1

 Overall, the studies reported similar or improved outcomes for the early catheter removal strategy (compared with either concurrent or historical controls, as varied by study design), including the shortest catheterization periods investigated (all 4 studies investigating same-day removal

). Many studies also reported significantly reduced length of stay and either calculated or presumed reduced hospital costs for cases employing the early catheter removal protocols.

 details the panel responses across a group of common scenarios. Removing the urethral catheter on the day of surgery was deemed appropriate for patients both with and without preexisting catheter needs. However, a voiding trial on the day of surgery was rated inappropriate for patients with a difficult catheter placement after surgery or those with intraoperative findings of clinically significant prostatic capsule perforation (eg, obvious urine extravasation).

Based on our panel ratings, urinary catheter removal and first trial of void on POD 1 was appropriate for all scenarios except clinically significant perforations. In this case, waiting until POD 3 was deemed the earliest appropriate timing for removal. The panel was split with respect to catheter removal on PODs 1 and 2 for the perforation scenario, with some panelists rating removal appropriate and others rating it inappropriate. Furthermore, the perforation scenario was the only appropriate indication for waiting 7 or more days for initial trial of void. Conversely, waiting 3 or more days to remove the catheter for patients with or without preexisting catheter needs, or for those with difficult catheter placement in the operating room, was rated as inappropriate.

The next question was: “After failing the first trial of void after transurethral resection or ablation of prostate for BPH, how appropriate is this type of urinary catheterization to manage retention (ISC and/or indwelling urethral catheter)?” We instructed the panelists to make several assumptions, including no evidence of prostate cancer, acontractile bladder, urethral stricture, need for continuous bladder irrigation for hematuria, difficult catheter placement, or intraoperative clinically significant perforation. The panel uniformly rated both ISC and indwelling urethral catheter placement as appropriate.

, the panel was then asked “If a Foley was placed after a failure of first trial of void after transurethral resection or ablation (including enucleation) of prostate for benign prostatic hyperplasia, what is the appropriateness of the following durations of Foley catheter use before second trial of void?” to specifically inquire about appropriateness according to hospitalization status (inpatient, discharged home). The panel rated a second trial of void appropriate after 1 or 2 days in hospitalized patients. For patients discharged home, waiting up to 4 days was the longest appropriate duration per the expert panel, with waiting 5 to 7 days for a second trial of void rated as uncertain or raising disagreement among the panelists. However, waiting 8 or more days for a second trial of void for patients discharged home was deemed inappropriate.

We used the RAND/UCLA Appropriateness Method to determine appropriate urinary catheter management strategies following transurethral resection and ablation procedures for BPH, including enucleation. We found that waiting 3 or more days to remove a catheter for a first voiding trial after these procedures was inappropriate for the majority of patients. In other words, unless a clinically significant perforation occurs at the time of the procedure, it was deemed appropriate for most patients to have their catheter removed for a trial of void on POD 1. We also found that both ISC and indwelling catheter placement were acceptable approaches to manage patients after failing their first trial of void. If an indwelling catheter was placed after an initial failed trial of void, the panel indicated that hospitalized patients should be given another trial of void within 1 or 2 days. For patients discharged home, the appropriate duration was extended up to 4 days to allow for a second trial of void.

Given the lack of guidelines for urethral catheter duration and this robust expert panel approach to examining appropriate catheter use after one of the most common procedures in the United States, our findings may help improve the consistency and quality of care for patients undergoing transurethral surgery for BPH.

The implications of our findings depend on the extent to which current clinical practice varies with respect to catheter duration after transurethral prostate surgery. However, outside of limited research studies, the duration of catheter use after surgery is poorly understood for at least 2 reasons. First, research has focused on the tremendous variation in population-based rates of transurethral surgery itself, rather than catheter duration. Second, there is a paucity of BPH measures to motivate providers and healthcare systems to track quality of care. Nonetheless, length of stay is arguably a proxy for catheter duration and varies tremendously across hospitals. In fact, a recent study demonstrated greatest variation in length of stay after transurethral prostate resection compared with other common benign urologic surgery types including percutaneous nephrolithotomy and pyeloplasty.

 This is an important consideration for healthcare systems seeking to minimize length of stay after common procedures like transurethral prostate surgery. Standardizing postoperative catheter use is likely to decrease practice and length of stay variation, minimize catheter discomfort for patients, and potentially lower the risk of postoperative complications (eg, UTI).

The lack of quality indicators for transurethral prostate surgery is striking given the variation in practice across the United States. According to the Dartmouth Atlas, the adjusted rate of inpatient transurethral surgery per 1000 male Medicare beneficiaries varies 6-fold across hospital referral regions, from 0.3 to 1.8.

 In light of this heterogeneity across varied indications and preferences for the procedure, perhaps focusing initially on perioperative processes of care is warranted. Our appropriateness recommendations from practicing clinicians support urinary catheter duration after surgery as a potential initial quality measure to promote consistent, appropriate care for men surgically treated for BPH. In fact, the RAND/UCLA Appropriateness Method has been used to develop quality indicators across disease and treatment types, setting the precedent for such an approach.

 Finally, there is interest by the CDC, which collects measures of catheter-associated UTI and catheter use, in developing a measure of urinary catheter appropriateness that could be applied based on data from the electronic health record (eg, procedure type, comorbidity).

 In fact, an adjusted standardized utilization ratio for urinary catheter use has been proposed,

 and thus, our work would inform future modifications of this and similar metrics based on urinary catheter appropriateness to be used across healthcare organizations.

There are limitations to our approach. First, our panel included 11 members. However, all 8 of the participating urologists performed at least 1 type of transurethral prostate surgery, and the spectrum of procedures (resection, ablation, enucleation) was covered within their clinical expertise. Moreover, the panelists represented a variety of institutions and practice settings across the United States, increasing the generalizability of the findings. Additionally, including a mix of specialists and nonspecialists on our panel provided differing perspectives.

 Second, there were few randomized trials included in our literature review to support appropriateness ratings. However, we did use a systematic approach to identify relevant literature across a range of study types, although several of the studies were published in the 1990s. The panelists were instructed to use their best clinical judgment in addition to the literature review to inform their ratings. Third, our final ratings applied to 4 common clinical scenarios for men undergoing transurethral prostate surgery that emerged through our rating process. Although these likely address the majority of cases in real-world practice, other patient, disease, and procedure characteristics outside of our appropriateness ratings could create challenges for implementation. We did include prostate size in our initial rating tool, but it was removed from our final rating tool based on the limited perceived relevance to catheter duration among the panelists. Furthermore, we did not specifically inquire regarding anticoagulation, although this is typically temporarily discontinued for the majority of patients undergoing this surgery. Last, we did not include all BPH surgery types in our literature review or appropriateness ratings. Although patients having prostate enucleation, by virtue of the degree of tissue removal, may slightly differ from the general population having transurethral resection surgery (eg, larger gland size), the panel felt that catheter management strategies afterward should be largely similar to other transurethral surgery approaches. Overall, our findings are relevant to the majority of transurethral prostate resection, ablation, and enucleation surgery types.

We defined clinically relevant guidance statements for the appropriateness of catheter duration after transurethral prostate surgery. Findings from our robust methodological approach, including urethral catheter removal and trial of void on POD 1 being rated as appropriate for all scenarios except clinically significant perforations, may help promote the consistency and quality of care for patients undergoing transurethral surgery for BPH within and across healthcare delivery systems.

The authors thank Helen McGuirk for her assistance with coordination of the panels, and they also thank all of the panelists who participated in this project.

 VA Center for Clinical Management Research, VA Ann Arbor Healthcare System (TAS, KEF, SJB, JM), Ann Arbor, MI; Department of Urology, University of Michigan (TAS, CAD), Ann Arbor, MI; Division of General Medicine, Department of Internal Medicine, University of Michigan (JDM, SJB, JM), Ann Arbor, MI; School of Public Health, University of Michigan (SJB), Ann Arbor, MI; Division of General Pediatrics, Department of Pediatrics and Communicable Diseases, University of Michigan (JM), Ann Arbor, MI.

 This panel was funded by a contract from the Agency for Healthcare Research and Quality (AHRQ) (HHSA2902010000025I/HHSA29032001T). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the AHRQ or the Department of Veterans Affairs.

 This work was accepted as a moderated poster for the 2017 Annual Meeting of the American Urological Association in Boston, Massachusetts.

 Dr Meddings is an investigator for an AHRQ subcontract for development and implementation of an intervention to prevent catheter-associated urinary tract infection in the intensive care unit setting. Her research involves development of products to improve patient safety by reducing hospital-acquired complications. Her team has a provisional patent involving one of these products (US20180339133A1). She has no associations with any companies or manufacturers, has no ownership in a commercial entity, and receives no royalties. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (TAS, KEF, JM); acquisition of data (TAS, KEF, JM); analysis and interpretation of data (TAS, CAD, KEF, SJB, JM); drafting of the manuscript (TAS, CAD, SJB); critical revision of the manuscript for important intellectual content (TAS, CAD, KEF, JDM, SJB, JM); provision of patients or study materials (TAS, JM); obtaining funding (JM); administrative, technical, or logistic support (TAS, KEF, JDM); and supervision (TAS, JM).

 Ted A. Skolarus, MD, MPH, University of Michigan, Taubman Center, Room 3875, 1500 E Medical Center Dr, SPC 5330, Ann Arbor, Michigan 48109. Email: tskolar@med.umich.edu.

1. Malaeb BS, Yu X, McBean AM, Elliott SP. National trends in surgical therapy for benign prostatic hyperplasia in the United States (2000-2008). 

. 2012;79(5):1111-1116. doi: 10.1016/j.urology.2011.11.084.

2. Appropriate use criteria program. CMS website. cms.gov/medicare/quality-initiatives-patient-assessment-instruments/appropriate-use-criteria-program/index.html. Updated February 21, 2019. Accessed June 12, 2019.

3. Ghomrawi HM, Schackman BR, Mushlin AI. Appropriateness criteria and elective procedures—total joint arthroplasty. 

. 2012;367(26):2467-2469. doi: 10.1056/NEJMp1209998.

4. Yajima N, Tsujimoto Y, Fukuma S, et al. The development of quality indicators for systemic lupus erythematosus using electronic health data: a modified RAND Appropriateness Method [published online June 26, 2019]. 

5. Chartrand M, Guénette L, Brouillette D, et al. Development of quality indicators to assess oral anticoagulant management in community pharmacies for patients with atrial fibrillation. 

. 2018;24(4):357-365. doi: 10.18553/jmcp.2018.24.4.357.

Articles

This manuscript synthesizes findings from a multidisciplinary panel following the RAND/UCLA Appropriateness Method to guide standardization of urinary catheter use after transurethral prostate surgery.

Catheter Management After Benign Transurethral Prostate Surgery: RAND/UCLA Appropriateness Criteria

 Prostate cancer treatment is a significant source of morbidity and healthcare spending. Evolving clinical data have supported expanding surveillance as a means to “right-size” treatment. Integrated delivery systems afford the possibility of hastening this objective.

 Retrospective cohort study of Medicare beneficiaries.

 We used a 20% sample of national Medicare claims to assess the impact of healthcare integration on rates of treatment and potential overtreatment in men newly diagnosed with prostate cancer between 2007 and 2011. Rates were measured according to the extent of integration within a market (ie, none, low, intermediate, and high). Generalized estimating equations were used to assess the relationship between integration and utilization, adjusting for confounders.

Rates of treatment declined across all markets (

 <.01 for overall time trend), but the rate of decline was similar for the 4 market types (

 = .27). In the most integrated markets, the rate decreased by 28.8%, or from 55.5 per 10,000 population in 2007 to 39.5 per 10,000 in 2011. After adjusting for confounders, men residing in the most integrated markets were 2.1% less likely to be treated with curative intent compared with those living in areas without integrated delivery systems (

 = .04). However, rates of potential overtreatment were similar across all markets regardless of the level of integration (

Healthcare integration was associated with small declines in prostate cancer treatment in newly diagnosed men, but not with potential overtreatment. Integrated care alone may be insufficient to curtail potential overtreatment of prostate cancer.

		Integration was associated with a small decrease in the treatment of prostate cancer. 

		Integration was not associated with treatment among men least likely to benefit (ie, those with a high risk of noncancer mortality). 

		Integration alone may be insufficient to promote optimal care for prostate cancer. 

		Financial risk, implied by evolving delivery system reforms, may help to promote better stewardship of preference-sensitive diseases, such as prostate cancer.

Prostate cancer is among the most common malignancies in men in the United States.

 Ongoing uncertainties about how best to treat the disease, coupled with the availability of multiple options, have led to wide variations in both the quantity and quality of care.

 Prostate cancer spending has increased by 11% annually over the last decade, outpacing rates for other common conditions (such as cardiovascular and pulmonary diseases) and resulting in $12 billion in yearly expenditures.

 Although the merits of screening are a subject of ongoing debate in the field, consensus is growing that some newly diagnosed men with prostate cancer stand to gain little from treatment.

Improving the efficiency of the delivery system and eliminating wasteful spending have long been priorities for payers and policy makers, and many hope that accountable care organizations (ACOs) and related components of healthcare reform will do just that. By encouraging closer alignment between hospitals and caregivers, ACOs aim to focus on improving quality and cutting costs—both of which may affect prostate cancer care. To a large extent, ACOs are extensions of integrated delivery systems that, due to their emphasis on evidence-based medicine and minimizing unnecessary healthcare, are associated with providing higher quality.

 Thus, understanding the implications of integrated delivery systems for prostate cancer care will help us to anticipate the likely effect of evolving reforms of the Affordable Care Act.

For this reason, we performed a national study to determine the impact of healthcare integration on the management of prostate cancer. We hypothesized that the most integrated markets would be more selective in the use of curative treatment for prostate cancer, particularly among those unlikely to benefit from intervention.

Using a 20% sample of Medicare claims, we performed a retrospective cohort study of fee-for-service beneficiaries newly diagnosed with prostate cancer between January 1, 2007, and December 31, 2011. We limited our study to men continuously enrolled in Parts A and B for at least the 12-month periods prior to and after the prostate cancer diagnosis. To ensure that we had complete claims on all patients, we excluded patients in risk-bearing Medicare managed care plans. Patients were followed through December 31, 2012.

To identify incident prostate cancer cases in national Medicare claims, we developed an algorithm and validated it using Surveillance Epidemiology and End Results (SEER) cancer registry data. Briefly, we used a 5% sample of Medicare beneficiaries residing in a catchment area of a SEER registry in the years 2003 to 2005. We selected men with at least 2 “Evaluation and Management” visit codes in which the line diagnosis 

International Classification of Diseases, Ninth Revision, Clinical Modification 

) code was 185 for prostate cancer. We further required that all incident cases underwent prostate biopsy within 180 days of the first visit code associated with a prostate cancer diagnosis. Men with any claim in the preceding 12-month period that was associated with an 

 diagnosis code of 185 (prostate cancer) or V10.46 (personal history of malignant neoplasm of prostate) were excluded. Finally, we validated this approach against the Patient Entitlement Denominator Summary File, which identifies all incident cases in SEER regions, and found our algorithm to have specificity and positive predictive values of 99.8% and 88.7%, respectively. We then implemented this algorithm in our 20% national sample of Medicare claims to identify incident cases, which compose our study population.

We used hospital referral regions (HRRs), as described by the Dartmouth Atlas,

 to reflect distinct healthcare markets. There are 306 HRRs in the United States, each of which represents a collection of zip codes in which Medicare patients receive the bulk of their healthcare. We determined each market’s level of integration by measuring the proportion of hospital discharges occurring from an integrated delivery system, which were identified from public reports based on data from IMS Health

 and have been used in a similar context.

 These data provide information on delivery system relationships, including affiliations between hospitals and physician practices. Each health system is rated for 33 attributes in 8 domains: overall integration, integrated technology, hospital utilization, financial stability, services, access, contract capabilities, and physicians. Domain-specific scores are summed to yield an overall score for the delivery system, with higher scores reflecting a greater degree of integration.

All HRRs were characterized by the proportion of hospital discharges occurring from a top-100 integrated delivery system.

 Of these, 127 HRRs had no discharges from a top integrated delivery system. The remaining 179 HRRs were sorted into 3 equal groups (ie, terciles) ranging from least integrated (a mean of 14% of discharges from an integrated delivery system) to most (a mean of 71% of discharges from an integrated delivery system). For the purpose of analysis, our market level of integration exposure was treated as a categorical variable with 4 levels: none, low, intermediate, and high.

Because our measure of integration relies on public reports of integrated delivery systems and does not necessarily distinguish between clinical and/or financial integration, it is possible that it may be insensitive to real differences in quality. For this reason, we assessed whether this measure was able to identify differences in publicly available measures

 of quality—including patients with diabetes undergoing glycated hemoglobin (A1C) testing, eye examinations, and blood lipid testing&mdash;between 2008 and 2011. Generally, we observed a monotonic relationship between the level of integration and adherence to each measure. In all instances, the most integrated markets performed better than those without integration. For instance, beneficiaries in highly integrated markets were 21% more likely to undergo A1C testing than were those residing in markets without a top-100 integrated delivery system. These data support that our exposure can detect measurable differences in quality in circumstances in which clinical integration is important.

Our primary outcome was the rate of curative treatment (ie, surgery, external beam radiation therapy, brachytherapy, and cryotherapy) within 12 months of diagnosis, measured at the HRR level. For this calculation, the numerator was determined by the annual count of newly diagnosed patients undergoing any of the aforementioned treatments in a given HRR; the denominator was determined by the eligible male Medicare population residing in an HRR in a given year.

We also measured potential overtreatment. Because of their emphasis on quality and cost containment, integrated delivery systems and, presumably, the markets in which they dominate may be more selective in the services they provide. In accordance with clinical guidelines, we would expect more-integrated markets to have lower rates of treatment among those patients with a high risk of noncancer mortality within 10 years of diagnosis. These patients typically have a low probability of dying from the disease, even absent treatment.

 We assessed treatment among this population of patients (ie, the quartile of men with the highest risk of dying from noncancer causes within 10 years) by implementing methods developed by Gross and colleagues.

 Using a 5% sample of Medicare beneficiaries without cancer, we built a robust patient-level model to predict mortality (C-index = 0.91). This enabled us to estimate the 10-year mortality risk of patients in our prostate cancer cohort absent their cancer diagnosis. Those patients in the top quartile had a 78% risk of non—prostate cancer mortality within 10 years of their diagnosis. Potential overtreatment was assessed at the patient level, and population-based rates were calculated as described above (ie, the numerator was the number of potentially overtreated newly diagnosed patients with prostate cancer, and the denominator was the number of eligible Medicare beneficiaries).

We first contrasted patient and regional characteristics according to the healthcare integration exposure (ie, none, low, intermediate, and high). Statistical inference was made using the 

 tests for continuous data. We then fit a Poisson model with an offset term for the population denominator to assess a trend in population-based rates of curative treatment over time across the integration groups.

To test the independent effect of healthcare integration on rates of prostate cancer treatment, we fit a multivariable logistic regression model using patient-level treatment as the outcome. Our healthcare integration exposure was incorporated into the model at the HRR level. To account for the nested nature of the data (ie, patients within HRRs), generalized estimating equations were used. We adjusted the model for patient-level differences, including age, race, comorbidity, and socioeconomic class. Comorbidity was calculated with patient claims for the 12-month window prior to diagnosis using established methods.

 Socioeconomic class was estimated using a composite measure developed at the zip code level, as described by Diez-Roux and colleagues.

 We used the Area Resource File to derive several market-level variables to include in the model, including measures of social capital (% female head-of-household) and education (% with a high school degree or more), and 2 supply side variables (urologists and hospital beds per capita). We computed adjusted percentages for the use of treatment for each level of our market integration exposure by back-transforming the predicted use from the model. Using similar methods, a separate model was then fit to derive adjusted percentages of treatment among those least likely to benefit. Further, we assessed trends in rates by both curative (ie, surgery, external beam radiation therapy, brachytherapy, cryotherapy) and noncurative (ie, hormone therapy, no treatment) modalities in the population least likely to benefit from treatment.

All analyses were carried out using computerized software SAS version 9.4 (SAS Institute, Cary, North Carolina). All tests were 2-tailed and the probability of type 1 error was set at 0.05. The study protocol was judged to be exempt by the Institutional Review Board at the University of Michigan.

Patient and regional characteristics were contrasted according to the market level of integration (

). Although statistically significant differences among market types were evident for some variables (eg, age), the magnitudes of the absolute differences were small, with one exception. Among markets with at least some discharges from integrated delivery systems, patients in markets with the lowest level of integration were more affluent than those in more integrated ones. For instance, 28.8% of patients in the least integrated markets resided in the highest quartile of socioeconomic class compared with 21.9% of patients in the most integrated ones (

Between 2007 and 2011, population-based rates of curative treatment among men newly diagnosed with prostate cancer declined across all markets (

), regardless of the level of integration (

 <.01 for overall time trend). For example, in the most integrated markets, rates of treatment decreased from 55.5 per 10,000 male beneficiaries in 2007 to 39.5 per 10,000 in 2011, a relative decrease of 28.8%. In markets without integrated delivery systems (ie, no discharges from these facilities), rates of treatment declined by 30.1%, or from 57.2 per 10,000 to 40.0 per 10,000 over the same period. The rate of decline in treatment over time was similar for the 4 market types (

, rates of potential overtreatment—treatment of those patients with the highest probability of death from noncancer causes&mdash;also decreased, albeit to a lesser extent (

 <.01 for overall time trend). In the most integrated markets, curative treatment in this population decreased by 19%, from 10.0 per 10,000 to 8.1 per 10,000. Rates of decline were similar for the less integrated markets (

 = .98). We next explored changes in treatment, by modality, among those least likely to benefit from treatment (

). Rates of radical prostatectomy in these patients were low but remained stable over time (

 = .99). Conversely, rates of brachytherapy, cryotherapy, and external beam radiation therapy decreased during the course of the study (each 

 <.01 for overall time trend), but these declines did not vary with respect to the market level of integration (each 

 >.20). Similar trends were observed for both noncurative modalities.

Finally, we used multivariable modeling to account for the observed subtle differences between patients and healthcare markets (

). Among beneficiaries with newly diagnosed prostate cancer, we found that men residing in the most integrated markets were 2.1% less likely to be treated with curative intent compared with those living in areas without integrated delivery systems (

 = .04). However, rates of potential overtreatment (ie, treatment of men with a high probability of death from noncancer causes) were similar across all markets regardless of the level of integration (

Rates of treatment for prostate cancer among Medicare beneficiaries declined significantly between 2007 and 2011. Similarly, treatment among men with a high risk of noncancer mortality (ie, potential overtreatment) decreased over the same period, albeit to a lesser extent. The use of nonsurgical approaches decreased significantly in men with the highest probability of noncancer death within 10 years, while rates of surgery remained stable in this population. Rates of observation (ie, no treatment) and hormone therapy in this population declined similarly. This supports the theory that secular declines in screening and diagnosis, which are well established,

 underlie the observed trends, as opposed to more selective curative treatment by physicians. Although patients residing in markets with the highest level of integration were less likely to undergo curative treatment for their cancer, the difference in magnitude relative to less integrated markets was small and likely of limited clinical significance. Notwithstanding the advantages of highly integrated markets to deliver better evidence-driven healthcare, we found that the use of treatment among those least likely to benefit did not vary significantly according to the degree of integration in a market.

Although prostate cancer remains a common cause of cancer-related death in the United States, consensus is growing about the potential pitfalls of screening and the indolent nature of some cancers.

 Approaches to management of prostate cancer have evolved considerably over the last decade, and surveillance has increasingly been recognized as a strategy in some men to prevent overtreatment. Despite this recognition, there is little evidence that the use of treatment in men unlikely to benefit is decreasing; rather, population-based data suggest that it is increasing.

 In our study of national Medicare beneficiaries, we noted a significant decline in potential overtreatment across all markets regardless of their level of integration. However, these declines were relatively modest given the nature of the population (ie, having a very high risk of noncancer mortality within 10 years of diagnosis).

That healthcare integration was associated with small declines in treatment, but not with potential overtreatment, is surprising. Indeed, integrated delivery systems are more apt to follow evidence-based practice guidelines, adopt electronic health information systems, and implement strategies for performance improvement; ultimately, they are associated with providing higher-quality healthcare.

 Because of these qualities, we would expect healthcare markets dominated by these systems to be more selective in whom they treat for prostate cancer, particularly among those who are unlikely to benefit.

With recent health reforms toward more accountable care, collections of providers and, in some cases, hospitals, are evolving to become better stewards of population health. To some extent, ACOs are the result of the natural evolution of high-performing integrated delivery systems. However, in addition to sharing a focus on care coordination and other aspects of integrated care, ACOs imply a level of financial risk likely surpassing that which was assumed by integrated delivery systems in the era prior to health reform. Perhaps it is this risk, added to the pressure to realize savings at the beneficiary level, that will promote better stewardship of preference-sensitive conditions in which there are clear tradeoffs with treatment, such as prostate cancer.

One potential limitation of our findings is the absence of measures of disease severity (ie, cancer grade, stage, and prostate-specific antigen levels) in national Medicare claims. Although we classify potential overtreatment using the probability of noncancer death, our findings may underestimate the scope of treatment in the population, as patients with low-risk prostate cancer, regardless of life expectancy, are well-accepted candidates for surveillance.

 Further, the absence of disease severity measures has little implication for our comparison of treatment rates across healthcare markets, as the distribution of cancer grade and stage tends to be similar across geographic regions.

Our findings have important implications for evolving reforms aimed at improving the efficiency of healthcare delivery. We observed a small, albeit significant, association between higher levels of integration and more constrained use of prostate cancer treatment. However, rates of potential overtreatment of men with prostate cancer were similar, regardless of the extent of market integration. Collectively, these findings suggest that integration alone may be insufficient for optimizing the management of conditions such as prostate cancer, in which considerable uncertainty exists about the tradeoffs between treatment and observation. Future research should explore how the added financial risk associated with ACOs is able to modulate the management of preference-sensitive diseases such as prostate cancer.

 Dow Division of Health Services Research, Department of Urology (BKH, MJB, SRK, LH, TAS, DCM) and the Kidney Epidemiology Cost Center (VBS), University of Michigan, Ann Arbor, MI.

 This work was supported by a Research Scholar Grant RSGI-13-323-01-CPHPS to BKH from the American Cancer Society. VBS is supported by funding from the National Cancer Institute (R01 CA168691). DCM is supported by funding from the National Cancer Institute (R01 CA174768).

 Dr Hollenbeck received a grant from the American Cancer Society. Dr Miller received a grant from the National Cancer Institute. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (BKH, DCM, VBS); acquisition of data (BKH); analysis and interpretation of data (MJB, BKH, LH, SRK, TAS, VBS); drafting of the manuscript (BKH, LH, DCM, TAS); critical revision of the manuscript for important intellectual content (MJB, BKH, LH, SRK, DCM, TAS, VBS); statistical analysis (MJB, SRK); obtaining funding (BKH, DCM, VBS); and supervision (SRK).

 Brent K. Hollenbeck, MD, MS, Dow Division of Health Services Research, Department of Urology, University of Michigan, 2800 Plymouth Rd, NCRC Bldg 16, Ann Arbor, MI 48109-2800. E-mail:

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. 

. 2015;65(1):5-29. doi: 10.3322/caac.21254.

2. Cooperberg MR, Broering JM, Carroll PR. Time trends and local variation in primary treatment of localized prostate cancer. 

. 2010;28(7):1117-1123. doi: 10.1200/JCO.2009.26.0133.

3. Spencer BA, Miller DC, Litwin MS, et al. Variations in quality of care for men with early-stage prostate cancer. 

. 2008;26(22):3735-3742. doi: 10.1200/JCO.2007.13.2555.

4. Roehrig C, Miller G, Lake C, Bryant J. National health spending by medical condition, 1996-2005. 

. 2009;28(2):w358-w367. doi: 10.1377/hlthaff.28.2.w358.

5. Mariotto AB, Yabroff KR, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010-2020. 

. 2011;103(2):117-128. doi: 10.1093/jnci/djq495.

6. Klotz L, Vesprini D, Sethukavalan P, et al. Long-term follow-up of a large active surveillance cohort of patients with prostate cancer. 

. 2015;33(3):272-277. doi: 10.1200/JCO.2014.55.1192.

Healthcare integration was associated with small declines in treatment, but no change in overtreatment of prostate cancer. Integrated care delivery alone may be insufficient to curtail overtreatment.

Implications of Evolving Delivery System Reforms for Prostate Cancer Care

To assess the feasibility and validity of developing electronic clinical quality measures (eCQMs) of cancer care quality from existing metrics, using electronic health records, administrative, and cancer registry data.

Retrospective comparison of quality indicators using chart abstracted versus electronically available data from multiple sources.

We compared the sensitivity and specificity of eCQMs created from structured data from electronic health records (EHRs) linked to administrative and cancer registry data to data abstracted from patients’ electronic health records. Twenty-nine measures of care were assessed in 15,394 patients with either incident lung or prostate cancer from 2007 and 2008, respectively, and who were treated in the Veteran’s Health Administration (VHA).

It was feasible to develop eCQMs for 11 of 18 (61%) lung cancer measures, 4 (22%) of which were considered to be valid measures of the care constructs. Among prostate cancer measures, 6 of 11 (55%) were feasible, and 4 (36%) were both feasible and valid. Of the 29 metrics, data was available to create eCQMs for 17 (59%) cancer care metrics, and 8 (28%) were considered valid.

In a large integrated healthcare system with nationally standardized electronic health records, administrative, and cancer registry data, 28% of cancer quality measures developed for chart abstraction could be translated into valid eCQMs. These results raise much concern about the development of electronic clinical quality measures for cancer care, particularly in healthcare environments where data are disparate in both form and location.

		Only 28% of cancer quality measures could be translated into electronic clinical quality metrics, even in the largest integrated healthcare system in the United States with nationally standardized electronic data repositories.

		These results raise concern about the use of electronic clinical quality measures for cancer care, particularly outside integrated healthcare environments, and suggest that it may be premature to think that meaningful measures of cancer care delivery can be developed for widespread use across electronic health record platforms.

		Current health data systems are inadequate to support meaningful electronic quality measurement in cancer care.

As the healthcare system moves toward a paradigm that rewards “value” in healthcare, or the maximizing of quality while minimizing costs, accurate measurement of the quality of healthcare delivery has taken center stage in reform efforts.

 Currently, most quality measures rely on for data acquisition chart abstraction, which is considered the gold standard. While chart abstraction allows access to all types of medical data, clinical notes in particular, it is time-consuming and prohibitively expensive to permit broad and timely measurement of care quality.

 It has long been hoped that electronic measurement of care delivery would open the door to timely, meaningful, and actionable information for many stakeholders.

 Electronic health records (EHRs), now adopted by more than 70% of physicians, and health information exchanges are 2 key components of the visions of data interoperability.

 This is coupled with technical developments, like natural language processing (NLP) and structured clinical templates, which are expected to facilitate computerized access to the content of free text that underlies much of primary clinical documentation, and

 payment reforms, like accountable care organizations, which encourage linkage of healthcare data silos.

7 While human chart abstractors are able to bridge data silos and interpret the full range of available data sources, it is still unclear if the above reforms will make electronic clinical quality measures (eCQMs) and measurement feasible. Many studies have illustrated wide gaps in agreement among electronically available data sources and between these sources and chart abstraction.

 Furthermore, eCQMs that are solely based on administrative data have been criticized for failing to measure what matters.

 Cancer care, additionally, is especially challenging to measure, in part because it relies on the multifactorial process of staging and because care is typically delivered across multiple specialties and care settings.

Large integrated healthcare systems, like the Veterans Health Administration (VHA), having already achieved national data integration across all aspects of care delivery, with a common EHR and national, standardized, electronically available data warehouses, offer an opportunity to examine the feasibility and validity of using available electronic data to measure cancer care quality.

 Recently, the VHA performed quality assessments of care for lung and prostate cancer patients using manual chart abstraction.

We hypothesized that the data needed to measure quality performance was available in the VHA’s electronic clinical, administrative, and cancer registry repositories and could be used to create eCQMs from existing quality measures.

This was a retrospective, national VHA cohort study sponsored by the VHA Office of Analytics and Business Intelligence. The VHA Greater Los Angeles Institutional Review Board approved this study.

VHA expert panels reviewed the relevant literature and proposed quality measures—using the RAND/ UCLA modified Delphi technique, for national assessments of the quality of care for lung cancer and prostate cancer diagnosed in 2007 and 2008, respectively&mdash;to be performed using data abstracted from patients’ EHRs.

 There were 29 quality metrics (18 for lung cancer and 11 for prostate cancer) spanning the continuum of care (see Appendix for more detailed descriptions). Many measures replicated, or were similar to, metrics endorsed by non-VHA organizations (eAppendix). Twenty measures addressed diagnosis, treatment, and management; 5 addressed supportive care; and 4 addressed end-of-life care. Twenty-six measures were patient level, 2 were visit level, and 1 was medical center level.

Data necessary for 29 measures of the quality of lung and prostate cancer care were abstracted from patients’ VHA Computerized Patient Record System (CPRS) medical records by External Peer Review Program (EPRP) nurse abstractors conducting the national VHA assessments of the quality of cancer care (chart abstraction cohort). Using an abstraction tool specifically developed for the quality assessments, EPRP abstractors systematically abstracted the data necessary to score the quality indicators. Case-level quality indicator and timeliness results were provided to VAMCs for review and correction, and the final results were calculated based on these field-corrected data.

Data for 29 eCQMs were obtained from extracts from the VHA’s EHR through the Corporate Data Warehouse (CDW) and Decision Support System (DSS), both of which are national, standardized, and near real-time clinical and administrative data repositories; and from the VHA Central Cancer Registry (VACCR). The eCQM data set was created by linking data from the VACCR, CDW and DSS. This included inpatient and outpatient encounter and procedure coding, as well as pharmacy, lab, radiology, allergy, problem list, and vital sign data.

In both the chart abstraction and eCQM cohorts, case identification criteria were identical: all newly diagnosed, pathologically confirmed cases of lung cancer during 2007 (n = 8125) and prostate cancer during 2008 (n = 12,572) that were reported to the VACCR. Patients were excluded if the pathologic diagnosis was not identified in the EHR at the index facility that reported the case to the registry (lung, 1297; prostate, 489), a situation that could occur if patients were treated at more than 1 facility.

Patients were excluded from the chart abstraction cohorts if they were diagnosed at autopsy; death occurred in 30 days or less after cancer diagnosis; hospice enrollment occurred 30 days or less after cancer diagnosis (lung, 947; prostate, 16); they were enrolled if they had a pre-existing or concurrent diagnosis of metastatic cancer other than lung or prostate cancer (lung, 540; prostate, 627); they were enrolled in a cancer clinical trial (lung, 57; prostate, 147); there was documentation of comfort measures only in a hospital discharge summary or nursing home note 30 days or less after cancer diagnosis (lung, 91; prostate, 14); or documentation of life expectancy of 6 months or less in their Problem List at the time of diagnosis (lung, 39; prostate, 4). The resulting lung cancer and prostate cancer abstraction cohorts included 4865 and 11,211 patients, respectively.

Patients were excluded from the lung and prostate cancer eCQM cohorts if they died within 60 days of diagnosis (lung, 1482; prostate, 674); if there was no pathologic confirmation of disease according to the VACCR (lung, 419; prostate, 49); or if there were less than 2 encounters with an 

International Classificiation of Diseases, Ninth Revision

 (ICD-9) diagnosis code for prostate or lung cancer (lung, 229; prostate, 8). The eCQM cohort included 5995 patients with lung cancer and 11,700 patients with prostate cancer.

The final validation cohort included patients common to both the lung and prostate abstraction and the eCQM cohorts, n = 4865 and n = 10,529, respectively.

Adaptation of quality measures for eCQMs.

 Using the linked eCQM data set, we identified the elements specified by each quality indicator and, where more than 1 option existed, determined the optimal specification iteratively. Two measures, “No adjuvant chemotherapy for stage IA NSCLC” and “No radiation therapy for resected stage I, II NSCLC,” were excluded from analysis prior to eCQM creation because adherence, by chart abstraction, was greater than 99% with no variation seen across facilities.

eCQMs were considered feasible if electronically available data could specify each aspect of the measure. Validity was evaluated by comparing the sensitivity and specificity of the denominator and numerator for each measure, using chart abstraction as the gold standard, and by comparing overall pass rates between the electronic and the chart-abstracted versions of the measures. Although there is no accepted cut-point for numerator and denominator sensitivity and specificity in determining measure validity, nor an accepted level of agreement between pass rates that marks validity, we considered at least 80% specificity for both the denominator and the numerator (compared with chart abstraction data) as valid for an eCQM. However, quality measures with less than 80% specificity or sensitivity were assessed individually and considered likely valid as eCQMs if internal validity could be supported—even if results from measure use might not be comparable across data collection methodologies. Analyses were performed using SAS statistical software, version 9.1 (Cary, North Carolina).

Patient demographic characteristics were similar to national trends in lung and prostate cancer (

Even in a fully integrated healthcare system, only 28% of cancer quality measures could be validated by using electronically available data.

Ted A. Skolarus, MD, MPH

Articles

Catheter Management After Benign Transurethral Prostate Surgery: RAND/UCLA Appropriateness Criteria

Implications of Evolving Delivery System Reforms for Prostate Cancer Care

Validating Electronic Cancer Quality Measures at Veterans Health Administration