 To describe the prevalence and incidence of opioid and nonsteroidal anti-inflammatory drug (NSAID) use before and since the start of the Veterans Health Administration (VHA) Opioid Safety Initiative (OSI) and to assess rates of adverse events (AEs).

 The OSI began in August 2012 and was fully implemented by the end of fiscal year (FY) 2013. The study timeframe was categorized into baseline (FY 2011-2012), transition (FY 2013), and postimplementation (FY 2014-2015) phases. Prevalence and incidence rates were calculated for opioid and NSAID users by quarter between FY 2011 and FY 2015. For AEs among new users of an NSAID or opioid, Cox proportional hazards models with inverse probability weighting were used to adjust for potential confounding.

 There were 3,315,846 regular users of VHA care with at least 1 opioid and/or NSAID outpatient prescription between FYs 2011 and 2015. The quarterly opioid prevalence rate was approximately 21% during the baseline and transition phases, then decreased to 17.3% in the postimplementation phase. NSAID prevalence remained constant at about 16%. Opioid incidence rates gradually decreased (2.7% to 2.2%) during the study, whereas NSAID incidence rates remained about 2.2%. After inverse probability weighting, patients receiving opioids had a greater risk of cardiovascular events (hazard ratio [HR], 1.41; 95% CI, 1.36-1.47), acute kidney injury (HR, 2.60; 95% CI, 2.51-2.68), gastrointestinal bleeding (HR, 1.68; 95% CI, 1.56-1.81), and all-cause mortality (HR, 3.73; 95% CI, 3.60-3.87) than NSAID users.

 Opioid use declined following implementation of the OSI, whereas NSAID use remained constant. Rates of AEs were higher among opioid users, which provides additional rationale for efforts to use NSAIDs for pain management when appropriate.

The percentages of patients receiving new and continuing opioid prescriptions decreased with implementation of an opioid safety initiative (OSI), whereas the percentages of nonsteroidal anti-inflammatory drug (NSAID) users remained constant. However, patients receiving opioids had greater risks of cardiovascular events, acute kidney injury, gastrointestinal bleeding, and all-cause mortality than NSAID users.

Our concern was the potential for expanded NSAID use among patients at increased risk for NSAID-related adverse drug events (eg, older patients) as an unintended consequence of the OSI; it is reassuring that an increase in adverse events was not identified among incident NSAID users.

The results provide support for ongoing efforts to use nonopioid strategies, such as NSAIDs, for pain management as appropriate.

Further research is needed to examine the incidence of serious adverse outcomes, and causes of death, with opioids.

Opioid safety continues to receive widespread attention given the ongoing increase in the number of opioid-related deaths.

 In August 2012, the Veterans Health Administration (VHA) implemented its Opioid Safety Initiative (OSI) to improve the safe and effective use of opioids. This comprehensive initiative included implementation of a national dashboard to help providers identify patients at risk of serious adverse events (AEs) related to high-dose opioid use and practice guideline-concordant use of opioid therapy in the management of chronic pain. The OSI dashboard includes data on pharmacy patients dispensed an opioid, patients on long-term opioids who received a urine drug screen, patients who received an opioid and a benzodiazepine in the same quarter of a fiscal year (FY), and the average morphine-equivalent daily dose of opioids. In addition, the VHA National Pain Management Program Office developed an OSI Toolkit that includes advice for tapering opioids and benzodiazepines, nonpharmacologic and nonopioid treatment alternatives, consent for long-term opioid therapy, and patient educational materials.

 Given the emphasis on decreasing high-risk opioid use, prescribing of nonopioid analgesics, such as nonsteroidal anti-inflammatory drugs (NSAIDs), may have increased after the OSI. NSAIDs have their own risks, however, including adverse cardiovascular, renal, and gastrointestinal (GI) effects.

 Because the VHA patient population includes a high proportion of elderly patients with multiple comorbidities, NSAID-related AEs might increase if these medications are used inappropriately either to replace an existing opioid or to avoid initiating an opioid.

Various multifaceted interventions (eg, stakeholder involvement, education, and audit and feedback) have been effective in improving the safety of opioid prescribing.

 However, it is unclear how these safety initiatives may affect the utilization of other analgesics, such as NSAIDs, and the subsequent rate of AEs. We examined VHA databases to describe the prevalence and incidence of opioid and NSAID use and assess the rates of adverse outcomes typically associated with NSAIDs among incident opioid and NSAID users, before and since the OSI.

Study patients were 18 years or older and regular users of VHA care. We further identified patients with at least 1 prescription for an opioid or NSAID during the study timeframe of FY 2011 (October 1, 2010-September 30, 2011) through FY 2015. “Regular users” were defined as patients with at least 2 VHA outpatient visits and/or inpatient stays during the FY containing their index date (date of first opioid or NSAID prescription during the study period) and the prior FY. The OSI started in August 2012 and was fully implemented by the end of FY 2013; therefore, the study timeframe was categorized into baseline (FY 2011-2012), transition (FY 2013), and postimplementation (FY 2014-2015) phases (

). The study was approved by the institutional review board for the Hines/North Chicago VA Medical Centers; protected health information was used.

To characterize patients at the index date, we obtained data from the Outpatient and Inpatient National Medical SAS datasets during the year prior to the index date for demographics, VHA care utilization, and diagnosis codes. Comorbidities were defined using the Deyo et al adaptation of the Charlson Comorbidity Index

 and included other disease states that could influence opioid or NSAID use. We linked patient zip code to Federal Information Processing Standard county code and then mapped county code to the Area Health Resource File for describing the patient’s county of residence as urban, rural, or highly rural and classifying its Census region. Outpatient opioid and NSAID use and concomitant medications at index date were obtained from the VHA Pharmacy Benefits Management (PBM) Services database (version 3.0). Adverse outcomes typically associated with NSAIDs were primarily based on 

International Classification of Diseases, Ninth Revision

) codes from the National Medical SAS datasets; serum creatinine values from PBM laboratory data were also included to identify acute kidney injury (AKI). Mortality data were obtained from the Vital Status file.

 Prevalence (ie, continuing users) and incidence (ie, new users) rates were individually calculated for opioids and NSAIDs (

]) by quarter between FY 2011 and FY 2015. Quarterly-specific prevalent users were patients who had a supply of the drug(s) during the quarter of interest. Patients receiving an NSAID and opioid appeared in both groups. Quarterly-specific incident users were subsets of prevalent users with no opioids and/or NSAIDs during the year prior to the index date. The denominator was regular users of VHA care during both the FY containing the quarter of interest and the preceding FY.

 Adverse outcomes included cardiovascular events, AKI, GI bleeding requiring hospitalization or an emergency department (ED) visit, and all-cause mortality. Serious events associated with opioids (eg, overdose) were not included because the focus was on adverse outcomes commonly seen with NSAIDs. All were defined by 

 diagnosis codes associated with the hospitalization or ED visit, except death, and coronary revascularization was defined by Current Procedural Terminology and 

). To improve the identification of AKI not coded during an encounter, serum creatinine lab values were used as defined by Lafrance and Miller (eAppendix B).

For patients who were regular VHA users with a prescription(s) for opioids or NSAIDs during the study timeframe, we described patient characteristics at the index date. Analysis of variance or χ

 tests were used to compare differences across groups. Over the 5-year timeframe, we calculated the quarterly prevalence and incidence rates of opioid and NSAID users. We computed 95% CIs for all rates, and the lower and upper bounds were the same as the point estimates to the first decimal place, given the huge sample size. Therefore, we focused on rate changes with clinical significance rather than statistical significance.

AEs were identified among incident users of opioids or NSAIDs only when the patient was receiving the medication (ie, release date of the prescription + the day-supply). Results were calculated for the entire study period, at baseline prior to implementation of the OSI (FY 2011), and after implementation (FY 2014, to ensure time for follow-up). Only incident users were included, because prevalent users were tolerating the treatment.

 Patients were followed for a maximum of 1 year after the index date and were censored when a drug from the other group was initiated (eg, on an opioid, then NSAID started) or an AE occurred, including death. The exposure time was the number of days the patient received the medication during the follow-up period, which could vary across AEs. Patients could have multiple types of AEs, but they were censored when the first event of the same outcome occurred (eg, first admission for AKI). For the aggregate outcomes of total AEs and cardiovascular events, patients were censored when the first adverse outcome of any type or first cardiovascular event occurred, respectively. In sensitivity analyses, we extended the medication exposure window to 1.25 times the day-supply of the medication or the day-supply plus 15 days, whichever was less.

In order to address factors that may confound the effect of treatment on the occurrence of an AE, we used inverse probability of treatment weighting (IPW) with normalized weights.

 Weights were estimated using a logistic regression model, with receipt of an NSAID as the dependent variable and patient demographics, VHA healthcare utilization, comorbidities, concomitant medications, and FY at index date as independent variables. We assessed the range of weights; less than 0.2% were above 20. We then conducted a sensitivity analysis with weights truncated at 20 to prevent an inflated influence of outliers. The results of this analysis were very similar, with less than a 4% change in hazard ratio (HR) estimates (data not shown). After weighting, we assessed the balance of baseline characteristics between NSAID and opioid groups using standardized difference. A standard difference of less than 0.1 was considered negligible.

 Then, we used Cox proportional hazards models to assess the association between NSAID and opioid use and AEs, with and without application of IPW. We assessed whether the proportional hazards assumption held true. Analyses were carried out using SAS version 9.4 (SAS Institute Inc; Cary, North Carolina). Statistical tests were 2-sided, with a 

 value <.05 considered statistically significant.

Of the 3,315,846 opioid and/or NSAID users during the study period, 50.4% had prescriptions for opioids only at the index date; 42.3% received NSAIDs only, and 7.4% received both opioids and NSAIDs (

). Both prevalent and incident opioid and NSAID users had clinically similar characteristics; therefore, their characteristics are not presented separately in Table 1.

On average, opioid-only users were older than NSAID-only users (mean age of 62 vs 56 years) (Table 1). Patients in all groups were predominantly male (92%). Compared with NSAID users, opioid-only users had a higher proportion of white patients (67% vs 60%); more utilization of VHA care; a higher Charlson Comorbidity Index score (1.5 vs 0.7); greater proportions with diagnoses of cancer, AKI, GI bleeding, and cardiovascular comorbidities; and higher percentages receiving medications that could influence the decision to prescribe an opioid or NSAID (eg, potential decrease in NSAID use among patients receiving antithrombotic therapy). NSAID users had higher proportions with behavioral health conditions. The characteristics of opioid and NSAID users did not change much between FYs 2011 and 2015 (

Between FYs 2011 and 2015, VHA users increased annually from 5.5 million to 6 million. Of these, there were 3,986,683 (72% in FY 2011) to 4,392,545 (73% in FY 2015) patients who were regular users of VHA and 18 years or older who were assessed for opioid and NSAID use.

Opioid prevalence rates remained constant during the baseline phase (20.8% in the first quarter [Q1] of FY 2011), then started to gradually decrease during the transition phase in FY 2013 (

). Opioid prevalence decreased more sharply during the second year of the postimplementation phase to 17.3% by Q4 of FY 2015. From the beginning to the end of the study period, opioid prevalence decreased by 3.5%, or 16.8% of the original rate. NSAID prevalence essentially remained constant (15.8% in Q1 of FY 2011; 16.0% in Q4 of FY 2015).

The opioid incidence rate gradually decreased during the study period, from 2.7% in Q1 of FY 2011 to 2.2% in Q4 of FY 2015, a decrease of 0.5%, or 18.5% of the original rate (Figure 2). NSAID incidence remained relatively constant throughout the study period (2.2% in Q2 of FY 2011; 2.1% in Q4 of FY 2015).

Adverse Outcomes Among Incident Opioid and NSAID Users

There were no significant differences in the measured characteristics of incident opioid and NSAID users after applying IPW; all standardized differences were less than 0.1 (

). Of the 1,155,420 incident opioid-only users and 979,277 incident NSAID-only users in the entire study period, opioid users had higher incidence rates for all adverse outcomes than NSAID users, with unadjusted HRs ranging from 2.24 to 9.48 (

). The proportional hazards assumption was satisfied. After IPW, the hazard for all adverse outcomes evaluated (except acute coronary syndrome) remained significantly greater for incident opioid users, with HRs ranging from 1.32 to 3.73. Specifically, incidence rates for total AEs were 118 per 1000 person-years for opioid users versus 23 per 1000 person-years for NSAID users, with an unadjusted HR of 5.13 (95% CI, 4.97-5.28) and an HR of 2.05 (95% CI, 2.00-2.10) after IPW. The incidence rates for all-cause mortality were 85 per 1000 person-years for opioid users versus 9 per 1000 person-years for NSAID users, with an unadjusted HR of 9.48 (95% CI, 9.06-9.93) and an HR of 3.73 (95% CI, 3.60-3.87) after IPW. The results were similar in sensitivity analyses, although the incidence rates increased about 10% for both NSAID and opioid users when we extended the medication exposure window (data not shown). When comparing adverse outcomes in incident opioid versus NSAID users in FY 2011 with FY 2014, the HRs in both years after IPW were similar to the results from the entire sample (

Similar to studies on other initiatives addressing high-risk opioid use,

 we found that opioid prevalence and incidence rates declined following implementation of the VHA OSI. Mosher and colleagues identified a sharp increase in opioid prevalence (18.9% to 33.4%) from FY 2004 to FY 2012 and a modest increase in opioid incidence (8.7% to 9.6%) among patients with regular VHA medication use.

 Their rates and ours were essentially identical in the overlapping study years (ie, FY 2011 and FY 2012). Moving forward, we found that opioid prevalence rates started to gradually decrease during the transition phase in FY 2013 and dropped more sharply beginning in Q4 of FY 2014 post implementation of the OSI. The steeper decline could be related to some medical centers focusing more intently on opioid safety. These trends reflect the earlier emphasis on managing chronic pain and the more recent focus on promoting safe opioid use to guard against harm and abuse.

 Recent study findings show that the VHA OSI also led to decreases in the use of high-dose opioids and concurrent prescribing of opioids with benzodiazepines.

In contrast with opioid use, the prevalence and incidence of NSAID use remained constant, possibly indicating that fewer patients used opioids without moving to NSAIDs. The lack of an increase in prevalence and incidence after the OSI was unanticipated, especially because nonacetylated salicylates and selective cyclooxygenase-2 inhibitors were included as NSAIDs in our study. There are several possible explanations, including OTC NSAID use. Also, we did not assess whether pain went untreated or whether the use of other medications (eg, acetaminophen, duloxetine) or nonpharmacologic therapies (eg, complementary and alternative medicine, physical therapy) may have substituted for opioid use. Providers may have avoided NSAIDs in patients who were elderly; had a history of cardiovascular disease, AKI, or GI bleeding; or were taking medications that could interact with an NSAID. As seen in the baseline characteristics, a higher proportion of opioid users had these attributes, and the characteristics of opioid and NSAID users did not change over time.

Because the patients who received opioids differed from those who received NSAIDs and likely confounded the effect of treatment on the occurrence of an AE, we used IPW. There were no significant differences in the measured characteristics of incident opioid and NSAID users after applying IPW. Using methods similar to ours, Solomon and colleagues compared the safety outcomes of opioids, selective cyclooxygenase-2 inhibitors, and nonselective NSAIDs in elderly patients with arthritis.

 After propensity score matching, they also found that patients on opioids experienced a higher risk of cardiovascular events (HR, 1.77; 95% CI, 1.39-2.24), AKI (HR, 1.53; 95% CI, 1.12-2.09), and all-cause mortality (HR, 1.87; 95% CI, 1.39-2.53) in comparison with patients on nonselective NSAIDs.

 In 3 short-term randomized studies of celecoxib versus an opioid, a significantly higher percentage of patients in the opioid groups experienced AEs.

 However, no treatment-related serious AEs were reported in these studies.

We evaluated whether the risk of AEs with opioids versus NSAIDs changed from baseline to FY 2014. If higher-risk patients were receiving NSAIDs after implementation of the OSI, then the HRs would move toward the null or possibly indicate a decreased risk of AEs with opioids. However, the HRs were similar in both years, except that there was an increase in the hazard of AKI, all-cause mortality, and total AEs among opioid versus NSAID users in FY 2014. The increase could be related to a rise in the proportion of opioid users with more severe disease/pain in the years after implementation of the OSI because of the emphasis on decreasing opioid use. Those with more severe disease may be at greater risk of AEs, especially all-cause mortality. In addition, perhaps there was improved documentation of AEs among opioid users due to heightened awareness of the harms.

Although our findings of higher rates of cardiovascular, renal, and GI AEs with opioids are unexpected, they are consistent with prior reports. In addition to the study by Solomon and colleagues,

 LoCosale et al reported that among patients initiating chronic opioid therapy, rates of myocardial infarction (MI), stroke, and heart failure per 1000 person-years in the United States were 10.7, 9.3, and 37.2, respectively.

 Carman et al estimated incidence rate ratios (IRRs) for MI and MI/coronary revascularization in a cohort of chronic opioid users versus a matched cohort from the general population.

 The adjusted IRRs in opioid users versus the general cohort were 2.66 (95% CI, 2.30-3.08) and 2.39 (95% CI, 2.15-2.63) for MI and MI/coronary revascularization, respectively.

 In a nested case-control study using a UK General Practice Research Database, the authors found the odds of an MI were increased among current opioid users versus nonusers (adjusted odds ratio, 1.28; 95% CI,1.19-1.37).

 However, unmeasured confounding cannot be ruled out in these studies. Regarding adverse renal events, a recent review discussed the effects of opioids on the kidneys, including AKI and chronic kidney disease.

 Although the incidence is unknown, AKI due to opioids can result from dehydration, urinary retention, and rhabdomyolysis.

 Most dramatically, overall mortality was greater in incident opioid versus NSAID users, including the IPW cohorts. Other studies have found higher mortality rates in users of opioids relative to NSAIDs and other medications for pain; this was mainly due to cardiovascular and respiratory causes, as well as overdoses.

Although our study included a large national population of opioid and NSAID users with cohorts that were well balanced across measured baseline characteristics after IPW, there were limitations. We did not include AEs outside of the VHA. However, we only included regular VHA users and we do not anticipate differential use of non-VHA hospitals between cohorts. Although we included many characteristics in the IPW that may influence the choice of an opioid versus an NSAID, there could have been unmeasured residual confounding (eg, smoking, severity of illness); sicker patients, or patients with more severe pain, still may have received opioids. A high-dimensional propensity score may have helped with this issue and is worthwhile to consider for future analyses. In addition, while patients received the medications, it was not possible to know if and when they actually took them; opioids and NSAIDs are frequently used as needed. Therefore, patients may not have been taking the medications at the time of the outcomes, and there was an increase in the AE rates when we extended the medication exposure window beyond the day-supply in the sensitivity analysis. We also did not capture OTC NSAID use, which may have increased after implementation of the OSI. Finally, we did not consider the use of other medications and nonpharmacologic therapies that may have replaced opioids and contributed to a lower-than-anticipated use of NSAIDs.

We found that the rates of new and continuing opioid users decreased following nationwide implementation of the OSI in the VHA, while the rates of NSAID users remained constant. Our concern was the potential for expanded NSAID use among patients at increased risk for NSAID-related adverse drug events (eg, older patients) as an unintended consequence of the OSI. Thus, it is reassuring that we did not identify an increase in AEs among incident NSAID users. Indeed, our findings support ongoing efforts to use nonopioid strategies, such as NSAIDs, for pain management when appropriate. Further research is needed to compare the incidence of serious adverse outcomes among users of opioid and nonopioid medications, as well as the cause of death to determine the plausibility of the analgesic’s role.

 Kansas City VA Medical Center (VF), Kansas City, MO; VA Center for Medication Safety/Pharmacy Benefits Management Services (SLA, CBG, FEC), Hines, IL; VA Center for Health Equity Research and Promotion (SLA, XZ, CBG), Pittsburgh, PA; VA Center for Healthcare Organization and Implementation Research (DRM), Bedford, MA; Birmingham VA Medical Center (JAS), Birmingham, AL; University of Alabama at Birmingham (JAS), Birmingham, AL.

 There was no funding for this study. The work was supported by VA Pharmacy Benefits Management Services (Hines, IL), and VA Pittsburgh Healthcare System (Pittsburgh, PA). The views expressed in this paper are those of the authors, and no official endorsement by the Department of Veterans Affairs or the United States Government is intended or should be inferred.

 Dr Singh has consulted for Savient, Takeda, Regeneron, Merz, Iroko, Bioiberica, Crealta/Horizon, and Allergan (pharmaceuticals); WebMD; UBM, LLC; and the American College of Rheumatology. Dr Singh has also received grants from Takeda and Savient, is a member of the executive committee of OMERACT, and is an editor of the Cochran UAB satellite center. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (VF, SLA, XZ, DRM, JAS, CBG, FEC); acquisition of data (XZ, FEC); analysis and interpretation of data (VF, SLA, XZ, DRM, JAS, CBG, FEC); drafting of the manuscript (VF, SLA, CBG); critical revision of the manuscript for important intellectual content (SLA, DRM, JAS, CBG); statistical analysis (XZ, DRM); administrative, technical, or logistic support (JAS, FEC); and supervision (CBG).

 Sherrie Aspinall, PharmD, MSc, BCPS, VA Pittsburgh Healthcare System, University Dr (151C) Bldg 30, Pittsburgh, PA 15240. Email: sherrie.aspinall@va.gov.

1. Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid-involved overdose deaths — United States, 2010-2015 [erratum in 

. 2017;66(1):35. doi: 10.15585/mmwr.mm6601a10]. 

. 2016;65(50-51):1445-1452. doi: 10.15585/mmwr.mm655051e1.

2. Jones CM, Mack KA, Paulozzi LJ. Pharmaceutical overdose deaths, United States, 2010. 

. 2013;309(7):657-659. doi: 10.1001/jama.2013.272.

3. VHA Pain Management: Opioid Safety Initiative (OSI). US Department of Veterans Affairs website. va.gov/PAINMANAGEMENT/Opioid_Safety_Initiative_OSI.asp. Updated October 1, 2017. Accessed February 22, 2017.

Articles

WEB EXCLUSIVE

Opioid use incidence and prevalence rates decreased with implementation of an opioid safety initiative, whereas nonsteroidal anti-inflammatory drug rates remained constant. Rates of adverse events were higher among opioid users.

Trends in Opioid and Nonsteroidal Anti-Inflammatory Use and Adverse Events

 To characterize patterns of emergency department (ED) utilization for ambulatory care—sensitive conditions (ACSCs) among patients with established care within a patient-centered medical home.

 Retrospective chart review using Michigan Medicine’s (formerly University of Michigan Health System) electronic health record.

 Ten general medicine (GM) physicians reviewed 256 ambulatory care—sensitive ED encounters that occurred between January 1, 2014, and December 31, 2014, among patients of a GM medical home. Physician reviewers abstracted from the medical record the day and time of ED presentation and the source of ED referral (eg, patient self-referral vs physician referral). Physicians assessed the appropriateness of the care location (eg, ED vs primary care). Interrater reliability was assessed using the kappa statistic, and the χ

 test was used to assess differences in the appropriateness of the care location according to ED referral source.

 Compared with all other days of the week, the fewest number of ED visits occurred on weekend days, and nearly half of patients (48%) presented to the ED after daytime hours, which were defined as 8 am to 3:59 pm. The majority (n = 185; 72%) of patients were self-referred to the ED. The ED was considered the appropriate care location in more than half (53%) of the reviewed cases. Among the 119 cases considered appropriate for GM management, the majority (86%) were self-referred to the ED.

 Patients with ACSCs often presented to the ED without contacting their medical home. Frequently, the ED is the most appropriate location given symptoms at presentation.

Among policy makers and insurers, emergency department (ED) encounters for ambulatory care—sensitive conditions (ACSCs) are an indicator of primary care resources and accessibility within a community. The patient-centered medical home model is encouraged as a strategy to improve primary care access and decrease preventable ED encounters. However, this study identified 2 main limitations of this approach:

Patients frequently present to the ED for management of an ACSC without first contacting their medical home.

Emergent care is often required, given the patient’s presenting symptoms, and the ED may be the most appropriate care location.

Mrs Smith, a 50-year-old woman with a history of hypertension, noted redness and pain in her posterior left calf when she awoke in the morning. Over the course of the day, her pain increased and she observed swelling of the left lower extremity. She otherwise felt well, but recalling that her father had similar symptoms due to a blood clot, she presented to the emergency department (ED) at 3 pm for evaluation.

Was it appropriate for Mrs Smith to seek immediate care in the ED? Primary care providers and insurers may disagree. Among providers, Mrs Smith’s symptoms raise concern for a deep venous thrombosis (DVT), a potentially life-threatening condition that warrants emergent evaluation. In contrast, insurers use the clinical diagnosis at ED discharge, not the presenting symptoms, to determine whether emergent care was necessary. For example, the ED encounter would be considered appropriate if Mrs Smith was, in fact, found to have a DVT, but it would be considered inappropriate if she was diagnosed with an ambulatory care—sensitive condition (ACSC), such as cellulitis.

ACSCs are a heterogeneous set of acute and chronic conditions for which early and effective management in the primary care setting may prevent an ED encounter.

 As healthcare spending within the United States continues to rise,

 decreasing preventable ED encounters has been targeted as a potential means of cost containment.

 At least one-third of annual ED visits are nonurgent,

 and management of these cases in alternative settings, such as primary care clinics and urgent care centers, could save an estimated $4.4 billion per year.

 Insurers and policy makers consider ambulatory care—sensitive ED encounters to be an indicator of primary care resources within a community, and, increasingly, these encounters are used as a quality metric to guide third-party reimbursement.

Various approaches have been used to improve access to primary care resources and thereby reduce ambulatory care—sensitive ED encounters. The patient-centered medical home (PCMH) model, for example, strives to achieve “accessible, continuous, comprehensive, and coordinated”

 care through strategies such as extended clinic hours (eg, evenings and weekends), individualized care and disease management programs, and multidisciplinary care teams that integrate patients and families.

 and walk-in clinics in nonmedical facilities, like drug stores, have been implemented to decrease nonurgent ED visits.

Despite these clinical initiatives and national policies

 that aim to reduce preventable ED visits, little is known about patterns of ED utilization for ACSCs among established PCMH patients. For example, are ambulatory care—sensitive ED encounters more common at certain times of the day? How often do patients initiate primary care contact prior to ED presentation? How frequently is the ED the most appropriate care location given the patient’s presenting symptoms? Although others have described ED utilization rates among PCMH patients versus non-PCMH patients,

 these key questions have not been previously addressed. Yet their answers are critical to understanding the medical home’s potential effectiveness to reduce ambulatory care—sensitive ED visits. For example, patient-level factors such as convenience,

 and perceived lack of primary care access

 may drive ED utilization independent of PCMH access. Further, it can be clinically difficult for a provider to assess a condition as urgent or not before in-person evaluation, so immediate evaluation in the ED may be warranted to provide the most patient-centered care.

To study patterns of ED utilization of PCMH patients, we asked general medicine (GM) physicians within our large academic medical center, Michigan Medicine (formerly University of Michigan Health System), to review the records of patients who presented to the ED with an ACSC despite having access to a medical home. These physicians characterized: 1) patterns of ED utilization (eg, day of the week and time of day) among PCMH patients, 2) attempts made by PCMH patients to access their medical home prior to ED presentation, and 3) the appropriateness of the care location (eg, ED vs primary care). This study provides insight into the epidemiology of ambulatory care—sensitive ED visits and may inform future policies and initiatives that aim to reduce these encounters.

The University of Michigan Faculty Group Practice provides physician leadership and project management to the Michigan Primary Care Transformation Project (MiPCT), the largest PCMH initiative in the nation.

 Reductions in ED and inpatient use for ACSCs are a central goal of this initiative, and, in accordance with national PCMH guidelines, MiPCT offers at least 8 hours of extended access primary care clinic appointments per clinic per week. Specifically, clinics offer a combination of early morning weekday appointments (7 am to 9 am), late afternoon weekday appointments (5 pm to 8 pm), and Saturday appointments from 9 am to 12 pm. We conducted this locally motivated study to help inform national policies and similar primary care initiatives.

Using our institution’s electronic health record (EHR) data, we identified ED encounters by GM PCMH patients that occurred between January 1, 2014, and December 31, 2014. We were specifically interested in understanding the interactions that occur between patients and their medical home before ED presentation. Therefore, we limited our selection to include only patients with established primary care, which we defined as at least 2 primary care visits at our institution within 2 years of ED presentation; 1 of these visits must have occurred within 13 months of the ED encounter. We excluded patients who were seen in the psychiatric ED, as mental health diagnoses are independent risk factors for ambulatory care—sensitive ED encounters and reductions in these visits may require specific, targeted interventions.

 We also excluded patients who were seen in the pediatric ED or our Women’s Hospital to focus on ambulatory conditions with a higher probability of having an outpatient GM practice as the appropriate clinical location of care. Finally, we excluded patients with 3 or more ED visits in a year, as they represented a unique subset of patients using the ED and are currently assigned to a case manager in our system.

We limited our results to encounters deemed ambulatory care—sensitive (using the 

 diagnosis code for Blue Care Network’s 2012 list of ACSCs). Our chart selection and exclusion process are detailed in 

Ten physicians were involved in this study, and a random 12% sample of eligible patient charts was selected for review (n = 263). The reviewers had access to all clinical information in all settings (notes, labs, radiology, etc) in our integrated EHR. They documented the following information using the Qualtrics survey platform: day of ED presentation; time of ED presentation; reason for visit; ED referral source (eg, GM, subspecialist, patient self-referral), as documented in telephone and/or clinic and ED notes; patient hospital admission status; need for laboratory testing; and need for advanced imaging, such as computed tomography scan, magnetic resonance imaging, or angiography (plain films were not included, as these would be expected to be readily available in primary care offices). We defined daytime hours as those between 8 am and 3:59 pm and evening hours as those between 4 pm and 7:59 am, because clinic appointment schedules are usually full for the day for patients calling after 4 pm for an urgent issue. Finally, physician reviewers were asked to respond to 2 questions: 1) “Was the ED the appropriate care location?” and 2) “Would the case have been more appropriate for management in GM clinic?” If the ED was determined to be the most appropriate care location, physicians were asked to indicate why the case would have been inappropriate for management in the GM clinic.

 test to assess differences in the relative frequencies of hospitalization and the use of advanced imaging among patients who presented to the ED following GM referral versus patients who presented with the ED without GM referral. Additionally, we used the χ

 test to assess differences in the appropriateness of the care location according to ED referral source. A 

 value <.05 was considered statistically significant.

To examine variability in clinical assessments regarding the appropriateness of the care location, 65 (25%) of the 263 charts were reviewed in duplicate, and the Kappa statistic

 was calculated to assess interrater reliability.

No judgments were made regarding the clinical decision making of the ED providers or the appropriateness of the patient’s final disposition from the ED (ie, hospital admission or discharge).

The University of Michigan Institutional Review Board assessed this quality of care study as exempt from review.

Between January 2014 and December 2014, 2711 established GM patients were seen in our institution’s ED for acute care diagnoses that were classified as ambulatory care—sensitive. Of these, 461 (17%) were identified as high healthcare utilizers based on their having 3 or more ED visits in a year and their charts were excluded from the review process. Of the remaining 2250 patient charts, a random sample of 263 charts was selected for physician review. However, only 256 charts were included in our final analysis, given missing data for 7 charts (Figure 1).

The majority (n = 199; 78%) of ED encounters for ACSCs occurred on weekdays, with the greatest number of visits occurring on Friday (n = 44; 17%). Compared with all other days of the week, the fewest number of visits occurred on Sunday (n = 27; 11%) and Saturday (n = 30; 12%) (

). Nearly half of the patients (n = 123; 48%) presented to the ED outside of daytime hours (8 am to 3:59 pm).

The majority (n = 185; 72%) of patients were self-referred to the ED, which was determined by the absence of EHR-documented contact with their primary care medical home or another healthcare provider (eg, a subspecialist or surgeon). Forty-three patients (17%) were referred to the ED by a GM provider; 12 (5%) were sent directly from a primary care clinic visit and 31 (12%) were advised to seek immediate care after contacting the GM clinic. Twenty-three patients (9%) were referred to the ED by a subspecialist, and 5 patients (2%) presented to the ED through other means (eg, police escort or transfer from urgent care).

Most patients (n = 217; 85%) were not admitted to the hospital. However, as shown in the 

, the patients referred to the ED by GM (n = 43) were significantly more likely to be admitted to the hospital than those who were self-referred to the ED (26% vs 10%; 

 = .01). Similarly, most patients (n = 183; 71%) did not receive advanced imaging, but patients referred to the ED by GM were more likely to undergo advanced imaging than those who were self-referred to the ED, although this difference was not statistically significant (40% vs 26%; 

In 53% of the reviewed cases (n = 136), physician reviewers indicated that the ED was the most appropriate care location, and in 46% of the reviewed cases (n = 119) physician reviewers indicated that GM would have been a more appropriate care location. Of note, there were several instances in which physicians deemed both care locations to be appropriate (eg, urinary symptoms and fever) and several instances in which physicians deemed neither care location to be appropriate (eg, medication refill that could be handled over the phone). Physicians showed moderate agreement (kappa = 0.58) for the question “Was the ED visit appropriate?” and substantial agreement (kappa = 0.68) for the question “Would GM have been more appropriate?”

Physicians cited the following as reasons why GM management would not have been appropriate: acuity of condition (46%), clinic closed (13%), urgent laboratory testing required (4%), advanced imaging required (16%), and other (21%). Among reasons classified as “other,” physicians described scenarios where the patient required nonurgent care by a subspecialist (eg, nonurgent dialysis or need for dermatology evaluation) or the patient perceived that the severity of their condition required emergent care. For example, 1 patient had minor gastrointestinal bleeding that resolved prior to ED presentation and did not require emergent endoscopy. Another patient had recently been discharged from the hospital and believed she was experiencing the symptoms for which she had been instructed to seek emergent care.

Physicians reviewers identified 46% (n = 119) of ambulatory care—sensitive ED encounters as appropriate for the GM clinic. However, most (n = 102; 86%) of these patients were self-referred to the ED; 4% were referred by a subspecialist and 10% were referred by GM. The ED was more often considered the appropriate care location for GM-referred patients compared with self-referred patients (

 <.001). The GM clinic was more often considered the appropriate care location for self-referred patients compared with GM-referred patients (

Mrs Smith presented to the ED. A lower extremity Doppler was performed, which did not show evidence of a DVT. She was diagnosed with cellulitis, an ACSC, and she was discharged home with an oral antibiotic after a period of observation in the ED.

Mrs Smith’s case provides a practical and policy-relevant context for interpreting our study’s findings. First, some ED visits for ACSCs are warranted. Mrs Smith perceived the potential severity of her condition and appropriately sought emergent care, despite having a regular source of primary care. Second, even with optimized primary care availability and resources within the medical home, patients may still choose to bypass this system and instead seek care in the ED. Third, a condition is deemed “ambulatory care—sensitive” only after a thorough work-up has been completed and potentially life-threatening conditions have been excluded from the clinical differential. Without consideration for the presenting symptoms or circumstantial factors (eg, time of day) that led to the encounter, the ambulatory care–sensitive designation overestimates the number of ED visits that are truly preventable.

In our study, most patients (72%) who presented to the ED for management of an ACSC were self-referred and did not contact their medical home prior to ED presentation. Physicians identified nearly half (46%) of cases as appropriate for management in the GM clinic; however, the majority (86%) of these patients were self-referred to the ED. Physicians identified the ED as the most appropriate care location for more than half (53%) of the reviewed cases and cited the acuity of the condition as the main reason why management in the GM outpatient setting would have not been appropriate. In such circumstances, the ED may serve as an extension of the medical home—“a safety net for providers”

—and is the most appropriate care location,

This study is, to our knowledge, the first descriptive evaluation of ambulatory care—sensitive ED encounters among patients with established access to a medical home. Although the PCMH model emphasizes after-hours care as a means to reduce preventable ED visits, we found that over half (52%) of the reviewed encounters occurred during daytime hours (8 am to 3:59 pm) and fewer encounters took place on either weekend day compared with all other days of the week. This may indicate patient preference for the ED due to factors such as convenience, personal preference, and perception of need.

 Consistent with the findings of prior studies,

 we found that ED encounters for ACSCs may not be an accurate reflection of primary care accessibility, as patients often present to the ED without first seeking primary care.

 demonstrated that only 22% of patients sought care from their primary care provider before presenting to the ED, and only a minority of these individuals cited lack of primary care availability as the reason for the ED visit. Another study found that Medicaid patients used the ED as their usual source of care despite an adequate number of physicians within the study area and auto-assignment of a primary care physician to all Medicaid beneficiaries.

This retrospective study had some important limitations. First, we conducted our study in a single academic center and our findings may not generalize to all clinical environments. Second, GM physicians may have reviewed medical records of patients from their own practice or clinic site, which may have biased their assessment of the appropriateness of the care location. However, given the large number of ambulatory care—sensitive ED visits, the random nature of the chart selection process, and the low rate of GM referral to the ED, it is highly unlikely that a physician would have reviewed the chart of a known patient. Further, some physicians assessed the ED to be an inappropriate care location despite GM referral, suggesting that reviewers aimed to remain objective in their assessments. A final limitation of this study is that we did not have access to patient insurance information and therefore cannot assess known barriers or facilitators to care, such as co-pays or transportation.

Despite these limitations, this study provides new descriptive insight into the usage of ED services for ACSCs among established PCMH patients within a large academic medical center. Our findings suggest that increasing primary care accessibility may be insufficient to reduce ED visits for ACSCs, as most patients do not contact their medical home prior to ED presentation. Future work should explore the reasons PCMH patients seek care in the ED for conditions that may be appropriate for management in the primary care setting. Additionally, many cases deemed “ambulatory care—sensitive” do, in fact, require immediate care, and the ED is often the most appropriate care location. If policy makers and insurers wish to use ACSCs as a means to identify preventable ED visits, consideration must be given to the patient’s initial condition as well as the diagnostic evaluation process, not simply the discharge diagnosis.

The authors thank Susan Blitz, MD, for chart abstraction and Laura Petersen, MHSA, for providing assistance with references and manuscript editing.

 Division of General Medicine, Department of Internal Medicine (DH, LFM, LB, FJB, JC, ME, AF, YK, KK, TO, EO, HR, JM), and Division of General Pediatrics, Department of Pediatrics and Communicable Diseases (LB, KK, JM), University of Michigan Medical School, Ann Arbor, MI; Robert Wood Johnson Foundation Clinical Scholars Program, University of Michigan (DH), Ann Arbor, MI; Ann Arbor VA Medical Center (DH, JM), Ann Arbor, MI; University of Michigan Institute for Social Research (DH), Ann Arbor, MI; University of Michigan School of Public Health (LFM), Ann Arbor, MI.

 University of Michigan Division of General Medicine.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (LFM); acquisition of data including chart review (LFM, LB, FJB, ME, AF, YK, KK, TO, EO); analysis and interpretation of data (DH, LFM, FJB, JC, ME, AF, YK, KK, TO, EO, HR, JM); drafting of the manuscript (DH, LFM, HR); critical revision of the manuscript for important intellectual content (DH, LFM, LB, ME, AF, YK, TO, EO, JM); statistical analysis (HR); provision of patients or study materials (FJB, KK); obtaining funding (LFM); administrative, technical, or logistic support (LFM, JM); and supervision (LFM, JM).

Patients often self-refer to the emergency department (ED) for management of an ambulatory care–sensitive condition, and the ED may be the most appropriate care location.

Ambulatory Care–Sensitive Emergency Visits Among Patients With Medical Home Access

To provide an overview of alternative payment models (APMs) and describe how leading national organizations involved with oncology care and payment are linking quality improvement initiatives and payment reform.

 For this review, we summarized the literature on APMs and their goals of improving healthcare quality while jointly controlling the cost of care. We described the types of APMs that have been examined in the real-world setting, specifically in the area of oncology, and how they have affected the quality of oncology care.

Currently, the following types of APMs are actively being explored by public- and private-sector insurers, specifically in oncology: accountable care organizations, bundled payments, clinical pathways, and patient-centered medical homes. To a great extent, the driving force behind implementing APMs tied to quality can be attributed to the initiatives of several leading national organizations, including the National Academy of Medicine, the American Society of Clinical Oncology, the National Committee for Quality Assurance, HHS, and CMS. Real-world evidence of APMs shows that progress is being made toward improving the quality of oncology care in the United States while simultaneously reducing costs.

The effective pairing of quality initiatives with healthcare reimbursement structures will likely be key to the long-term success of such APMs.

Examples of alternative payment models (APMs) being implemented in real-world settings include accountable care organizations, bundled payments, clinical pathways, and patient-centered medical homes.

The driving force behind APMs tied to quality can be attributed to several leading national organizations that are involved in developing and endorsing quality measures, developing guidelines and care improvement models, accrediting and certifying providers and health plans, and using data to monitor outcomes and for public reporting.

Although sparse and lacking in many important reported health outcomes, real-world evidence of APMs shows that progress is being made toward improving the quality of oncology care in the United States while simultaneously reducing costs.

As of January 2014, 14.5 million Americans with a history of cancer were alive. By 2024, this number of cancer survivors is projected to increase to 19 million.

 In 2010, the estimated cost of cancer care in the United States was $125 billion, which is projected to increase by 38% to $173 billion in 2020.

 This rising cost of cancer care has far outpaced US overall inflation rates, which hovered between 0.8% and 1.1% from 2014 to 2016.

 The American Society of Clinical Oncology (ASCO) provided a synopsis of the many challenges currently facing the US cancer care system in its 2016 State of Cancer Care in America report.

 In brief, there has been progress made in the care of patients with cancer, with new drugs approved and new tests for the diagnosis and management of patients with cancer alongside improvements in 5-year survival rates for many types of cancer.

 However, growth in the number of new patients with cancer and survivors, inequities across racial and ethnic groups, and disparities between rural versus urban regions remain, and exponentially increasing cancer care costs have put the system in crisis.

 It is also of great concern that variations in healthcare delivery across different sites of care can lead to diverse outcomes.

A 2013 report from the Institute of Medicine (now known as the National Academy of Medicine [NAM]), “Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis,” states that cancer care often is not patient-centered nor evidence-based and that many patients do not receive palliative care.

 NAM defines healthcare quality as the degree to which health services and technologies for individuals and populations increase the likelihood of evidence-based desired health outcomes.

 To improve the quality of cancer care, a component of the NAM conceptual framework is that payers should transition to new payment models that demonstrate both increased quality and affordability.

 To improve quality and stem the rising cost of cancer care, multiple public and private payers have been experimenting with alternative payment models (APMs) in oncology care in recent years. The primary goal of oncology-specific APMs is to link high-quality cancer care with payment reform. To a great extent, the driving force behind implementing APMs tied to quality can be attributed to the initiatives of several national leading organizations, including those of NAM, ASCO, the National Committee for Quality Assurance (NCQA), HHS, and CMS. These organizations have been involved collaboratively with 1 or more of the following initiatives: developing and endorsing quality measures, developing guidelines and care improvement models, accrediting and certifying providers and health plans, and using data to monitor outcomes and for public reporting.

The objectives of this review were to provide an overview of APMs and to describe how leading national organizations instrumental in oncology care and payment are involved with linking quality improvement initiatives and payment reform. Additionally, we present real-world applications of APMs in the area of oncology and how they have complemented quality improvement with payment reform.

Shift Toward Tying Payments to Quality and Value

APMs have the goals of improving healthcare quality while jointly controlling the cost of care.

In January 2015, HHS announced 2 internal goals: 1) By the end of 2016, tie 30% of Medicare payments to quality or value through APMs, and 2) tie 85% of Medicare fee-for-service (FFS) payments to quality or value.

 HHS has invited private payers to match or exceed these internal goals of Medicare.

 In April 2015, the Medicare Access and CHIP Reauthorization Act (MACRA) was passed to help achieve the HHS goals. MACRA changes how Medicare pays providers in 3 ways:

1) It ends the Sustainable Growth Rate formula for determining Medicare payments, 2) it creates a new framework for rewarding healthcare providers for providing quality care, and 3) it combines the existing quality reporting programs into 1 new system.

MACRA’s goals are to more rapidly achieve paying for value and better care and to make it easier to participate in the CMS quality programs with the Merit-Based Incentive Payment System (MIPS) or APMs, scheduled for implementation in January of 2019 (

 The performance measuring period for determining MIPS payments began in 2017.

 MIPS consolidates 3 existing programs, Meaningful Use, the Physician Quality Reporting System, and the Value-Based Payment Modifier, into a single program

 and will assess individual physician performance in 4 categories: quality, resource use, meaningful use of certified electronic health record (EHR) technology, and clinical practice improvement activities.

 APMs are defined as any of the following under MACRA: 1) an innovative payment model expanded under the Center for Medicare and Medicaid Innovation (CMMI), including Comprehensive Primary Care initiative participants, but not Health Care Innovation Award recipients; 2) a Medicare Shared Savings Program Accountable Care Organization (ACO); and 3) participants in the Medicare Health Care Quality Demonstration Program or Medicare Acute Care Episode Demonstration Program or another demonstration program required by federal law.

A subset of APMs (ie, Advanced APMs) will be eligible to earn incentive payments and be exempt from MIPS reporting requirements under the Quality Payment Program of MACRA.

 Advanced APM participants must use quality measures comparable to those of MIPS, use certified EHR technology, bear more than “nominal financial risk” or be a medical home expanded under CMMI, and have increasing percentages of payments linked to value through Medicare or all-payer APMs.

The CMMI recently implemented the Oncology Care Model (OCM), which, under MACRA, is considered an APM with potential for qualifying as an Advanced APM.

 The OCM Medicare FFS model incorporates a 2-part payment system for participating practices: 1) a monthly $160 per beneficiary care management payment and 2) a performance-based payment for episodes of chemotherapy care.

 Practices utilizing the OCM will have to provide these core functions

: patient navigation; documenting a care plan that contains the 13 components in the Care Management Plan outlined in the previously mentioned NAM report

; providing patient access to a clinician 24 hours a day, 7 days a week; treating patients with therapies consistent with nationally recognized guidelines; using data to drive continuous quality improvement; and using a certified EHR.

Under the OCM, quality is measured by the degree to which practices provide such services in efforts to increase the chances of achieving desired health outcomes.

 In March 2016, CMS started testing the OCM to evaluate the impact of a shift in oncology payments from FFS to fee-for-value.

 The OCM started July 1, 2016, and will run through June 30, 2021. As of February 2018, 191 practices and 14 payers were participating in the OCM.

ASCO has critiqued the OCM in that it is a hybrid model and does not totally replace the FFS reimbursement policy

 and has proposed the Patient-Centered Oncology Payment (PCOP) model.

 The PCOP APM will provide supplemental nonvisit payments to oncology practices performing the following activities

: new patient treatment planning, care management during treatment, care management during active monitoring, and participation in clinical trials.

These payments are in addition to payments under the Medicare Physician Fee Schedule for evaluation and management services, infusions of chemotherapy, advanced care planning, testing and imaging, and other procedures and services received by patients.

 Participating practices would also continue to be paid for drugs provided to patients.

 Oncology practices that participate in the PCOP system would be accountable for providing high-quality evidence-based care.

 ASCO and CMS are working together to develop quality measures, specifically in the area of oncology, and to ensure the design of the PCOP is aligned with the objectives of MACRA.

ACOs are healthcare providers or systems accountable to a third-party payer for the overall care, quality, and cost for a population of beneficiaries.

 The ACO APM still involves FFS payments to clinicians/providers, but via changes made in care delivery to ensure a coordinated continuum of care, a shared savings framework is built.

 To receive a portion of the shared savings, providers are monitored and held accountable for the quality of care they deliver.

 An example of an ACO in oncology is the partnership between Aetna, Texas Oncology, and the Innovent Oncology Program (McKesson Specialty Health), which launched 3 programs: implementation of level I pathways, which are treatment guidelines for an evidence-based oncology program; patient support services; and advance care planning.

 Patient support services include a telephone nursing intervention program to support patients receiving chemotherapy.

 The goals of these programs are to demonstrate that applying evidence-based guidelines pathways to cancer care can help reduce variability in care and lead to equal or better health outcomes at lower costs.

Overview of alternative payment models and how leading national organizations are involved with linking quality improvement initiatives and payment reform.

March 2018

Trends in Opioid and Nonsteroidal Anti-Inflammatory Use and Adverse Events

Ambulatory Care–Sensitive Emergency Visits Among Patients With Medical Home Access

Improving Quality of Care in Oncology Through Healthcare Payment Reform

Development and Implementation of an Academic Cancer Therapy Stewardship Program

Incorporating Value Into Physician Payment and Patient Cost Sharing

False-Positive Mammography and Its Association With Health Service Use

Assessing Medical Home Mechanisms: Certification, Asthma Education, and Outcomes

Overuse and Insurance Plan Type in a Privately Insured Population

Patients Discharged From the Emergency Department After Referral for Hospitalist Admission

Patient-Reported Denials, Appeals, and Complaints: Associations With Overall Plan Ratings