Trends in Opioid and Nonsteroidal Anti-Inflammatory Use and Adverse Events

March 15, 2018
Veronica Fassio, PharmD

Sherrie L. Aspinall, PharmD, MSc

Xinhua Zhao, PhD

Donald R. Miller, ScD

Jasvinder A. Singh, MD, MPH

Chester B. Good, MD, MPH

Francesca E. Cunningham, PharmD

Volume 24, Issue 3

Opioid use incidence and prevalence rates decreased with implementation of an opioid safety initiative, whereas nonsteroidal anti-inflammatory drug rates remained constant. Rates of adverse events were higher among opioid users.


Objectives: To describe the prevalence and incidence of opioid and nonsteroidal anti-inflammatory drug (NSAID) use before and since the start of the Veterans Health Administration (VHA) Opioid Safety Initiative (OSI) and to assess rates of adverse events (AEs).

Study Design: Historical cohort study.

Methods: The OSI began in August 2012 and was fully implemented by the end of fiscal year (FY) 2013. The study timeframe was categorized into baseline (FY 2011-2012), transition (FY 2013), and postimplementation (FY 2014-2015) phases. Prevalence and incidence rates were calculated for opioid and NSAID users by quarter between FY 2011 and FY 2015. For AEs among new users of an NSAID or opioid, Cox proportional hazards models with inverse probability weighting were used to adjust for potential confounding.

Results: There were 3,315,846 regular users of VHA care with at least 1 opioid and/or NSAID outpatient prescription between FYs 2011 and 2015. The quarterly opioid prevalence rate was approximately 21% during the baseline and transition phases, then decreased to 17.3% in the postimplementation phase. NSAID prevalence remained constant at about 16%. Opioid incidence rates gradually decreased (2.7% to 2.2%) during the study, whereas NSAID incidence rates remained about 2.2%. After inverse probability weighting, patients receiving opioids had a greater risk of cardiovascular events (hazard ratio [HR], 1.41; 95% CI, 1.36-1.47), acute kidney injury (HR, 2.60; 95% CI, 2.51-2.68), gastrointestinal bleeding (HR, 1.68; 95% CI, 1.56-1.81), and all-cause mortality (HR, 3.73; 95% CI, 3.60-3.87) than NSAID users.

Conclusions: Opioid use declined following implementation of the OSI, whereas NSAID use remained constant. Rates of AEs were higher among opioid users, which provides additional rationale for efforts to use NSAIDs for pain management when appropriate.

Am J Manag Care. 2018;24(3):e61-e72Takeaway Points

The percentages of patients receiving new and continuing opioid prescriptions decreased with implementation of an opioid safety initiative (OSI), whereas the percentages of nonsteroidal anti-inflammatory drug (NSAID) users remained constant. However, patients receiving opioids had greater risks of cardiovascular events, acute kidney injury, gastrointestinal bleeding, and all-cause mortality than NSAID users.

  • Our concern was the potential for expanded NSAID use among patients at increased risk for NSAID-related adverse drug events (eg, older patients) as an unintended consequence of the OSI; it is reassuring that an increase in adverse events was not identified among incident NSAID users.
  • The results provide support for ongoing efforts to use nonopioid strategies, such as NSAIDs, for pain management as appropriate.
  • Further research is needed to examine the incidence of serious adverse outcomes, and causes of death, with opioids.

Opioid safety continues to receive widespread attention given the ongoing increase in the number of opioid-related deaths.1,2 In August 2012, the Veterans Health Administration (VHA) implemented its Opioid Safety Initiative (OSI) to improve the safe and effective use of opioids. This comprehensive initiative included implementation of a national dashboard to help providers identify patients at risk of serious adverse events (AEs) related to high-dose opioid use and practice guideline-concordant use of opioid therapy in the management of chronic pain. The OSI dashboard includes data on pharmacy patients dispensed an opioid, patients on long-term opioids who received a urine drug screen, patients who received an opioid and a benzodiazepine in the same quarter of a fiscal year (FY), and the average morphine-equivalent daily dose of opioids. In addition, the VHA National Pain Management Program Office developed an OSI Toolkit that includes advice for tapering opioids and benzodiazepines, nonpharmacologic and nonopioid treatment alternatives, consent for long-term opioid therapy, and patient educational materials.3 Given the emphasis on decreasing high-risk opioid use, prescribing of nonopioid analgesics, such as nonsteroidal anti-inflammatory drugs (NSAIDs), may have increased after the OSI. NSAIDs have their own risks, however, including adverse cardiovascular, renal, and gastrointestinal (GI) effects.4-9 Because the VHA patient population includes a high proportion of elderly patients with multiple comorbidities, NSAID-related AEs might increase if these medications are used inappropriately either to replace an existing opioid or to avoid initiating an opioid.

Various multifaceted interventions (eg, stakeholder involvement, education, and audit and feedback) have been effective in improving the safety of opioid prescribing.10-14 However, it is unclear how these safety initiatives may affect the utilization of other analgesics, such as NSAIDs, and the subsequent rate of AEs. We examined VHA databases to describe the prevalence and incidence of opioid and NSAID use and assess the rates of adverse outcomes typically associated with NSAIDs among incident opioid and NSAID users, before and since the OSI.


Study Setting and Population

Study patients were 18 years or older and regular users of VHA care. We further identified patients with at least 1 prescription for an opioid or NSAID during the study timeframe of FY 2011 (October 1, 2010-September 30, 2011) through FY 2015. “Regular users” were defined as patients with at least 2 VHA outpatient visits and/or inpatient stays during the FY containing their index date (date of first opioid or NSAID prescription during the study period) and the prior FY. The OSI started in August 2012 and was fully implemented by the end of FY 2013; therefore, the study timeframe was categorized into baseline (FY 2011-2012), transition (FY 2013), and postimplementation (FY 2014-2015) phases (Figure 1). The study was approved by the institutional review board for the Hines/North Chicago VA Medical Centers; protected health information was used.

Data Collection and Source

To characterize patients at the index date, we obtained data from the Outpatient and Inpatient National Medical SAS datasets during the year prior to the index date for demographics, VHA care utilization, and diagnosis codes. Comorbidities were defined using the Deyo et al adaptation of the Charlson Comorbidity Index15 and included other disease states that could influence opioid or NSAID use. We linked patient zip code to Federal Information Processing Standard county code and then mapped county code to the Area Health Resource File for describing the patient’s county of residence as urban, rural, or highly rural and classifying its Census region. Outpatient opioid and NSAID use and concomitant medications at index date were obtained from the VHA Pharmacy Benefits Management (PBM) Services database (version 3.0). Adverse outcomes typically associated with NSAIDs were primarily based on International Classification of Diseases, Ninth Revision (ICD-9) codes from the National Medical SAS datasets; serum creatinine values from PBM laboratory data were also included to identify acute kidney injury (AKI). Mortality data were obtained from the Vital Status file.

Outcome Measures

Prevalence and incidence rates. Prevalence (ie, continuing users) and incidence (ie, new users) rates were individually calculated for opioids and NSAIDs (eAppendix A [eAppendices available at]) by quarter between FY 2011 and FY 2015. Quarterly-specific prevalent users were patients who had a supply of the drug(s) during the quarter of interest. Patients receiving an NSAID and opioid appeared in both groups. Quarterly-specific incident users were subsets of prevalent users with no opioids and/or NSAIDs during the year prior to the index date. The denominator was regular users of VHA care during both the FY containing the quarter of interest and the preceding FY.

Adverse outcomes. Adverse outcomes included cardiovascular events, AKI, GI bleeding requiring hospitalization or an emergency department (ED) visit, and all-cause mortality. Serious events associated with opioids (eg, overdose) were not included because the focus was on adverse outcomes commonly seen with NSAIDs. All were defined by ICD-9 diagnosis codes associated with the hospitalization or ED visit, except death, and coronary revascularization was defined by Current Procedural Terminology and ICD-9 procedure codes (eAppendix B). To improve the identification of AKI not coded during an encounter, serum creatinine lab values were used as defined by Lafrance and Miller (eAppendix B).16

Statistical Analyses

For patients who were regular VHA users with a prescription(s) for opioids or NSAIDs during the study timeframe, we described patient characteristics at the index date. Analysis of variance or χ2 tests were used to compare differences across groups. Over the 5-year timeframe, we calculated the quarterly prevalence and incidence rates of opioid and NSAID users. We computed 95% CIs for all rates, and the lower and upper bounds were the same as the point estimates to the first decimal place, given the huge sample size. Therefore, we focused on rate changes with clinical significance rather than statistical significance.

AEs were identified among incident users of opioids or NSAIDs only when the patient was receiving the medication (ie, release date of the prescription + the day-supply). Results were calculated for the entire study period, at baseline prior to implementation of the OSI (FY 2011), and after implementation (FY 2014, to ensure time for follow-up). Only incident users were included, because prevalent users were tolerating the treatment.17 Patients were followed for a maximum of 1 year after the index date and were censored when a drug from the other group was initiated (eg, on an opioid, then NSAID started) or an AE occurred, including death. The exposure time was the number of days the patient received the medication during the follow-up period, which could vary across AEs. Patients could have multiple types of AEs, but they were censored when the first event of the same outcome occurred (eg, first admission for AKI). For the aggregate outcomes of total AEs and cardiovascular events, patients were censored when the first adverse outcome of any type or first cardiovascular event occurred, respectively. In sensitivity analyses, we extended the medication exposure window to 1.25 times the day-supply of the medication or the day-supply plus 15 days, whichever was less.

In order to address factors that may confound the effect of treatment on the occurrence of an AE, we used inverse probability of treatment weighting (IPW) with normalized weights.18 Weights were estimated using a logistic regression model, with receipt of an NSAID as the dependent variable and patient demographics, VHA healthcare utilization, comorbidities, concomitant medications, and FY at index date as independent variables. We assessed the range of weights; less than 0.2% were above 20. We then conducted a sensitivity analysis with weights truncated at 20 to prevent an inflated influence of outliers. The results of this analysis were very similar, with less than a 4% change in hazard ratio (HR) estimates (data not shown). After weighting, we assessed the balance of baseline characteristics between NSAID and opioid groups using standardized difference. A standard difference of less than 0.1 was considered negligible.19 Then, we used Cox proportional hazards models to assess the association between NSAID and opioid use and AEs, with and without application of IPW. We assessed whether the proportional hazards assumption held true. Analyses were carried out using SAS version 9.4 (SAS Institute Inc; Cary, North Carolina). Statistical tests were 2-sided, with a P value <.05 considered statistically significant.


Patient Characteristics

Of the 3,315,846 opioid and/or NSAID users during the study period, 50.4% had prescriptions for opioids only at the index date; 42.3% received NSAIDs only, and 7.4% received both opioids and NSAIDs (Table 1 [part A, part B, and part C]). Both prevalent and incident opioid and NSAID users had clinically similar characteristics; therefore, their characteristics are not presented separately in Table 1.

On average, opioid-only users were older than NSAID-only users (mean age of 62 vs 56 years) (Table 1). Patients in all groups were predominantly male (92%). Compared with NSAID users, opioid-only users had a higher proportion of white patients (67% vs 60%); more utilization of VHA care; a higher Charlson Comorbidity Index score (1.5 vs 0.7); greater proportions with diagnoses of cancer, AKI, GI bleeding, and cardiovascular comorbidities; and higher percentages receiving medications that could influence the decision to prescribe an opioid or NSAID (eg, potential decrease in NSAID use among patients receiving antithrombotic therapy). NSAID users had higher proportions with behavioral health conditions. The characteristics of opioid and NSAID users did not change much between FYs 2011 and 2015 (eAppendix C).

Prevalence and Incidence Rates

Between FYs 2011 and 2015, VHA users increased annually from 5.5 million to 6 million. Of these, there were 3,986,683 (72% in FY 2011) to 4,392,545 (73% in FY 2015) patients who were regular users of VHA and 18 years or older who were assessed for opioid and NSAID use.

Opioid prevalence rates remained constant during the baseline phase (20.8% in the first quarter [Q1] of FY 2011), then started to gradually decrease during the transition phase in FY 2013 (Figure 2). Opioid prevalence decreased more sharply during the second year of the postimplementation phase to 17.3% by Q4 of FY 2015. From the beginning to the end of the study period, opioid prevalence decreased by 3.5%, or 16.8% of the original rate. NSAID prevalence essentially remained constant (15.8% in Q1 of FY 2011; 16.0% in Q4 of FY 2015).

The opioid incidence rate gradually decreased during the study period, from 2.7% in Q1 of FY 2011 to 2.2% in Q4 of FY 2015, a decrease of 0.5%, or 18.5% of the original rate (Figure 2). NSAID incidence remained relatively constant throughout the study period (2.2% in Q2 of FY 2011; 2.1% in Q4 of FY 2015).

Adverse Outcomes Among Incident Opioid and NSAID Users

There were no significant differences in the measured characteristics of incident opioid and NSAID users after applying IPW; all standardized differences were less than 0.1 (Table 2 [part A, part B, and part C]). Of the 1,155,420 incident opioid-only users and 979,277 incident NSAID-only users in the entire study period, opioid users had higher incidence rates for all adverse outcomes than NSAID users, with unadjusted HRs ranging from 2.24 to 9.48 (Table 3A). The proportional hazards assumption was satisfied. After IPW, the hazard for all adverse outcomes evaluated (except acute coronary syndrome) remained significantly greater for incident opioid users, with HRs ranging from 1.32 to 3.73. Specifically, incidence rates for total AEs were 118 per 1000 person-years for opioid users versus 23 per 1000 person-years for NSAID users, with an unadjusted HR of 5.13 (95% CI, 4.97-5.28) and an HR of 2.05 (95% CI, 2.00-2.10) after IPW. The incidence rates for all-cause mortality were 85 per 1000 person-years for opioid users versus 9 per 1000 person-years for NSAID users, with an unadjusted HR of 9.48 (95% CI, 9.06-9.93) and an HR of 3.73 (95% CI, 3.60-3.87) after IPW. The results were similar in sensitivity analyses, although the incidence rates increased about 10% for both NSAID and opioid users when we extended the medication exposure window (data not shown). When comparing adverse outcomes in incident opioid versus NSAID users in FY 2011 with FY 2014, the HRs in both years after IPW were similar to the results from the entire sample (Table 3B and Table 3C).


Similar to studies on other initiatives addressing high-risk opioid use,10-14 we found that opioid prevalence and incidence rates declined following implementation of the VHA OSI. Mosher and colleagues identified a sharp increase in opioid prevalence (18.9% to 33.4%) from FY 2004 to FY 2012 and a modest increase in opioid incidence (8.7% to 9.6%) among patients with regular VHA medication use.20 Their rates and ours were essentially identical in the overlapping study years (ie, FY 2011 and FY 2012). Moving forward, we found that opioid prevalence rates started to gradually decrease during the transition phase in FY 2013 and dropped more sharply beginning in Q4 of FY 2014 post implementation of the OSI. The steeper decline could be related to some medical centers focusing more intently on opioid safety. These trends reflect the earlier emphasis on managing chronic pain and the more recent focus on promoting safe opioid use to guard against harm and abuse.21,22 Recent study findings show that the VHA OSI also led to decreases in the use of high-dose opioids and concurrent prescribing of opioids with benzodiazepines.23

In contrast with opioid use, the prevalence and incidence of NSAID use remained constant, possibly indicating that fewer patients used opioids without moving to NSAIDs. The lack of an increase in prevalence and incidence after the OSI was unanticipated, especially because nonacetylated salicylates and selective cyclooxygenase-2 inhibitors were included as NSAIDs in our study. There are several possible explanations, including OTC NSAID use. Also, we did not assess whether pain went untreated or whether the use of other medications (eg, acetaminophen, duloxetine) or nonpharmacologic therapies (eg, complementary and alternative medicine, physical therapy) may have substituted for opioid use. Providers may have avoided NSAIDs in patients who were elderly; had a history of cardiovascular disease, AKI, or GI bleeding; or were taking medications that could interact with an NSAID. As seen in the baseline characteristics, a higher proportion of opioid users had these attributes, and the characteristics of opioid and NSAID users did not change over time.

Because the patients who received opioids differed from those who received NSAIDs and likely confounded the effect of treatment on the occurrence of an AE, we used IPW. There were no significant differences in the measured characteristics of incident opioid and NSAID users after applying IPW. Using methods similar to ours, Solomon and colleagues compared the safety outcomes of opioids, selective cyclooxygenase-2 inhibitors, and nonselective NSAIDs in elderly patients with arthritis.9 After propensity score matching, they also found that patients on opioids experienced a higher risk of cardiovascular events (HR, 1.77; 95% CI, 1.39-2.24), AKI (HR, 1.53; 95% CI, 1.12-2.09), and all-cause mortality (HR, 1.87; 95% CI, 1.39-2.53) in comparison with patients on nonselective NSAIDs.9 In 3 short-term randomized studies of celecoxib versus an opioid, a significantly higher percentage of patients in the opioid groups experienced AEs.24,25 However, no treatment-related serious AEs were reported in these studies.24,25

We evaluated whether the risk of AEs with opioids versus NSAIDs changed from baseline to FY 2014. If higher-risk patients were receiving NSAIDs after implementation of the OSI, then the HRs would move toward the null or possibly indicate a decreased risk of AEs with opioids. However, the HRs were similar in both years, except that there was an increase in the hazard of AKI, all-cause mortality, and total AEs among opioid versus NSAID users in FY 2014. The increase could be related to a rise in the proportion of opioid users with more severe disease/pain in the years after implementation of the OSI because of the emphasis on decreasing opioid use. Those with more severe disease may be at greater risk of AEs, especially all-cause mortality. In addition, perhaps there was improved documentation of AEs among opioid users due to heightened awareness of the harms.

Although our findings of higher rates of cardiovascular, renal, and GI AEs with opioids are unexpected, they are consistent with prior reports. In addition to the study by Solomon and colleagues,9 LoCosale et al reported that among patients initiating chronic opioid therapy, rates of myocardial infarction (MI), stroke, and heart failure per 1000 person-years in the United States were 10.7, 9.3, and 37.2, respectively.26 Carman et al estimated incidence rate ratios (IRRs) for MI and MI/coronary revascularization in a cohort of chronic opioid users versus a matched cohort from the general population.27 The adjusted IRRs in opioid users versus the general cohort were 2.66 (95% CI, 2.30-3.08) and 2.39 (95% CI, 2.15-2.63) for MI and MI/coronary revascularization, respectively.27 In a nested case-control study using a UK General Practice Research Database, the authors found the odds of an MI were increased among current opioid users versus nonusers (adjusted odds ratio, 1.28; 95% CI,1.19-1.37).28 However, unmeasured confounding cannot be ruled out in these studies. Regarding adverse renal events, a recent review discussed the effects of opioids on the kidneys, including AKI and chronic kidney disease.29 Although the incidence is unknown, AKI due to opioids can result from dehydration, urinary retention, and rhabdomyolysis.29 Most dramatically, overall mortality was greater in incident opioid versus NSAID users, including the IPW cohorts. Other studies have found higher mortality rates in users of opioids relative to NSAIDs and other medications for pain; this was mainly due to cardiovascular and respiratory causes, as well as overdoses.9,30


Although our study included a large national population of opioid and NSAID users with cohorts that were well balanced across measured baseline characteristics after IPW, there were limitations. We did not include AEs outside of the VHA. However, we only included regular VHA users and we do not anticipate differential use of non-VHA hospitals between cohorts. Although we included many characteristics in the IPW that may influence the choice of an opioid versus an NSAID, there could have been unmeasured residual confounding (eg, smoking, severity of illness); sicker patients, or patients with more severe pain, still may have received opioids. A high-dimensional propensity score may have helped with this issue and is worthwhile to consider for future analyses. In addition, while patients received the medications, it was not possible to know if and when they actually took them; opioids and NSAIDs are frequently used as needed. Therefore, patients may not have been taking the medications at the time of the outcomes, and there was an increase in the AE rates when we extended the medication exposure window beyond the day-supply in the sensitivity analysis. We also did not capture OTC NSAID use, which may have increased after implementation of the OSI. Finally, we did not consider the use of other medications and nonpharmacologic therapies that may have replaced opioids and contributed to a lower-than-anticipated use of NSAIDs.


We found that the rates of new and continuing opioid users decreased following nationwide implementation of the OSI in the VHA, while the rates of NSAID users remained constant. Our concern was the potential for expanded NSAID use among patients at increased risk for NSAID-related adverse drug events (eg, older patients) as an unintended consequence of the OSI. Thus, it is reassuring that we did not identify an increase in AEs among incident NSAID users. Indeed, our findings support ongoing efforts to use nonopioid strategies, such as NSAIDs, for pain management when appropriate. Further research is needed to compare the incidence of serious adverse outcomes among users of opioid and nonopioid medications, as well as the cause of death to determine the plausibility of the analgesic’s role.Author Affiliations: Kansas City VA Medical Center (VF), Kansas City, MO; VA Center for Medication Safety/Pharmacy Benefits Management Services (SLA, CBG, FEC), Hines, IL; VA Center for Health Equity Research and Promotion (SLA, XZ, CBG), Pittsburgh, PA; VA Center for Healthcare Organization and Implementation Research (DRM), Bedford, MA; Birmingham VA Medical Center (JAS), Birmingham, AL; University of Alabama at Birmingham (JAS), Birmingham, AL.

Source of Funding: There was no funding for this study. The work was supported by VA Pharmacy Benefits Management Services (Hines, IL), and VA Pittsburgh Healthcare System (Pittsburgh, PA). The views expressed in this paper are those of the authors, and no official endorsement by the Department of Veterans Affairs or the United States Government is intended or should be inferred.

Author Disclosures: Dr Singh has consulted for Savient, Takeda, Regeneron, Merz, Iroko, Bioiberica, Crealta/Horizon, and Allergan (pharmaceuticals); WebMD; UBM, LLC; and the American College of Rheumatology. Dr Singh has also received grants from Takeda and Savient, is a member of the executive committee of OMERACT, and is an editor of the Cochran UAB satellite center. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (VF, SLA, XZ, DRM, JAS, CBG, FEC); acquisition of data (XZ, FEC); analysis and interpretation of data (VF, SLA, XZ, DRM, JAS, CBG, FEC); drafting of the manuscript (VF, SLA, CBG); critical revision of the manuscript for important intellectual content (SLA, DRM, JAS, CBG); statistical analysis (XZ, DRM); administrative, technical, or logistic support (JAS, FEC); and supervision (CBG).

Address Correspondence to: Sherrie Aspinall, PharmD, MSc, BCPS, VA Pittsburgh Healthcare System, University Dr (151C) Bldg 30, Pittsburgh, PA 15240. Email:

1. Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid-involved overdose deaths — United States, 2010-2015 [erratum in MMWR Morb Mortal Wkly Rep. 2017;66(1):35. doi: 10.15585/mmwr.mm6601a10]. MMWR Morb Mortal Wkly Rep. 2016;65(50-51):1445-1452. doi: 10.15585/mmwr.mm655051e1.

2. Jones CM, Mack KA, Paulozzi LJ. Pharmaceutical overdose deaths, United States, 2010. JAMA. 2013;309(7):657-659. doi: 10.1001/jama.2013.272.

3. VHA Pain Management: Opioid Safety Initiative (OSI). US Department of Veterans Affairs website. Updated October 1, 2017. Accessed February 22, 2017.

4. Huerta C, Castellsague J, Varas-Lorenzo C, García Rodríguez LA. Nonsteroidal anti-inflammatory drugs and risk of ARF in the general population. Am J Kidney Dis. 2005;45(3):531-539. doi: 10.1053/j.ajkd.2004.12.005.

5. Schneider V, Lévesque LE, Zhang B, Hutchinson T, Brophy JM. Association of selective and conventional nonsteroidal antiinflammatory drugs with acute renal failure: a population-based, nested case-control analysis. Am J Epidemiol. 2006;164(9):881-889. doi: 10.1093/aje/kwj331.

6. Bhala N, Emberson J, Merhi A, et al; Coxib and traditional NSAID Trialists’ (CNT) Collaboration. Vascular and upper gastrointestinal effects of non-steroidal anti-inflammatory drugs: meta-analyses of individual participant data from randomised trials. Lancet. 2013;382(9894):769-779. doi: 10.1016/S0140-6736(13)60900-9.

7. FDA Drug Safety Communication: FDA strengthens warning that non-aspirin nonsteroidal anti-inflammatory drugs (NSAIDs) can cause heart attacks or strokes. FDA website. Published July 9, 2015. Accessed December 1, 2016.

8. Nissen SE, Yeomans ND, Solomon DH, et al; PRECISION Trial Investigators. Cardiovascular safety of celecoxib, naproxen, or ibuprofen for arthritis. N Engl J Med. 2016;375(26):2519-2529. doi: 10.1056/NEJMoa1611593.

9. Solomon DH, Rassen JA, Glynn RJ, Lee J, Levin R, Schneeweiss S. The comparative safety of analgesics in older adults with arthritis. Arch Intern Med. 2010;170(22):1968-1976. doi: 10.1001/archinternmed.2010.391.

10. Westanmo A, Marshall P, Jones E, Burns K, Krebs EE. Opioid dose reduction in a VA health care system—implementation of a primary care population-level initiative. Pain Med. 2015;16(5):1019-1026. doi: 10.1111/pme.12699.

11. Garcia MM, Angelini MC, Thomas T, Lenz K, Jeffrey P. Implementation of an opioid management initiative by a state Medicaid program. J Manag Care Spec Pharm. 2014;20(5):447-454. doi: 10.18553/jmcp.2014.20.5.447.

12. Von Korff M, Dublin S, Walker RL, et al. The impact of opioid risk reduction initiatives on high-dose opioid prescribing for patients on chronic opioid therapy. J Pain. 2016;17(1):101-110. doi: 10.1016/j.jpain.2015.10.002.

13. Brady KT, McCauley JL, Back SE. Prescription opioid misuse, abuse, and treatment in the United States: an update. Am J Psychiatry. 2016;173(1):18-26. doi: 10.1176/appi.ajp.2015.15020262.

14. Kuehn B. CDC: major disparities in opioid prescribing among states: some states crack down on excess prescribing. JAMA. 2014;312(7):684-686. doi: 10.1001/jama.2014.9253.

15. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613-619. doi: 10.1016/0895-4356(92)90133-8.

16. Lafrance JP, Miller DR. Defining acute kidney injury in database studies: the effects of varying the baseline kidney function assessment period and considering CKD status. Am J Kidney Dis. 2010;56(4):651-660. doi: 10.1053/j.ajkd.2010.05.011.

17. Danaei G, Tavakkoli M, Hernán MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. Am J Epidemiol. 2012;175(4):250-262. doi: 10.1093/aje/kwr301.

18. Austin PC, Mamdani MM. A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use. Stat Med. 2006;25(12):2084-2106. doi: 10.1002/sim.2328.

19. Mamdani M, Sykora K, Li P, et al. Reader’s guide to critical appraisal of cohort studies: 2. assessing potential for confounding. BMJ. 2005;330(7497):960-962. doi: 10.1136/bmj.330.7497.960.

20. Mosher HJ, Krebs EE, Carrel M, Kaboli PJ, Vander Weg MW, Lund BC. Trends in prevalent and incident opioid receipt: an observational study in Veterans Health Administration 2004-2012. J Gen Intern Med. 2015;30(5):597-604. doi: 10.1007/s11606-014-3143-z.

21. Graf J. Analgesic use in the elderly: the “pain” and simple truth: comment on “The comparative safety of analgesics in older adults with arthritis.” Arch Intern Med. 2010;170(22):1976-1978. doi: 10.1001/archinternmed.2010.442.

22. Ballantyne JC, Sullivan MD. Intensity of chronic pain—the wrong metric? N Engl J Med. 2015;373(22):2098-2099. doi: 10.1056/NEJMp1507136.

23. Lin LA, Bohnert ASB, Kerns RD, Clay MA, Ganoczy D, Ilgen MA. Impact of the Opioid Safety Initiative on opioid-related prescribing in veterans. Pain. 2017;158(5):833-839. doi: 10.1097/j.pain.0000000000000837.

24. O’Donnell JB, Ekman EF, Spalding WM, Bhadra P, McCabe D, Berger MF. The effectiveness of a weak opioid medication versus a cyclo-oxygenase-2 (COX-2) selective non-steroidal anti-inflammatory drug in treating flare-up of chronic low-back pain: results from two randomized, double-blind, 6-week studies. J Int Med Res. 2009;37(6):1789-1802. doi: 10.1177/147323000903700615.

25. Gimbel JS, Brugger A, Zhao W, Verburg KM, Geis GS. Efficacy and tolerability of celecoxib versus hydrocodone/acetaminophen in the treatment of pain after ambulatory orthopedic surgery in adults. Clin Ther. 2001;23(2):228-241. doi: 10.1016/S0149-2918(01)80005-9.

26. LoCasale R, Kern DM, Chevalier P, Zhou S, Chavoshi S, Sostek M. Description of cardiovascular event rates in patients initiating chronic opioid therapy for noncancer pain in observational cohort studies in the US, UK, and Germany. Adv Ther. 2014;31(7):708-723. doi: 10.1007/s12325-014-0131-y.

27. Carman WJ, Su S, Cook SF, Wurzelmann JI, McAfee A. Coronary heart disease outcomes among chronic opioid and cyclooxygenase-2 users compared with a general population cohort. Pharmacoepidemiol Drug Saf. 2011;20(7):754-762. doi: 10.1002/pds.2131.

28. Li L, Setoguchi S, Cabral H, Jick S. Opioid use for noncancer pain and risk of myocardial infarction amongst adults. J Intern Med. 2013;273(5):511-526. doi: 10.1111/joim.12035.

29. Mallappallil M, Sabu J, Friedman EA, Salifu M. What do we know about opioids and the kidney? Int J Mol Sci. 2017;18(1):223-240. doi: 10.3390/ijms18010223.

30. Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Prescription of long-acting opioids and mortality in patients with chronic noncancer pain. JAMA. 2016;315(22):2415-2423. doi: 10.1001/jama.2016.7789.