State Prescription Drug Monitoring Programs and Fatal Drug Overdoses

May 26, 2017
Young Hee Nam, PhD

,
Dennis G. Shea, PhD

,
Yunfeng Shi, PhD

,
John R. Moran, PhD

Volume 23, Issue 5

This study investigates the impact of state prescription drug monitoring programs on drug overdose mortality rates for all drug categories.

ABSTRACT

Objectives: To examine the impact of prescription drug monitoring programs (PDMPs) on drug overdose deaths.

Study Design: We used variation in the timing of state PDMP legislation and implementation to estimate the impact of these programs on drug overdose mortality rates across all drug categories from 1999 to 2014 and separately for each category from 1999 to 2010. Data used include US all-jurisdiction mortality data, estimated population data, and sociodemographic data from the CDC and the US Census Bureau.

Methods: Multivariate regression models were applied to state panel data, including state and year fixed effects and state-specific linear time trends. Preprogram tests were used to assess the common trends assumption underlying our empirical approach.

Results: The implementation of PDMPs was not associated with reductions in overall drug overdose or prescription opioid overdose mortality rates relative to expected rates in the absence of PDMPs. For most categories, PDMPs were associated with increased mortality rates, but the associations were statistically insignificant. In a subsample analysis of states with PDMPs in operation for 5 or more years, the programs were found to be associated with significantly higher mortality rates in legal narcotics, illicit drugs, and other and unspecified drugs.

Conclusions: PDMPs were not associated with reductions in drug overdose mortality rates and may be related to increased mortality from illicit drugs and other, unspecified drugs. More comprehensive and prevention-oriented approaches may be needed to effectively reduce drug overdose deaths and avoid fatal overdoses from other drugs used as substitutes for prescription opioids.

Am J Manag Care. 2017;23(5):297-303

Takeaway Points

Prior studies of state prescription drug monitoring programs (PDMPs) and fatal drug overdoses have either not examined specific drug categories or examined only selected specific drug categories, focusing on prescription opioids. This study examined the impact of PDMPs on drug overdose mortality rates across all drug categories from 1999 to 2014 and separately for each category from 1999 to 2010, including illicit drugs and other and unspecified drugs, and found that:

  • PDMPs, on average, have not been effective in reducing drug overdose mortality rates.
  • PDMPs may be associated with increased overdose mortality rates in drug categories other than prescription opioids.

Prescription drug overdoses have become one of the fastest growing and most serious public health concerns in the United States. The number of deaths has increased more than 7-fold: from about 6100 in 1980 to 47,055 in 2014,1-3 or approximately 129 deaths every day. In 2014, the overall US mortality rate was 823.7 per 100,000, and the drug overdose mortality rate was 14.7 per 100,000.1,2,4,5 In addition, recent reports indicate that fatal drug overdoses significantly contributed to the unexpected increase in mortality among midlife non-Hispanic whites.1,3,6 More than half of drug overdose deaths are caused by prescription drugs, and more than 70% of prescription drug overdose deaths are caused by opioid pain relievers.1,7,8

Inappropriate prescription drug use not only affects health outcomes, but is also correlated with increasing fraud, waste, and additional costs for taxpayers, employers, and insurers.9-12 For example, a study that examined medical and pharmacy claims data from 16 self-insured employer health plans reported that enrollees identified as having drug abuse or dependence had hospitalization rates 12 times higher than those without these conditions and annual healthcare and drug costs 8 and 5 times higher, respectively.9 The total societal costs of prescription opioid overdose, abuse, and dependence in the United States in 2013 was estimated at $78.5 billion, including healthcare costs, lost productivity, and criminal justice costs.10

To address prescription drug misuse and abuse, an increasing number of states (currently more than 40) have implemented a prescription drug monitoring program (PDMP). These programs maintain statewide databases, collecting data on the prescribing, dispensing, and purchasing of controlled substances.13 These data can be used to identify suspected illegal activities, such as prescription diversions, doctor shopping, and pill mills; to inform public health initiatives; and to facilitate the treatment of drug addiction, among others. The Office of National Drug Control Policy (ONDCP) considers PDMPs an important tool to combat prescription drug overdose deaths. Implementing PDMPs was one of the 4 major areas in ONDCP’s 2011 Prescription Drug Abuse Prevention Plan, with the goals of having legislation in all 50 states establishing PDMPs within 36 months and decreasing by 15% the number of unintended opioid-related overdose deaths within 60 months.14 These goals were not met.

Despite the expansion of PDMPs, evidence on their effectiveness in reducing prescription drug abuse or misuse is inconclusive.15-22 For example, 1 study analyzed opioid abuse treatment admission data and found that PDMPs mitigated the increasing trend of opioid abuse and misuse.19 However, another study found no demonstrable decrease in prescription opioid abuse associated with PDMPs.20 Reviewing patients’ prescription history in a university medical center emergency department changed clinicians’ opioid prescription plans for 74 of 179 patients, with 61% receiving fewer opioids and 39% receiving more opioids than originally planned.21 A more recent study (2016) used the 2001 to 2010 data from the National Ambulatory Medical Care Survey and found that PDMPs were associated with a significant decrease in Schedule II opioids prescribing rates.22

The existing literature on PDMPs and drug overdose deaths is limited. The results of a small number of earlier studies on this topic suggest that PDMPs might not be effective in reducing fatal drug overdoses, but the findings were not consistent.23-25 A more recent study using an interrupted time series design found that implementing a PDMP was associated with a decrease in prescription opioid-related overdose mortality rates.26 These studies focused on prescription opioid-related overdose deaths and/or overall drug overdose deaths. To our knowledge, no study has broken down drug overdose deaths across different classes of drugs, an important gap in the literature. Although PDMPs monitor only prescription controlled substances, they might also affect the use of other drugs, as individuals may switch to nonprescription drugs or find alternative ways of obtaining prescription medications.

Our study aimed to investigate whether PDMPs were effective in reducing fatal drug overdoses across all drug categories and separately for each category. Given the magnitude of the prescription drug epidemic and the expansion of PDMPs at the national level, evaluating PDMPs’ overall impact may be helpful for assessing the relative effectiveness of public policies designed to reduce drug overdoses.

METHODS

Study Design

We conducted our analysis in 2 parts. First, using publicly available mortality data from the Centers for Disease Control and Prevention (CDC),27 we examined overall drug overdose mortality rates in PDMP and non-PDMP states in the United States between 1999 and 2014. This database suppresses mortality data when the number of deaths is fewer than 10. Next, we applied the same analysis to the unsuppressed CDC mortality data obtained from the National Center for Health Statistics (NCHS) from 1999 to 2010—the maximal range available from the multiple-cause-of-death data at the time of our analysis—identifying drug overdose deaths by subcategory.

In our study sample, 19 states began operating PDMPs sometime between 2002 and 2010: Alabama, Arizona, Colorado, Connecticut, Iowa, Louisiana, Maine, Minnesota, Mississippi, North Carolina, North Dakota, New Mexico, Ohio, South Carolina, Tennessee, Virginia, Vermont, West Virginia, and Wyoming. From 2011 to 2014, 15 more states implemented PDMPs: Alaska, Arkansas, Delaware, Florida, Georgia, Kansas, Maryland, Montana, Nebraska, New Hampshire, New Jersey, Oregon, South Dakota, Washington, and Wisconsin. Two states (“states” hereafter includes the District of Columbia) did not begin to operate PDMPs by the end of 2014: Missouri and the District of Columbia.28,29 The remaining 15 states were excluded as they implemented PDMPs before 2000 and did not have sufficient pre-implementation years available to support the empirical approach used in this paper. Figure 1 shows our study sample states.

Drug overdose deaths were identified by International Classification of Diseases Codes 10th Revision (ICD-10 codes): X40-X44, X60-X64, X85, and Y10-Y14 for the underlying-cause-of-death data and T36.0-T50.9 for the multiple-cause-of-death data. A complete list of these ICD-10 codes is shown in eAppendix Table 1 [eAppendices available at ajmc.com].

Data

The data used to measure drug overdose mortality rates include: 1) CDC Wide-ranging Online Data for Epidemiologic Research (WONDER) database of mortality27; 2) de-identified individual-level unsuppressed mortality data for all jurisdictions of the United States, obtained from the NCHS; and 3) estimated population data produced by the US Census Bureau and NCHS, which were also extracted from the CDC WONDER website.27 The mortality data are based on the information from all death certificates filed in all jurisdictions in the United States. Each death certificate contains a single underlying cause of death and up to 20 multiple causes. Data used for covariates include population estimates from the US Census Bureau and NCHS27 and sociodemographic data from the Current Population Survey, produced by the US Census Bureau.30

Statistical Analysis

To estimate the effect of PDMPs on fatal drug overdoses, we used multivariate regression models with state and year fixed effects and state-specific linear time trends. The inclusion of state fixed effects allows each state to serve as its own control group, eliminating all time-invariant unobserved differences across states. National and state trends in PDMP operation and drug overdose mortality rates, which might otherwise confound our estimates, were accounted for with year fixed effects and state-specific linear time trends.31 This approach contrasts with the analysis in Patrick et al, which relied on state fixed effects and a single national linear time trend.26 The exposure was defined as the state- and year-specific PDMP operation status (operated = 1, not operated = 0). The outcome variable was state-level, year-specific drug overdose mortality rates measured by the number of deaths per 100,000 individuals. To control for other differences across states, we included state-level, time-varying covariates that might be associated with drug overdose mortality rates and PDMP operation status; in particular, the percentages of a state’s population that is male, white, high school educated or better (age 25 or older), uninsured, and enrolled in the Medicaid program. We also controlled for median household income (in 2015 US$ for 1999-2014 and 2011 US$ for 1999-2010). Mortality rates were crude rates because the covariates we obtained were not age-adjusted.32 Clustered standard errors were used to correct for arbitrary patterns of serial correlation within states.

A key threat to establishing a cause-and-effect relationship between PDMPs and fatal drug overdoses is the possibility that states adopt PDMPs in response to changes in overdoses that depart from the state-specific linear time trends included in our models, or that adoption coincides with changes in these trends for other reasons. We tested for this possibility in an extended model presented in eAppendix Tables 2 and 3 by adding indicator variables for the year prior to enactment of the PDMP law and for the 2 years prior to enactment, labeled PRE1 and PRE2, respectively. If these preprogram indicators are small and statistically insignificant, it suggests that after adjusting for national and state trends, states adopting PDMPs would have experienced similar changes in fatal drug overdoses as the nonadopting states in the absence of a PDMP.33

We also examined potential PDMP enactment effects for the period following the enactment of a PDMP law, but prior to the PDMP becoming operational, by including an indicator labeled PEPO (post enactment and pre-operation) in the extended model for these periods (years of post enactment and pre-operation = 1; the other years = 0). In addition, to examine potentially important differences in the PDMP effect based on program duration, we also conducted a subsample analysis of PDMPs operating for 5 or more years.

Statistical analysis was conducted using SAS version 9.4 (SAS Institute Inc; Cary, North Carolina) and Stata/IC version 14 (StataCorp LP; College Station, Texas). Institutional review board approval was not needed because no human participants were involved in this study.

RESULTS

Overall, mortality rates from prescription drug overdoses increased from 1999 to 2014, in both PDMP and non-PDMP states. The PDMP coefficients from our regression models (Tables 1 and 2) capture the difference between the mortality rates expected to arise in the absence of a PDMP, as predicted by all other covariates in the model, and the rates that occur when a PDMP is present. We organized the results to show the PDMP effect in: 1) overall drug overdose mortality rates, from the underlying-cause-of-death data and the multiple-cause-of-death data, for 1999 to 2014 and 1999 to 2010, separately (Table 1); and 2) subcategories of multiple-cause-of-death data in 2 ways: a) subcategories, which are not mutually exclusive, but collectively comprehensive and b) subcategories with high mortality rates (mortality rates >1 per 100,000 for both the PDMP and non-PDMP states) for 1999 to 2010 (Table 2). Figures 2 and 3 illustrate the results in Tables 1 and 2, respectively.

For the overall overdose mortality rates, our estimates from the publicly available data (1999-2014) were 0.08 (95% confidence interval [CI], —0.89 to 1.04) and 0.02 (95% CI, –0.97 to 1.00) for underlying cause of deaths and multiple cause of deaths, respectively. The estimates from the unsuppressed data (1999-2010) were 0.02 (95% CI, –1.05 to 1.08) and 0.02 (95% CI, –1.07 to 1.11), respectively. In the extended model, the PDMP coefficients slightly increased, but were statistically insignificant (eAppendix Table 2 and eAppendix Figure 1). These results suggest that PDMP implementation had little impact on overall overdose mortality rates. Throughout the primary and extended models, all PDMP coefficients pertaining to overall overdose death rates were positive; however, their magnitude and statistical significance varied across model specifications.

In the subcategory analysis, PDMP coefficients were 0.02 (95% CI, —0.81 to 0.84) for legal narcotics and 0.85 (95% CI, –0.08 to 1.78) for other and unspecified drugs (Table 2). In the extended model, PDMPs were associated with significantly increased mortality rates for illicit drugs (0.92; 95% CI, 0.15-1.69) and cocaine (0.71; 95% CI, 0.11-1.31) (eAppendix Table 3 and eAppendix Figure 2).

Based on the subsample analysis for states with a PDMP in place for 5 or more years, all PDMP coefficients were positive for overall mortality rates and were significant for 1999 to 2010 (Table 1 and eAppendix Table 1). In the subcategory analysis, longer-standing PDMPs were associated with significantly increased mortality rates in several categories, including legal narcotics, illicit drugs, cocaine, other and unspecified drugs (Table 2), and illicit drugs and cocaine (eAppendix Table 2).

As shown in the extended model results, none of the preprogram indicators (PRE1 and PRE2) were significant, lending support to the model specification employed in our initial analysis. The PEPO indicator was positive and significant for some categories, which suggests an increase in mortality rates for those categories in the post enactment and preoperation periods.

DISCUSSION

This study investigated PDMP effects on fatal drug overdoses in the United States from 1999 to 2014. We found that PDMPs were not associated with a reduction in either overall or prescription opioid drug overdose mortality rates. Moreover, during the period from 1999 to 2010, for which we conducted the subcategory analysis, PDMPs were often associated with increased mortality rates in drug categories other than prescription opioids, such as illicit drugs or other and unspecified drugs, particularly among the states with longer-standing PDMPs. Although our study period was shorter when examining mortality rates for different drug categories, our results may reflect an unintended consequence of PDMPs, at least up through 2010, whereby reduced access to prescription drugs may have led some individuals with addictive disorders to seek out substitute drugs.34,35

Our findings have several policy and clinical implications. First, PDMPs do not seem to have been successful in reducing drug overdose mortality rates, even in the target categories of prescription opioids (T40.2) and legal narcotics (T40.2-T40.4). This is consistent with some previous studies.23-25 There are many possible reasons for this outcome. For example, PDMPs may not be able to fully address prescription diversions, doctor shopping, or other abusive behaviors, and under these programs, potential drug-related illegal activities are only detectable through prescription fillings. The rapidly growing online pharmaceutical sale space may have also increased the opportunities for individuals to evade state or federal regulations. (More than 90% of internet pharmacies are estimated to be illegal.36) Further, PDMPs may drive patients away from doctors who could help them address drug abuse or dependence.

Second, our results imply that PDMPs might be related to increases in drug overdose mortality rates attributable to illicit drugs or other and unspecified drugs. The existing literature has raised the possibility of these unintended consequences of PDMPs, although there has been little empirical evidence to date.23,34 By analyzing overdose deaths for different drug categories, including illicit drugs and other and unspecified drugs, our study provides some evidence for such a possibility. Future research is needed to further explore the unintended consequences of PDMPs and the potential mechanisms contributing to them (eg, PDMPs’ influence on clinical practices and individual behaviors). If the underlying problems of drug addiction or drug abuse are not effectively addressed, PDMPs might trigger some people to obtain illicit drugs as potential substitutes.

Third, our findings suggest that PDMPs may need to be combined with more comprehensive and prevention-oriented approaches to address drug overdose deaths. Examples of prevention-oriented approaches include: 1) improving patient education on the appropriate use of drugs, 2) ensuring proper access to prescription drugs for those with medical needs, 3) expanding treatment programs for those with drug abuse problems, and 4) improving provider education and clinical practices for pain management. Such approaches are consistent with the recent program, "Prescription Drug Overdose: Prevention for States," funded by the CDC in 16 states.37

Fourth, our findings suggest that policy makers, insurers, and managed care organizations might need to consider the effects of PDMPs when designing health plans, including such features as reimbursement, overall benefit design, and coverage criteria for specialty treatment of drug abuse. Currently, CMS provides the Medicare Part D Opioid Prescriber Summary File on its website, which includes the individual provider’s National Provider Identifier, last name, zip code, and number/percentage of prescription claims for opioid drugs.38 Although this information is based on Medicare claims data, physicians might be concerned about the potential effects the PDMP data could have on reimbursement when combined with administrative data.39 Reimbursement policies based on the number of opioid prescriptions, without consideration of medical need or value, might negatively affect quality of care and, ultimately, increase costs for payers if physicians limit opioid prescriptions in a way that runs counter to optimal patient care. In addition, there may also need to be better coordination between primary and specialty care providers for patients with drug problems.

Finally, this study suggests that restricting attention to overdose deaths caused by opioids or prescription drugs might not fully capture the impact of PDMPs. Researchers and policy makers may need to be cautious about the heterogeneous effects across different drug categories, partly due to drug substitutability. The results also call attention to the incomplete information on the cause of death in mortality data. The “other and unspecified drugs” category (T50.9) has a larger number of fatal overdoses than any other category (eAppendix Figure 3) and warrants further investigation.

Limitations

First, our analysis of overdose mortality rates by drug category only extended to 2010. Second, although our empirical approach mechanically eliminated many potential confounders and our preprogram tests provided support for the common trends assumption necessary for its use, we cannot rule out all sources of bias, such as those created by the time-varying factors we were unable to control for. Finally, our analysis uses binary indicators for PDMP implementation status, thus only estimating an average PDMP effect. The existing literature has documented heterogeneity in the design and implementation of PDMPs across states.40 Hence, the effects of PDMPs on drug overdose deaths may also differ across states. Previous studies examined the effects of some PDMP characteristics on overdose mortality. These studies focused on factors such as the type of governing agency, statutory authority to monitor noncontrolled substances, the requirement for committee oversight, exempting practitioners from the obligation to access PDMP data, and the provision of unsolicited reports to healthcare practitioners and law enforcement agencies, and did not find significant protective effects of any of these features.23,24 Another study reported that monitoring 4 or more drug schedules and more frequent updating of PDMP data were associated with greater reductions in overdose deaths.26 Future studies are warranted to evaluate the effectiveness of other PDMP characteristics.

CONCLUSIONS

PDMPs were not associated with a decrease in drug overdose mortality rates, even in the target category of prescription opioids. They may be associated with increased mortality rates in categories other than prescription opioids, especially in states where PDMPs have been operating for longer periods of time. Further research is needed to better understand the heterogeneous impacts of PDMPs. More comprehensive, prevention-oriented approaches, including improvement in patient education and clinical practices for pain management, may be needed to effectively reduce mortality caused by drug overdoses.

Acknowledgments

The authors are grateful to the National Center for Health Statistics, the CDC, and the various vital statistics jurisdictions for providing data used in this study.

Author Affiliations: University of Pennsylvania (YN), Philadelphia, PA; The Pennsylvania State University (DGS, YS, JRM), University Park, PA.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (YN, DGS, YS, JRM); acquisition of data (YN, DGS); analysis and interpretation of data (YN, DGS, YS, JRM); drafting of the manuscript (YN, DGS, YS, JRM); critical revision of the manuscript for important intellectual content (YN, DGS, YS, JRM); statistical analysis (YN, JRM).

Address Correspondence to: Young Hee Nam, PhD, Center for Pharmacoepidemiology Research and Training, Center for Clinical Epidemiology and Biostatistics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, 826 Blockley Hall, 423 Guardian Dr, Philadelphia, PA 19104. E-mail: ynam@mail.med.upenn.edu.

REFERENCES

1. Deaths from prescription opioid overdose. CDC website. http://www.cdc.gov/drugoverdose/data/overdose.html. Accessed May 20, 2015.

2. Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in drug and opioid overdose deaths—United States, 2000-2014. MMWR Morb Mortal Wkly Rep. 2016;64(50):1378-1382. doi: 10.15585/mmwr.mm6450a3.

3. Warner M, Chen LH, Makuc DM, Anderson RN, Miniño AM. Drug poisoning deaths in the United States, 1980-2008. CDC website. https://www.cdc.gov/nchs/data/databriefs/db81.pdf. Published December 2011. Accessed August 25, 2016.

4. Kochanek KD, Murphy SL, Xu J, Tejada-Vera B. Deaths: final data for 2014. National Vital Statistics Reports. CDC website. http://www.cdc.gov/nchs/data/nvsr/nvsr65/nvsr65_04.pdf. Published June 30, 2016. Accessed August 23, 2016.

5. National Center for Health Statistics. NCHS data on drug-poisoning deaths. CDC website. https://www.cdc.gov/nchs/data/factsheets/factsheet_drug_poisoning.pdf. Published March 2016. Accessed August 23, 2016.

6. Case A, Deaton A. Rising morbidity and mortality in midlife among white non-Hispanic Americans in the 21st century. Proc Natl Acad Sci USA. 2015;112(49):15078-15083. doi: 10.1073/pnas.1518393112.

7. CDC. Vital signs: overdoses of prescription opioid pain relievers—United States, 1999-2008. MMWR Morb Mortal Wkly Rep. 2011;60(43):1487-1492.

8. Jones CM, Mack KA, Paulozzi LJ. Pharmaceutical overdose deaths, United States, 2010. JAMA. 2013;309(7):657-659. doi: 10.1001/jama.2013.272.

9. White AG, Birnbaum HG, Mareva MN, et al. Direct costs of opioid abuse in an insured population in the United States. J Manag Care Pharm. 2005;11(6):469-479.

10. Florence CS, Zhou C, Luo F, Xu L. The economic burden of prescription opioid overdose, abuse, and dependence in the United States, 2013. Med Care. 2016;54(10):901-906. doi: 10.1097/MLR.0000000000000625.

11. Prescription painkiller overdoses in the US. CDC website. http://www.cdc.gov/VitalSigns/pdf/2011-11-vitalsigns.pdf. Published November 2011. Accessed September 10, 2013.

12. Tkacz J, Pesa J, Vo L, et al. Opioid analgesic-treated chronic pain patients at risk for problematic use. Am J Manag Care. 2013;19(11):871-880.

13. Finklea KM, Bagalman E, Sacco LN. Prescription drug monitoring programs. Federation of American Scientists website. https://fas.org/sgp/crs/misc/R42593.pdf. Published January 2013. Accessed August 25, 2013.

14. Epidemic: responding to America’s prescription drug abuse crisis. Obama White House archives website. https://obamawhitehouse.archives.gov/sites/default/files/ondcp/policy-and-research/rx_abuse_plan.pdf. Published 2011. Accessed September 10, 2013.

15. Briefing on PMP effectiveness. Prescription monitoring programs: an effective tool in curbing the prescription drug abuse epidemic. PMP Center of Excellence website. https://www.bja.gov/publications/brandeis_pmp_effectiveness_brief.pdf. Published March 2012. Accessed September 10, 2013.

16. 2010 KASPER satisfaction survey: executive summary. Commonwealth of Kentucky website. http://chfs.ky.gov/NR/rdonlyres/BDC0DFC9-924B-4F11-A10A-5EB17933FDDB/0/2010KASPERSatisfactionSurveyExecutiveSummary.pdf. Published December 31, 2010. Accessed August 25, 2013.

17. Blumenschein K, Fink JL, Freeman PR, Kirsh KL, Steinke DT, Talbert J. Independent evaluation of the impact and effectiveness of the Kentucky All Schedule Prescription Electronic Reporting Program (KASPER). Kentucky Cabinet for Health and Family Services and Kentucky Injury Prevention and Research Center website, http://www.chfs.ky.gov/NR/rdonlyres/24493B2E-B1A1-4399-89AD-1625953BAD43/0/KASPEREvaluationFinalReport10152010.pdf. Published October 2010. Accessed September 10, 2013.

18. Wang J, Christo PJ. The influence of prescription monitoring programs on chronic pain management. Pain Physician. 2009;12(3):507-515.

19. Reifler LM, Droz D, Bailey JE, et al. Do prescription monitoring programs impact state trends in opioid abuse/misuse? Pain Med. 2012;13(3):434-442. doi: 10.1111/j.1526-4637.2012.01327.x.

20. Twillman R. Impact of prescription monitoring programs on prescription patterns and indicators of opioid abuse. J Pain. 2006;7(4):S6. doi: 10.1016/j.jpain.2006.01.430.

21. Baehren DF, Marco CA, Droz DE, Sinha S, Callan EM, Akpunonu P. A statewide prescription monitoring program affects emergency department prescribing behaviors. Ann Emerg Med. 2010;56(1):19-23.e1-e3. doi: 10.1016/j.annemergmed.2009.12.011.

22. Bao Y, Pan Y, Taylor A, et al. Prescription drug monitoring programs are associated with sustained reductions in opioid prescribing by physicians. Health Aff (Millwood). 2016;35(6):1045-1051. doi: 10.1377/hlthaff.2015.1673.

23. Li G, Brady JE, Lang BH, Giglio J, Wunsch H, DiMaggio C. Prescription drug monitoring and drug overdose mortality. Inj Epidemiol. 2014;1(9):1-8. doi: 10.1186/2197-1714-1-9.

24. Paulozzi LJ, Kilbourne EM, Desai HA. Prescription drug monitoring programs and death rates from drug overdose. Pain Med. 2011;12(5):747-754. doi: 10.1111/j.1526-4637.2011.01062.x.

25. Paulozzi LJ, Stier DD. Prescription drug laws, drug overdoses, and drug sales in New York and Pennsylvania. J Public Health Policy. 2010;31(4):422-432. doi:10.1057/jphp.2010.27.

26. Patrick SW, Fry CE, Jones TF, Buntin MB. Implementation of prescription drug monitoring programs associated with reductions in opioid-related death rates. Health Aff (Millwood). 2016;35(7):1324-1332. doi: 10.1377/hlthaff.2015.1496.

27. Wide-ranging Online Data for Epidemiologic Research (WONDER). CDC website. https://wonder.cdc.gov. Accessed August 25, 2016.

28. State profiles reports [2013]. Alliance of States With Prescription Monitoring Programs website. http://www.pmpalliance.org/content/state-profiles-reports. Accessed September 10, 2013.

29. Prescription Drug Monitoring Program Training and Technical Assistance Center. http://www.pdmpassist.org/content/state-profiles. Accessed August 25, 2013.

30. Current Population Survey (CPS). US Census Bureau website. https://www.census.gov/cps/data/cpstablecreator.html. Accessed August 25, 2016.

31. Wooldridge JM. Econometric Analysis of Cross Section and Panel Data. 2nd ed. Cambridge, MA: MIT Press; 2010.

32. Rosenbaum PR, Rubin DB. Difficulties with regression analyses of age-adjusted rates. Biometrics. 1984;40(2):437-443. doi: 10.2307/2531396.

33. Heckman JJ, Hotz JV. Choosing among alternative nonexperimental methods for estimating the impact of social programs: the case of manpower training. J Am Stat Assoc. 1989;84(408):862-874. doi: 10.2307/2290059.

34. Unick GJ, Rosenblum D, Mars S, Ciccarone D. Intertwined epidemics: national demographic trends in hospitalizations for heroin- and opioid-related overdoses, 1993-2009. PLoS One. 2013;8(2):e54496. doi: 10.1371/journal.pone.0054496.

35. Dasgupta N, Creppage K, Austin A, Ringwalt C, Sanford C, Proescholdbell SK. Observed transition from opioid analgesic deaths toward heroin. Drug Alcohol Depend. 2014;145:238-241. doi: 10.1016/j.drugalcdep.2014.10.005.

36. Internet pharmacies: federal agencies and states face challenges combating rogue sites, particularly those abroad. Government Accountability Office website. http://www.gao.gov/assets/660/655751.pdf. Published July 2013. Accessed May 20, 2014.

37. Opioid overdose. CDC website. http://www.cdc.gov/drugoverdose/states/state_prevention.html. Updated August 30, 2016. Accessed September 5, 2016.

38. Medicare Part D opioid prescriber summary file 2014. CMS website. https://data.cms.gov/Public-Use-Files/Medicare-Part-D-Opioid-Prescriber-Summary-File-201/e4ka-3ncx. Accessed May 1, 2016.

39. Ashburn MA. The evolution of prescription drug monitoring programs. Pharmacoepidemiol Drug Saf. 2016;25(7):852-853. doi: 10.1002/pds.4036.

40. Manasco AT, Griggs C, Leeds R, et al. Characteristics of state prescription drug monitoring programs: a state-by-state survey. Pharmacoepidemiol Drug Saf. 2016;25(7):847-851. doi: 10.1002/pds.4003.