Emergency Department Visit Classification Using the NYU Algorithm | Page 2

Published Online: April 21, 2014
Sabina Ohri Gandhi, PhD; and Lindsay Sabik, PhD
Table 3 presents logistic regression results estimating the association between EDA visit classification and the outcome variables, mortality, and hospital admission. We found that emergent ED visit classification was positively and significantly associated with mortality (odds ratio [OR]: 3.79, P <.01) and with ED-related hospitalization (OR: 5.28, P <.01), relative to nonemergent classification. Intermediate classification was also positively associated with mortality (OR: 2.16, P = .12) and with ED-related hospitalization (OR: 4.14, P <.01), relative to nonemergent status. The results on the control variables were as expected. Older patients, males, and patients with public insurance or self-pay/charity as the primary payer (vs private insurance) were associated with significantly higher probabilities of ED-associated death and hospitalization. The findings regarding publicly insured and self-pay ED visits were consistent with other research that has shown these patient populations have poorer health and higher inpatient mortality than the privately insured.17-19

Specification Checks

We performed 2 specification checks to test the sensitivity of our results (full results available upon request). First, we estimated conditional logit models with year-specific ED fixed effects in order to control for any unobserved time-invariant ED-specific differences (such as staffing and resources) and to examine within-ED differences in hospitalization and mortality. Results from these models were very similar to our baseline models for both mortality (OR: 3.52, 95% confidence interval [CI]: 2.58-4.80) and hospital admission (OR: 5.22, 95% CI, 4.96-5.50). Second, following Ballard et al, we repeated our analyses using different probability thresholds (75% and 90%) for the ED measures in order to check the sensitivity of our results.9 In these models, visits were categorized as intermediate if the probability they were emergent was between 25% (10%) and 75% (90%). The results for our emergent variable represent the comparison of visits with a predicted probability of being emergent of 75% (90%) or greater with visits with a predicted probability of being emergent of 25% (10%) or less, thus we expect the ORs to increase, but be in the same direction as our baseline results. We found that this is generally the case, with ORs for our mortality outcome of 6.11 (95% CI, 3.73-10.02) and 4.90 (95% CI, 2.16-11.15) for the 75% and 90% cutoffs, respectively. Similarly, ORs for our admission outcome are 8.35 (95% CI, 7.60-9.17) and 11.26 (95% CI, 9.83-12.92) for the 75% and 90% cutoffs, respectively. All specification checks support our main finding that emergent visit status is significantly associated with hospital admission and death.


This analysis evaluated methods for classifying ED visits as emergent, using the EDA in a nationally representative sample of ED visits, and found that measures based on the EDA were significantly associated with mortality and EDassociated hospitalizations. The magnitude of association of emergent classification with hospitalization and mortality were similar to, but larger than, those from Ballard et al, who used data from privately insured individuals in a single integrated delivery system.9 The difference may be due to the fact that our sample included both the Medicaid and uninsured populations, and was not restricted to privately insured and Medicare patients in an integrated delivery system, who are likely to have better access to care. In addition, our sample excluded visits categorized as injury, which may also explain our larger result. Further, our results suggest that the EDA can be used to categorize visits as emergent and nonemergent by researchers when information is available on diagnosis codes but not on triage time (eg, claims data) or when a full chart review is not possible.

There were a few limitations to our approach. In categorizing visits using the EDA, there may have been some measurement error in determining if a visit is truly emergent. For example, some diagnoses may have been appropriately categorized as emergent, but were not associated with death or hospital admission (eg, a broken leg). We attempted to address this possible issue by excluding ED admissions for injury. Another limitation to our research was that we were only able to observe mortality and hospital admission as a direct admit from the ED, but not able to observe subsequent mortality or hospital admission within a limited time frame after the ED visit. Further, we used a blunt measure of hospital admission in order to directly compare our results with Ballard et al,9 which may not necessarily reflect severity in cases where hospitalizations resulted in shorter lengths of stay or were less acute. Finally, thresholds for hospital admission may have differed by payer type or other subjective patient characteristics. EDs may serve as a gateway through which to admit patients who do not have access to care through other channels due to their insurance status or other socioeconomic factors. This question is beyond the scope of this paper and requires further research.

This study demonstrated that the EDA can be used to identify ED visits associated with mortality and hospitalization. Classifying ED visits as emergent or nonemergent has been a shortcoming of the literature on ED use. The EDA is increasingly being used at the state and local levels20- 23 and, despite its limitations, the EDA has the potential to be a useful tool for understanding patterns of use and assessing the effects of policies and programs aimed at reducing nonemergent ED use. While the developers of the EDA have cautioned that the algorithm would not be appropriate to use for making individual reimbursementbased decisions, and recent research has supported this assessment,24 it can be applied to assess overall trends in ED use and to study how interventions and policies may affect these trends. For example, it has been used by researchers studying how new programs providing primary care for the uninsured affected ED use among these particular patient populations.20,21 In such contexts, where administrative data are available to assess how a program or policy change affected utilization, the EDA can be useful.

With the implementation of health reform, there will be major changes in the number of uninsured, the distribution of insurance coverage types, the payment and organization of healthcare providers, and other aspects of the healthcare system that could affect the way various patient populations utilize the ED. It will be important to have tools to classify ED visits and to test the success of interventions and policies designed to alter ED utilization and improve access to alternative sources of care. We have shown that the conclusions of earlier research validating the EDA in the context of managed care also hold when a nationally representative sample of ED visits is examined, suggesting that the EDA is a useful tool for health services and policy researchers.

Take-Away Points

The New York University Emergency Department (ED) Algorithm is a powerful tool that can be used to classify ED visits.
  • Researchers are in need of methods to categorize ED visits as emergent and nonemergent.

  • Public and private decision makers may use the algorithm to evaluate the impact of policies that alter ED utilization when information on diagnosis codes is available.
Author Affiliations: RTI International, Washington, DC (SOG); Department of Healthcare Policy and Research, Virginia Commonwealth University School of Medicine (LS).

Source of Funding: None reported.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (SOG, LS); acquisition of data (SOG, LS); analysis and interpretation of data (SOG, LS); drafting of the manuscript (SOG, LS); critical revision of the manuscript for important intellectual content (SOG, LS); statistical analysis (SOG, LS).

Address correspondence to: Sabina Ohri Gandhi, PhD, RTI International, 701 13th St NW, Ste 750, Washington, DC 20005-3967. E-mail: sgandhi@rti.org.
1. Institute of Medicine. Hospital-Based Emergency Care: At the Breaking Point. Washington, DC: National Academies Press; 2007.

2. National Center for Health Statistics; US Department of Health and Human Services. Health, United States, 2010: with special feature on death and dying. http://www.cdc.gov/nchs/data/hus/hus10.pdf#089. Published 2011. Accessed February 21, 2012.

3. Hsia RY, Kellerman AL, Shen YC. Factors associated with closures of emergency departments in the United States. JAMA. 2011;305(19): 1978-1985.

4. Tang N, Stein J, Hsia RY, Maselli JH, Gonzales RJ. Trends and characteristics of US emergency department visits, 1997-2007. JAMA. 2010;304(6):664-670.

5. Hsu J, Price M, Brand R, et al.Cost-sharing for emergency care and unfavorable clinical events: findings from the safety and financial ramifications of ED copayments study. Health Serv Res. 2006;41(5): 1801-1820.

6. Mortensen K. Copayments did not reduce Medicaid enrollees’ nonemergency use of emergency departments. Health Aff (Millwood). 2010;29(9):1643-1650.

7. Congressional Budget Office. Selected CBO publications reltaed to healthcare legislation, 2009-2010. Washington, DC: Congress of the United States; 2010.

8. Billings J, Parikh N, Mijanovich T. Emergency room use: the New York story. Commonwealth Fund issue brief 2000. http://wagner.nyu .edu/chpsr/index.html?p=25. Published 2000. Accessed October 31, 2011.

9. Ballard DW, Price M, Fung V, et al. Validation of an algorithm for categorizing the severity of hospital emergency department visits. Med Care. 2010;48(1):58-63.

10. Cunningham PJ, May J. Insured Americans drive surge in emergency department visits: issue brief for the Center for Studying Health System Change. 2003;(70):1-6.

11. Zuckerman S, Shen YC. Characteristics of occasional and frequent emergency department users: do insurance coverage and access to care matter? Med Care. 2004;42(2):176-182.

12. Inter-University Consortium for Political and Social Research. National Hospital Ambulatory Medical Care Survey series. http://www .icpsr.umich.edu/icpsrweb/ICPSR/series/42. Published 2011. Accessed February 21, 2012.

13. New York University Center for Health and Public Service Research. NYU ED algorithm. http://wagner.nyu.edu/chpsr/index.html?p=25. Published 2011. Accessed July 27, 2011.

14. Wharam JF, Landon BE, Galbraith AA, Kleinman KP, Soumerai SB, Ross-Degnan D. Emergency department use and subsequent hospitalizations among members of a high-deductible health plan. JAMA. 2007;297(10):1093-1102.

15. Billings J, Zeitel L, Lukomnik J, Carey TS, Blank AE, Newman L. Impact of socioeconomic status on hospital use in New York City. Health Aff (Millwood). 1993;12(1):162-173.

16. Agency for Healthcare Research and Quality.US Department of Health and Human Services. Addressing racial and ethnic disparities in healthcare. http://www.ahrq.gov/research/findings/factsheets/minority/ disparit/. Published April 2013. Accessed July 21, 2013.

17. Hasan O, Orav EJ, Hicks LS. Insurance status and hospital care for myocardial infarction, stroke, and pneumonia. J Hosp Med. 2010;5(8): 452-459.

18. Ayanian JZ, Kohler BA, Abe T, Epstein AM.The relation between health insurance coverage and clinical outcomes among women with breast cancer. N Engl J Med. 1993;29(359):326-331.

19. Weissman JS, Gatsonis C, Epstein AM. Rates of avoidable hospitalization by insurance status in Massachusetts and Maryland. JAMA. 1992;286(17):2388-2394.

20. Bradley CJ, Gandhi SO, Neumark D, Garland S, Retchin S. Lessons for coverage expansion: a Virginia primary care program for the uninsured reduced utilization and cut costs. Health Aff (Millwood). 2012; 31(2):350-359.

21. McLaughlin C, Colby M, Bee G, Libersky J. Healthy San Francisco: changes in access to and utilization of healthcare services. Ann Arbor: Mathematica Policy Research; 2011.

22. Utah Office of Healthcare Statistics. Primary care sensitive emergency department visits in Utah, 2001. http://health.utah.gov/hda/ reports/Primary_Care_ERvisits_Utah2001.pdf. Published April 2004. Accessed October 31, 2011.

23. Washington State Hospital Association. Washington emergency room use: safety net or unneeded services? http://wacmhc.org/documents/ WSHA%20Study.pdf. Published 2007. Accessed October 13, 2011.

24. Raven MC, Lowe RA, Maselli J, Hsia RY. Comparison of presenting complaint vs discharge diagnosis for identifying ‘nonemergency’ emergency department visits. JAMA. 2013;309(11):1145-1153.
Issue: April 2014
More on AJMC.COM