Emergency Department Visit Classification Using the NYU Algorithm
Published Online: April 21, 2014
Sabina Ohri Gandhi, PhD; and Lindsay Sabik, PhD
Emergency departments (EDs) are a critical component of the healthcare system, but face growing demand and often have inadequate capacity.1 Between 2000 and 2008, hospital-based ED visits grew by 15% to 124 million visits, while the total number of hospital-based EDs declined.2,3 Most of this increase in ED visits can be attributed to increases in visits by publicly insured patients.4 Further, EDs often serve as primary care providers despite the fact that they are not optimally designed to provide this type of care.1 In response, payers and policy makers have attempted to deter nonemergent ED use and increase efficiency using financial incentives. For example, some health insurance plans and state Medicaid programs have implemented or increased ED copayments in recent years.5,6 Efforts to improve access to primary care may also reduce nonemergent ED use. Researchers require reliable tools for classifying emergent and nonemergent ED use in order to evaluate policies and initiatives designed to improve access to primary care and reduce nonemergent use. Application of these methods to assess changes in access and reliance on EDs will become even more salient after implementation of the Patient Protection and Affordable Care Act in 2014, when much of the population will gain health insurance coverage and the number of publicly insured individuals will increase dramatically.7
To this end, we assessed a tool for categorizing ED visits as emergent, by applying each visit to a nationally representative sample of ED visits and testing its strength as a predictor of hospital admission and mortality. Specifically, we used the New York University ED Algorithm (EDA). The EDA is based on a full chart review, using information on patient complaints, symptoms, vital signs, diagnoses, procedures, and ED resource use in order to determine the emergent nature of ED visits.8 The EDA could be a useful tool to evaluate ED use, but has only been tested in settings with more limited generalizability and has not previously been validated using a nationally representative sample of ED visits.
A previous study evaluated the EDA using data on commercial and Medicare patients from an integrated delivery system from 1999 to 2001, and found that patients classified as having an emergent visit under the EDA were more likely to be hospitalized within 1 day or to die within 30 days than were patients with visits classified as nonemergent.9 Because the EDA may perform differently among groups of patients with limited access to care who are more likely to use the ED for nonemergent reasons, such as uninsured or Medicaid populations,10,11 results from that study may not generalize to other populations. Recent research has also shown significant changes in the demographics of ED patients over the past decade, so previous results may be outdated.4
This study improves on previous research in a number of ways. We applied the EDA to a national sample of hospital-based ED visits. We used the same criteria to assess the EDA as the previous study, but improve on it by using it in a nationally representative sample of all insurance groups in order to examine its generalizability to the US population, while previous research focused on privately insured and Medicare populations in a managed care context. Finally, we used several years of more recent data—from 2006 to 2009—while the previous study used data from 1999 to 2001. Similar to previous work, we examined the association between EDA categorization and hospital mortality or hospital admission, controlling for observed patient demographic and socioeconomic factors.
Data and Sample Selection
We used a nationally representative sample of visits to hospital-based EDs at noninstitutional, general, and short-stay hospitals (excluding Federal, military, and Veterans Administration hospitals) located in the 50 states and the District of Columbia, from repeated cross sections of the 2006 to 2009 National Hospital Ambulatory Medical Care Survey (NHAMCS). EDs within hospitals were randomly selected using a 4-stage probability design and surveyed over a randomly selected 4-week period each year. The sampling unit was the visit, with a target sample of 100 visits per ED sampled. Visit-level information was reported by hospital staff.12 Our sample included 117,477 ED visits with nonmissing primary diagnosis codes, representing a population of about 420 million. We used patient age, sex, and race, and socioeconomic indicators for the patient’s zip code, including median household income and urban-rural classification, as controls. We recoded the expected source of payment into private insurance, Medicare, Medicaid/State Children’s Health Insurance Program (SCHIP), self-pay/charity, and other insurance categories.
Classification of Emergent ED Visits Using the EDA
We applied the EDA to the NHAMCS data to categorize ED visits as emergent or nonemergent. The EDA used the primary International Classification of Diseases, Ninth Revision diagnosis code at discharge for each visit and assigned the probability of the visit being: (1) nonemergent (NE); (2) emergent but treatable in a primary care setting (E-PCT); (3) emergent/ED care required, but preventable or avoidable if appropriate ambulatory care had been received (E-PA); and (4) emergent/ED care required and not preventable or avoidable (E-NPA). The EDA also separately assigned primary diagnosis codes to categories for injury, mental health, alcohol, and drug-related diagnoses. For about 16% of diagnosis codes, the EDA was not able to assign a visit to any classification category due to missing or uncommon primary diagnosis codes.13
Following Ballard et al, we categorized ED visits into 3 levels: emergent, intermediate, and nonemergent.9 We classified a visit as emergent if the sum of the probabilities of E-NPA and E-PA were greater than 50%. We classified a visit as nonemergent if the sum of the probabilities of NE and E-PCT were greater than 50%. Visits where the sum of the probabilities of E-NPA and E-PA or the sum of the probabilities of NE and E-PCT equaled exactly 50% were categorized as intermediate. We also conducted a sensitivity analysis of probability thresholds at 75% and 90%, following other work using the EDA.14
To estimate the association of emergent and intermediate ED visit classification based on the EDA with mortality and hospitalization, we estimated logistic regression models, using NHAMCS complex survey weights. The 2 main outcome variables of interest were dichotomous indicators for whether an individual died (either on arrival to the ED, in the ED, or after being hospitalized) and whether the patient was admitted to the hospital from the ED. All models controlled for individual characteristics, including age, gender, race (white, Black, other race); payment source (private, Medicare, Medicaid/SCHIP, self-pay/charity, other); socioeconomic indicators at the patient zip code level, including low median household income and urban location; and survey year. The covariates in the model were selected because they are known to be associated with hospitalization and health outcomes.15,16
Table 1 presents the breakdown of ED visits as classified by the EDA. We were able to classify 73,076 of 111,970 total ED visits into the emergent, intermediate, and nonemergent categories. This represented a population of about 260 million ED visits. The remaining ED visits (38,894) were classified into injury (32,149), drug (304), alcohol (1718), and mental health (4723). We were not able to classify about 16% of ED visits using the EDA, which is similar to the percentage of diagnosis codes that are uncommon and could not be classified as reported by the developers of the EDA.13 We limited our analysis sample to patients with diagnosis codes that could be classified into emergent, intermediate, and nonemergent. We hypothesized that patients with ED visits classified as emergent would have a higher probability of mortality and hospitalization. Like Ballard et al, we excluded visits categorized as drug, alcohol, or mental health, since they may have been emergent but were less clearly associated with mortality or hospitalization.9 In addition, we excluded visits categorized as injury since they may have been justifiably emergent, but had a low probability of mortality or hospitalization. Within the sample that could be classified into 1 of the 3 levels, 24.1% of visits were categorized as emergent, 1.9% as intermediate, and 74.0% as nonemergent, using the 50% probability threshold.
Table 2 presents summary statistics for the analysis sample. Patients with ED visits in the sample had an average age of 37 years and were more likely to be female, white, have Medicaid or SCHIP as a primary payer, and live in an urban area. About 15% of patients were either admitted or died on arrival to the ED, in the ED, or in the hospital after an ED-associated visit.
Analysis of Emergent ED Visits Using the EDA
PDF is available on the last page.