Through analysis of multistate Medicaid data, this study identifies differences in 2 commonly used measures of emergency department (ED) utilization, ED visit count and ED reliance.
ABSTRACTObjectives: Emergency department (ED) utilization is often used as an indicator of poor chronic disease control and/or poor quality of care. We sought to determine if 2 ED utilization measures identify clinically or demographically different populations of children.
Study Design: Retrospective cohort study utilizing IBM Health/Truven MarketScan Medicaid data.
Methods: Children and adolescents were categorized based on the presence and complexity of chronic medical conditions using the 3M Clinical Risk Group system. Children and adolescents were categorized as high ED utilizers using 2 measures: (1) ED reliance (EDR) (number of ED visits / [number of ED visits + number of ambulatory visits]; EDR >0.33 = high utilizer) and (2) visit counts (≥3 ED visits = high utilizer). Logistic regression models identified patient factors associated with each of our outcome measures.
Results: A total of 5,438,541 children and adolescents were included; 65% were without chronic disease (WO-CD), 32% had noncomplex chronic disease (NC-CD), and 3% had complex chronic disease (C-CD). EDR identified 18% as frequent utilizers compared with 7% by the visit count measure. In the visit count model, children younger than 2 years and those classified as WO-CD and NC-CD were less likely to be identified as high utilizers. Conversely, in the EDR model, children and adolescents 2 years and older and those classified as WO-CD and NC-CD were more likely to be classified as high utilizers.
Conclusions: The ED utilization measures identify clinically and demographically different groups of patients. Future studies should consider the medical complexity of the population being studied before choosing the most appropriate measure to employ.
Am J Manag Care. 2020;26(6):267-272. https://doi.org/10.37765/ajmc.2020.43490
In this retrospective cohort study, we determined that the emergency department (ED) reliance measure and ED visit count measure identify different cohorts of children driven by age and medical complexity effects on expected outpatient visits per year.
The increase in emergency department (ED) visit rates in the United States for children and adolescents (33,042 per 100,000 children and adolescents in 2005 to 38,552 per 100,000 in 2016) contributes substantially to rising health care costs.1 Previous studies have shown that a significant proportion of ED visits are nonurgent and potentially avoidable.2 ED utilization can be assessed using administrative claims data, which have been used to identify populations with potentially avoidable ED visits. However, multiple measures of ED utilization exist without a consensus opinion of the most appropriate or accurate measure to use among children and adolescents. Further, these methods have not been examined in large national databases of children and adolescents to determine patient-level factors associated with identification of high ED utilization under each measure.
Two common definitions of high ED utilization include a simple count of annual ED visits (ED visit count) or the percentage of outpatient encounters that occur in the ED (ED visits / [ED visits + outpatient visits]; called ED reliance [EDR]). The ED visit count variably defines 2 to 5 ED visits per year as indicating high-utilizer status,3-5 whereas an EDR greater than 0.33 is considered high.3,6,7 The rationale behind EDR is the potential distinction between high ED use due to severity of chronic illness vs overreliance on the ED as the primary source of outpatient care. Under the EDR measure, a patient is essentially “allowed” 1 ED visit for every 2 outpatient visits. Although EDR considers the total number of outpatient visits, the underlying variation in health care utilization patterns among children and adolescents of different ages and with varying degrees of medical complexity may render it less successful as a one-size-fits-all measure. In addition, previous studies of EDR are limited in their ability to control for individual patient characteristics such as age and medical complexity.3,6
Understanding the limitations of each ED utilization measure will allow effective identification of target populations for interventions, quality-of-care assessments, and policy development aimed at reducing overreliance on the ED. This is relevant to practice managers, payers, and policy makers. With this in mind, our objective is to compare the EDR and ED visit count measures of high utilization using a large, national database of publicly insured children and adolescents to inform our understanding of these commonly reported measures of ED utilization in children. Specifically, we sought to determine patient-level factors associated with high-utilization categorization under each measure.
We conducted a retrospective cohort study of children and adolescents (aged 11 months-21 years) with continuous enrollment in Medicaid for at least 11 months in 2015 using the multistate Truven MarketScan Medicaid Database (Truven Health Analytics). This deidentified data set contains demographic and Medicaid claims data for fee-for-service Medicaid and Medicaid managed care pooled from 7 geographically diverse states. Because of Truven’s data sharing agreement, identification of the individual states is prohibited.
Definition of High ED Utilization
Count. We calculated the number of ED visits per child per year. Based on previous studies and the distribution of our data (7% with ≥3 ED visits per year vs only 4% with ≥4 ED visits per year vs 15% with ≥2 ED visits per year ), children and adolescents were categorized as high utilizers if their ED visit count was 3 or more in the year. This falls within the range of previously reported thresholds for high-utilizer categorization.3-5,8,9
EDR. We calculated patient-level EDR during the study period (number of ED visits / [number of ED visits + number of outpatient visits]). Children and adolescents with an EDR greater than 0.33 were categorized as high utilizers.3,6,7
In this analysis, urgent care visits are considered outpatient visits and not included as ED visits.
Patient Factors Examined
We examined 6 patient factors: disease complexity status, age, race/ethnicity, basis of Medicaid eligibility, Medicaid payment structure, and mental health classification. We categorized children and adolescents as being without chronic disease (WO-CD), having noncomplex chronic disease (NC-CD), or having complex chronic disease (C-CD) using established methods with the 3M Clinical Risk Group (CRG) classification system.10,11 Children and adolescents are classified as having C-CD if they have a significant chronic disease in 2 organ systems (eg, asthma and epilepsy), a progressive condition associated with deteriorating health (eg, muscular dystrophy), continuous dependence on technology (eg, tracheostomy), or malignancy (eg, leukemia).10,11 NC-CDs include both physical and mental health conditions, which may or may not be lifelong, follow a relapsing/remitting course, and are generally not progressive.10,11 Examples of common NC-CDs include asthma, obesity, diabetes, attention-deficit/hyperactivity disorder, and depression. Children and adolescents WO-CD are those who are healthy or who have significant acute illnesses (eg, otitis media, bronchiolitis).10,11 Using administrative claims data, the 3M CRG classification system places patients into CRG groups that, when combined, categorize children and adolescents into these 3 categories.11 For the purposes of this study, CRG 1 and 2 were considered WO-CD, CRG 3 through 6a were considered NC-CD, and CRG 6b through 9 were considered C-CD.12 Children and adolescents were categorized into 5 age categories: 11 months to 23 months, 2 to 5 years, 6 to 11 years, 12 to 18 years, and 19 to 21 years. Infants younger than 11 months were not eligible due to the continuous enrollment requirement. Race/ethnicity is self-reported in Medicaid data and is classified as white non-Hispanic, black non-Hispanic, Hispanic, and other.13 We examined the basis of Medicaid eligibility as a dichotomized variable (qualifying based on disability, yes/no). We also examined Medicaid payment structure (fee for service vs managed care). Finally, because mental health and substance abuse (MHSA) disorders are increasingly prevalent among children and adolescents and have previously been implicated in rising ED costs, we controlled for mental health status.14 MHSA classifications use Truven-defined categories including no MHSA, acute disease, manifestations of chronic disease, minor chronic disease, and moderate/dominant chronic disease.14-16
Categorical variables were summarized using frequencies and percentages, and continuous variables were summarized using medians and interquartile ranges (IQRs). Patients were divided into 1 of 4 ED utilization categories: high utilizer by EDR definition, high utilizer by visit count definition, high utilizer by both definitions, or not high utilizer by either definition. We compared patients across the 4 ED utilization categories using a χ2 test for association for categorical variables and a Kruskal-Wallis test for continuous variables. We then ran 2 multivariable logistic regression models: The first examined factors associated with the probability of being a high ED utilizer by count; the second model examined factors associated with the probability of being a high ED utilizer by EDR. Independent variables included in both models were age category, sex, race/ethnicity, basis of Medicaid eligibility, pay arrangement, CRG complexity classification, and MHSA classification. Due to the large sample size, P values <.001 were considered statistically significant. All statistical analyses were performed using SAS version 9.4 (SAS Institute). This study was not considered human subjects research by the institutional review board at Medical University of South Carolina (Charleston, South Carolina).
We identified 5,438,541 children and adolescents between the ages of 11 months and 21 years who had at least 11 months of continuous enrollment14,15 in Medicaid during 2015 from the Truven MarketScan database (Figure). The median (IQR) age was 9 (5-15) years. Sixty-five percent were classified as WO-CD, 32% as having NC-CD, and 3% as having C-CD. Fifty percent were male, 47% were white, and 33% were black. Sixty-six percent were under a capitated pay arrangement. Nineteen percent had an MHSA disorder (Table 1).
Identification of high ED utilizers. Seven percent of our study population had 3 or more ED visits, meeting our definition of high ED utilization (1.9% met this definition only, whereas an additional 5.1% met both the visit count and EDR definitions). Eighteen percent of the cohort were classified as being high ED utilizers using the EDR definition (EDR >0.33) (12.7% met the EDR definition only, whereas an additional 5.1% met both the EDR and visit count definitions).
Depending on identification method used, the age distribution and degree of medical complexity within the high utilizer group was variable. Children aged 11 to 23 months, who made up 6.4% of the overall cohort, are relatively overrepresented in the high ED visit count group (26.5%) but underrepresented in the high EDR group (3.0%). Adolescents aged 19 to 21 years, making up 7.7% of the overall cohort, are relatively overrepresented in the high ED visit count group (13.2%) and the group that meets both definitions of high utilization (16.8%). For disease complexity, children and adolescents WO-CD, making up 64.7% of the overall cohort, are relatively underrepresented in the high ED visit count group (31.0%), whereas children and adolescents with C-CD, making up 3.2% of our overall cohort, are overrepresented in the high visit count group (16.0%) (Table 1).
Logistic Regression Results
Factors associated with high ED use by EDR. Children and adolescents in all other age categories were more likely than those aged 11 to 23 months to be identified as high ED utilizers, with those aged 19 to 21 years being the most likely (odds ratio [OR], 3.41; 95% CI, 3.37-3.46) followed by those aged 2 to 5 years (OR, 1.65; 95% CI, 1.63-1.67). For disease complexity, children and adolescents WO-CD and children and adolescents with NC-CD were more likely than children and adolescents with C-CD to be identified as high utilizers (OR, 1.45; 95% CI, 1.43-1.47; and OR, 1.25; 95% CI, 1.23-1.26, respectively). Children and adolescents in capitated payment plans were more likely than those in fee-for-service plans to have a high EDR (OR, 1.32; 95% CI, 1.31-1.33). Compared with white non-Hispanic children and adolescents, black children and adolescents were more likely to have a high EDR (OR, 1.62; 95% CI, 1.61-1.63) (Table 2 [A]).
Factors associated with high ED use by count. Children and adolescents in all other age categories were less likely than those aged 11 to 23 months to be identified as high ED utilizers, with those aged 6 to 11 years the least likely (OR, 0.07; 95% CI, 0.07-0.07), followed by those aged 12 to 18 years (OR, 0.10; 95% CI, 0.10-0.10). For medical complexity, children and adolescents WO-CD and children and adolescents with NC-CD were less likely than children and adolescents with C-CD to be identified as high utilizers (OR, 0.10; 95% CI, 0.10-0.10; and OR, 0.37; 95% CI, 0.36-0.38, respectively). Children and adolescents in capitated payment plans were less likely than those in fee-for-service plans to have a high visit count (OR, 0.84; 95% CI, 0.83-0.85). Compared with white non-Hispanic children and adolescents, black children and adolescents were less likely to have a high visit count (OR, 0.71; 95% CI, 0.70-0.72) (Table 2 [B]).
In this large, multistate cohort of Medicaid insured children and adolescents, 7% were identified as high ED utilizers using the ED visit count measure and 18% were identified as high ED utilizers using the EDR measure. High ED utilization was closely associated with demographic and clinical characteristics. These data show that the ED visit count measure does not simply identify a proportionately smaller group of high utilizers compared with the EDR measure but, rather, that the makeup of the high-utilizer group is clinically and demographically different under the 2 measures. These findings are important to consider when determining which utilization measure to use for intervention targeting or quality-of-care measurement.
It is not surprising that younger children are less likely to qualify as high utilizers under the EDR measure, given the well-child check schedule during the first 2 years of life. With a high number of expected outpatient visits during this time, a significantly higher number of ED visits are necessary to meet criteria as a high utilizer under the EDR measure, assuming patients adhere to the recommended well-child check schedule. With 7 scheduled outpatient visits in the first year of life, an infant would need to have 3 or more ED visits to qualify as a high utilizer under the EDR measure, assuming that they had no additional outpatient sick visits for the entire year. Previous studies have shown that infants have the highest annual ED visit rate, at 88.5 visits per 100 infants, and these visits are often low acuity.17 In fact, infants are 1.8 times as likely as children and adolescents of other ages to have high-frequency, low—resource intensity ED visits that are more likely to be potentially avoidable.4,18 In light of the limitations of the EDR measure in this age group, use of an ED visit count or other metric may be more meaningful.
Similarly, medical complexity is an important determinant of meeting high-utilizer criteria under the EDR measure. Healthy children and adolescents or those WO-CD are relatively overrepresented in the EDR high-utilizer cohort, and children and adolescents with C-CD are relatively underrepresented with this measure. This could be due to children and adolescents with C-CD having multiple subspecialty physicians that increase their total number of expected annual outpatient visits, thereby blunting the effect of an ED visit, just as it is with young children with a high number of expected well-child checks. Unlike ED visits in infants, many ED visits for children and adolescents with C-CD are likely necessary, and previous research even suggests that children and adolescents with chronic disease are likely to be more acutely ill when arriving for an ED visit, with increased ED length of stay and higher rates of hospital and pediatric intensive care unit admission from the ED compared with children and adolescents without chronic conditions.19 Previous work describes an association between EDR and medical complexity but includes either a self-reported measure of a child’s disease complexity status or data from a single institution.3,6,19 The current study employs a national multistate sample and an objective measure of disease complexity, minimizing recall bias and ensuring that every child’s complexity is categorized using the same methods.
One additional finding from our analysis is that patients in capitated payment structures (managed care plans) have higher rates of high EDR compared with patients under fee-for-service models. Conversely, these patients in capitated payment structures are less likely to have high ED visit counts compared with patients under fee-for-service models. This suggests that patients in capitated plans have fewer total visits (ED + outpatient) compared with fee-for-service patients. In a previous analysis, we found a higher proportion of patients in fee-for-service payment structures within the C-CD cohort compared with patients with NC-CD and patients WO-CD.12 As we have described in this analysis, patients with C-CD will have more expected total visits per year, increasing the likelihood that they will have a high visit count but decreasing the likelihood that they will meet the threshold for a high EDR. The discrepant relationship between payment structure and measures of ED utilization is likely partially driven by variation in disease complexity by payment structure. This underscores the fact that these 2 measures of ED utilization are identifying different cohorts of patients. Managed care programs were originally designed to provide oversight and care management with the ultimate goal of improving care efficiency and decreasing costs. However, previous studies have found varying effects of Medicaid managed care programs. After the proportion of Medicaid recipients in managed care programs increased dramatically in the 1990s, studies demonstrated conflicting results regarding managed care enrollment’s effect on utilization patterns. Some failed to demonstrate changes in access to, use of, or satisfaction with health care services,7,20 whereas others found increased rates of outpatient visits and concomitant decreased rates of ED visits.21,22 Additionally, in a recent study of Medicaid-insured children and adolescents in foster care, those who transitioned to a managed care program had an increase in primary care visits as expected but those who remained in fee-for-service had greater improvement in EDR.23 Our analysis supports the notion that Medicaid managed care may not contribute to decreased ED utilization as it was designed to.
There are several limitations to this study. The study population is limited to only those children and adolescents enrolled in Medicaid. Our study findings are subject to the validity, accuracy, and completeness of International Classification of Diseases coding, a common limitation in administrative data analyses. CRG classification is based on the patient’s diagnosis codes from previous utilization but not directly linked to utilization count. Thus, although it is possible that children with more utilization events have more opportunities to have diagnoses captured, CRG classification is not inherently circular with utilization. Additionally, although the Truven MarketScan Medicaid database aggregated deidentified data from 7 geographically diverse states, we cannot guarantee that our sample is representative of the national Medicaid population. Our large multistate sample has implications for the 35 million publicly insured children and adolescents nationally.24 We excluded 27% of the population for not maintaining continuous enrollment for at least 11 months during the study year. If more than 1 ED visit occurred in a day, only 1 would be reported in the data and therefore included in our measure of ED utilization. Some studies of appropriate ED utilization exclude visits for injuries and poisonings because the ED is likely the most appropriate place for these visits. We elected to include them in our analysis so that our findings would lend to a practical application of the EDR measure that would not require exclusion of particular ED visits based on diagnosis codes. Additionally, the threshold for high utilization classification by EDR is 0.33, not 0, allowing for some appropriate ED visits (including for injuries and poisonings).
This analysis reveals distinct differences in the populations of children and adolescents identified as high ED utilizers under the EDR measure compared with the ED visit count measure. In particular, we found that age and medical complexity are important determinants of high utilizer classification. These findings highlight the importance of careful consideration when choosing a metric to measure quality of care or identify a population of patients for a targeted intervention. It also supports the concept that illness burden adjustment is necessary when using ED use metrics to compare across different populations (eg, physician groups, health plans). Future work may include categorizing patients identified as high utilizers with high-frequency but low—resource intensity ED visits to more clearly identify appropriate populations for intervention. Author Affiliations: Department of Pediatrics, Medical University of South Carolina (ALA), Charleston, SC; Department of Pediatrics, Children’s Mercy Hospital and School of Medicine, University of Missouri—Kansas City (JB, JC), Kansas City, MO; Department of Pediatrics, Washington University in St Louis (EH), St Louis, MO; Children’s Hospital Association (TR, MH), Lenexa, KS; Department of Pediatrics, Feinberg School of Medicine, Northwestern University (MLM), Chicago, IL; Department of Psychiatry and Behavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles (BTZ), Los Angeles, CA; Department of Pediatrics, Cincinnati Children’s Hospital Medical Center and University of Cincinnati (SSS, KAA), Cincinnati, OH; Department of Pediatrics, Seattle Children’s Hospital and University of Washington (JMN), Seattle, WA.
Source of Funding: None.
Author Disclosures: Dr Auger receives grants for research from the Agency for Healthcare Research and Quality. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (ALA, JB, EH, TR, MLM, BTZ, JC, MH, SSS, JMN, KAA); acquisition of data (TR, MH); analysis and interpretation of data (ALA, JB, EH, TR, MLM, BTZ, JC, MH, SSS, JMN, KAA); drafting of the manuscript (ALA, JB, TR, BTZ, MH, JMN); critical revision of the manuscript for important intellectual content (ALA, JB, EH, TR, MLM, BTZ, JC, MH, SSS, JMN, KAA); statistical analysis (TR, MH); and supervision (ALA, SSS).
Address Correspondence to: Annie Lintzenich Andrews, MD, MSCR, Department of Pediatrics, Medical University of South Carolina, 135 Rutledge Ave, MSC 561, Charleston, SC 29425. Email: firstname.lastname@example.org.REFERENCES
1. Sun R, Karaca Z, Wong HS. Trends in hospital emergency department visits by age and payer, 2006-2015. Healthcare Cost and Utilization Project. March 2018. Accessed April 29, 2020. https://hcup-us.ahrq.gov/reports/statbriefs/sb238-Emergency-Department-Age-Payer-2006-2015.jsp
2. Uscher-Pines L, Pines J, Kellermann A, Gillen E, Mehrotra A. Emergency department visits for nonurgent conditions: systematic literature review. Am J Manag Care. 2013;19(1):47-59.
3. Kroner EL, Hoffmann RG, Brousseau DC. Emergency department reliance: a discriminatory measure of frequent emergency department users. Pediatrics. 2010;125(1):133-138. doi:10.1542/peds.2009-0960
4. Cabey WV, MacNeill E, White LN, James Norton H, Mitchell AM. Frequent pediatric emergency department use in infancy and early childhood. Pediatr Emerg Care. 2014;30(10):710-717. doi:10.1097/PEC.0000000000000233
5. Christensen EW, Kharbanda AB, Velden HV, Payne NR. Predicting frequent emergency department use by pediatric Medicaid patients. Popul Health Manag. 2017;20(3):208-215. doi:10.1089/pop.2016.0051
6. Schlichting LE, Rogers ML, Gjelsvik A, Linakis JG, Vivier PM. Pediatric emergency department utilization and reliance by insurance coverage in the United States. Acad Emerg Med. 2017;24(12):1483-1490. doi:10.1111/acem.13281
7. Alessandrini EA, Shaw KN, Bilker WB, Perry KA, Baker MD, Schwarz DF. Effects of Medicaid managed care on health care use: infant emergency department and ambulatory services. Pediatrics. 2001;108(1):103-110. doi:10.1542/peds.108.1.103
8. Lin SC, Margolis B, Yu SM, Adirim TA. The role of medical home in emergency department use for children with developmental disabilities in the United States. Pediatr Emerg Care. 2014;30(8):534-539. doi:10.1097/PEC.0000000000000184
9. Neuman MI, Alpern ER, Hall M, et al. Characteristics of recurrent utilization in pediatric emergency departments. Pediatrics. 2014;134(4):e1025-e1031. doi:10.1542/peds.2014-1362
10. Simon TD, Cawthon ML, Stanford S, et al; Center of Excellence on Quality of Care Measures for Children With Complex Needs (COE4CCN) Medical Complexity Working Group. Pediatric medical complexity algorithm: a new method to stratify children by medical complexity. Pediatrics. 2014;133(6):e1647-e1654. doi:10.1542/peds.2013-3875
11. Neff JM, Clifton H, Popalisky J, Zhou C. Stratification of children by medical complexity. Acad Pediatr. 2015;15(2):191-196. doi:10.1016/j.acap.2014.10.007
12. Hoefgen ER, Andrews AL, Richardson T, et al. Health care expenditures and utilization for children with noncomplex chronic disease. Pediatrics. 2017;140(3):e20170492. doi:10.1542/peds.2017-0492
13. Race, ethnicity, and language data: standardization for health care quality improvement, 3: defining categorization needs for race and ethnicity data. Agency for Healthcare Research and Quality. September 2012. Updated May 2018. Accessed October 3, 2018. https://www.ahrq.gov/research/findings/final-reports/iomracereport/reldata3.html
14. Bettenhausen JL, Richardson TE, Shah SS, et al. Medicaid expenditures among children with noncomplex chronic diseases. Pediatrics. 2018;142(5):e20180286. doi:10.1542/peds.2018-0286
15. Kuo DZ, Hall M, Agrawal R, et al. Comparison of health care spending and utilization among children with Medicaid insurance. Pediatrics. 2015;136(6):e1521-e1529. doi:10.1542/peds.2015-0871
16. Averill RF, Goldfield NI, Eisenhandler J, et al. Development and evaluation of Clinical Risk Groups (CRGs). Semantic Scholar. Accessed April 29, 2020. https://pdfs.semanticscholar.org/505e/db4f2cf344b206aadc4a351de4ac956c9a60.pdf
17. Niska R, Bhuiya F, Xu J. National Hospital Ambulatory Medical Care Survey: 2007 emergency department summary. Natl Health Stat Report. 2010;(26):1-31.
18. Samuels-Kalow M, Peltz A, Rodean J, et al. Predicting Low-resource-intensity emergency department visits in children. Acad Pediatr. 2018;18(3):297-304. doi:10.1016/j.acap.2017.12.012
19. O’Mahony L, O’Mahony DS, Simon TD, Neff J, Klein EJ, Quan L. Medical complexity and pediatric emergency department and inpatient utilization. Pediatrics. 2013;131(2):e559-e565. doi:10.1542/peds.2012-1455
20. Long SK, Coughlin TA. Impacts of Medicaid managed care on children. Health Serv Res. 2001;36(1, pt 1):7-23.
21. Baker LC, Afendulis C. Medicaid managed care and health care for children. Health Serv Res. 2005;40(5, pt 1):1466-1488.
22. Zuckerman S, Brennan N, Yemane A. Has Medicaid managed care affected beneficiary access and use? Inquiry. 2002;39(3):221-242. doi:10.5034/inquiryjrnl_39.3.221
23. Bright MA, Kleinman L, Vogel B, Shenkman E. Visits to primary care and emergency department reliance for foster youth: impact of Medicaid managed care. Acad Pediatr. 2018;18(4):397-404. doi:10.1016/j.acap.2017.10.005
24. January 2020 Medicaid & CHIP enrollment data highlights. Medicaid. Accessed April 29, 2020. https://www.medicaid.gov/medicaid/program-information/medicaid-and-chip-enrollment-data/report-highlights/index.html