“Frequent flyers” significantly contribute to emergency department (ED) crowding. This study developed a predictive model that can be used to identify high-risk patients and reduce ED revisits.
Objectives: To develop a predictive model that hospitals or healthcare systems can use to identify patients at high risk of revisiting the emergency department (ED) within 30 days and thus reduce unnecessary ED use through proactive interventions.
Study Design: A retrospective analysis of fiscal years (FYs) 2013 and 2014 data from 4 Veterans Affairs hospitals in upstate New York.
Methods: This study developed a predictive model based on administrative data, a publicly available patient classification system, and logistic regression. The study data were from 4 Veterans Affairs hospitals in upstate New York; FY 2013 data were used to predict 30-day revisits in FY 2014. All 22,734 patients with ED visits were included in the analysis. The predictive variables were patient demographics, prior-year utilization, and comorbidities. To prevent overfitting, we validated the model by the split-sample method. The predictive power of the model is measured by C statistics.
Results: In the first model using only patient demographics, the C statistics were 0.568 (95% CI, 0.555-0.580) and 0.556 (95% CI, 0.543-0.568) for the development and validation samples, respectively. In the second model with prior-year utilization added, the C statistics were 0.748 (95% CI, 0.737-0.759) for both samples. In the final model with comorbidities added, the C statistics reached 0.773 (95% CI, 0.762-0.784) and 0.763 (95% CI, 0.753-0.774) for the development and validation samples, respectively.
Conclusions: The predictive model we developed in this study is straightforward to implement and offers significantly higher predictive power than other models reported in the literature. Hospitals and healthcare systems can use it to identify high-risk “frequent flyers” for early interventions to reduce ED revisits.
Am J Manag Care. 2018;24(11):e358-e364Takeaway Points
Providing care in the emergency department (ED) is expensive. The average cost of an ED visit is estimated at $1038 compared with $176 for a primary care visit in the United States.1 Moreover, the number of ED visits grew at roughly twice the population growth rate between 2001 and 2008.2 The CDC reported that the number of ED visits increased from 119.2 million (40.5 visits per 100 persons) in 2006 to 136.3 million (44.5 visits per 100 persons) in 2011.3,4 Furthermore, studies have revealed that “frequent-flyer” patients constitute a key factor of ED crowding, which results in treatment delays and excessive mortality.5,6 Thus, given limited resources, reducing repeated visits is warranted to improve ED effectiveness and timeliness for those truly in need.
To develop interventions to reduce ED revisits, reliable predictive modeling that can identify high-risk patients is desired. However, compared with inpatient 30-day readmissions, which have been measured by CMS since 2012 to adjust payments to hospitals,7 ED revisits have received less attention.8 Although substantial published literature has examined factors influencing ED revisits, research on predictive models is limited.9-22 As a result, the predictive models for ED revisits published in the literature have only moderate predictive power: The highest C statistics reached 0.70 for 30-day revisits2,22 and 0.73 for 6-month revisits.23
In the present study, we developed a statistical model that predicts the risk of revisiting the ED within 30 days of discharge. The model can be used to identify high-risk frequent flyers for proactive intervention. With rapid adoption and use of health information technology and especially electronic health records (EHRs), administrative data are coming increasingly closer to real time and offer greater potential for improving ED care. Our model, based on administrative data and a publicly available patient classification system, can be readily implemented in health systems to reduce unnecessary ED revisits.
Data Source and Study Variables
ED visit data from fiscal years (FYs) 2013 and 2014 in Veterans Healthcare Network Upstate New York (VISN 2 Upstate) were analyzed in this study. VISN 2 is 1 of the 18 networks through which the US Department of Veterans Affairs (VA) delivers care to more than 6 million patients annually. VISN 2 Upstate, with 5 medical centers and 31 outpatient clinics across upstate New York, serves approximately 140,000 patients with an annual budget of $1 billion (starting in FY 2016, VISN 2 was restructured to include New York downstate VA hospitals). In FY 2014 for VISN 2 Upstate, a total of 21,141 patients had ED visits in the 4 medical centers that provided ED services.
We used the VA National Patient Care Database (NPCD) hosted at the VA Information and Computing Infrastructure as the primary data source for this study. The Outpatient Care File (OPC) and clinical stop code 130 were used to identify index ED visits and revisits. In addition to encounter information, such as visit dates and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes, OPC also contains patient demographic and socioeconomic information, such as age, gender, race, and income. Data in NPCD, including OPC, are routinely used in VA operational analysis and research. Most of the data fields, such as visit dates and clinical information like ICD-9-CM codes, are regularly and rigorously validated with strict business rules. The patient income information is means-tested. One exception is that its race information is often incomplete because the VA health system does not mandate veterans to report race. However, for the last several years, the VA has systematically gathered racial and ethnic information from other data sources, such as Medicare and the Department of Defense; as a result, the updated racial and ethnic data are deemed accurate and reliable.24,25
We also used Managerial Cost Accounting (MCA; formerly Decision Support System) files that contain actual patient care costs, rather than amounts claimed or paid as in private health plans. MCA costs are the primary financial data for internal operations and Congressional inquiries. For case mix or comorbidities, we used a publicly available and widely used system, Clinical Classifications Software (CCS), developed by the Agency for Healthcare Research and Quality (AHRQ),26 which classifies patients into 285 homogeneous groups based on ICD-9-CM codes.
The dependent variable in this study was dichotomous (yes = 1, no = 0), indicating whether a patient had any ED revisit(s) within 30 days after being discharged from the ED in FY 2014. The explanatory or predictive variables used in this study were from FY 2013 and can be grouped into 4 categories: (1) demographics (age, sex, marital status, race, disability rating, and period of military service); (2) socioeconomic variables (patient income, homeless [yes = 1, no = 0], and patient insurance status [ie, not covered by any insurance (yes = 1, no = 0), enrolled in Medicare (yes = 1, no = 0), enrolled in Medicaid (yes = 1, no = 0), and covered by private insurance (yes = 1, no = 0)]); (3) prior-year utilization and cost (ED revisit within 30 days [yes = 1, no = 0], number of ED revisits within 30 days, total number of ED visits, number of primary care visits, number of telehealth encounters, total outpatient visits, number of hospitalizations, and total cost); and (4) patient risk or comorbidities (285 clinically homogeneous groups produced by CCS, which is developed by AHRQ26).
The present study did not need or use any identifiable patient private information and therefore had expedited institutional review board review.
Modeling and Analysis
We employed logistic regression to predict the probability or risk of 30-day ED revisits. Logistic regression has been the most extensively used model in predicting binary outcomes where the dependent variable equals 1 if the event happened and otherwise equals 0. The model’s predictive power or discriminative ability is measured by the C statistic, which is defined as the proportion of times that the model correctly discriminates a random pair of individuals with or without an event. It is also equivalent to the area under the receiver operating characteristic curve. A C statistic of 0.5 indicates that the model is no better than flipping a coin, a C statistic of 0.7 to 0.8 suggests that the model has good discriminative ability, and a C statistic of 0.8 or higher indicates great discriminative ability or predictive power.27
To prevent model overfit, we only included variables with P values smaller than .05 (based on the stepwise procedure) in the final regression analysis, and we also calculated shrinkage coefficient, an indicator of overfit.28 Further, we fitted the model by the split-sample method.28,29 With this method, the full sample (after merging the dependent variable from FY 2014 and the independent variables from FY 2013) was randomly split into a development sample (50%) and a validation sample (50%).30 The model was fitted on the development sample and then the estimated coefficients were applied to the validation sample to produce the risk score (probability) and the goodness-of-fit statistics. The split-sample method has been extensively used to prevent predictive models from fitting random noises rather than a true trend. In addition, we also visually examined the prediction accuracy by comparing the predicted and observed numbers of patients with any 30-day revisit(s) in 5 estimated risk categories, which equally divided all of the patients into 5 groups based on their estimated risk. The analyses were conducted using PROC LOGISTIC of SAS 9.3 (SAS Institute Inc; Cary, North Carolina).
To demonstrate the predictive power of different explanatory variables, we configured and fitted 3 models, from simple to comprehensive. For model 1, only demographic and socioeconomic variables were included in the model as the independent variables (ie, age, sex, marital status, race, income, enrolled in Medicaid, enrolled in Medicare, or covered by other private insurance [no insurance status was omitted in the regression as reference]). We also used 3 dummy variables (1 was omitted as reference) as the fixed effect to take into account the potentially different practice patterns among the 4 medical centers. Model 2 included the variables in model 1 and prior-year utilizations and cost (ie, ED revisit within 30 days [yes/no], number of ED revisits within 30 days, total number of ED visits, number of primary care visits, number of telehealth encounters, number of all outpatient visits, number of hospitalizations, and total cost). Model 3 included the variables in model 2 and patient comorbidities based on CCS. The inclusion or exclusion of the variables in the final regression depended on the P values produced by the stepwise procedure.
All 22,734 patients who had ED visits in FY 2014 were included in this study. Among the 22,734 patients, 4937 returned to the EDs within 30 days; the overall 30-day revisit rate was 22%. The total number of ED visits in FY 2014 was 42,192. The independent variables (except the 285 CCS indicator variables) and their descriptive statistics are reported in Table 1. Table 2 shows the top 20 most frequent diagnoses of patients with 30-day ED revisits. All the statistically significant variables (P <.05) in the final model are reported along with their coefficient estimates, P values, odds ratios, and 95% CIs in Table 3 [part A and part B]. Note that the age groups were kept in the model as a convention regardless of their statistical significance (the group aged 75 years or older was omitted as the baseline).
In predicting 30-day ED revisits, the first model included only demographics, socioeconomic characteristics, and the fixed effect of the medical centers, in which 10 variables were statistically significant (P <.05) and kept in the model. The C statistics were 0.568 (95% CI, 0.555-0.580) and 0.556 (95% CI, 0.543-0.568) for the development and validation samples, respectively. In the second model, 13 variables were statistically significant and kept in the model. The C statistics were 0.748 (95% CI, 0.737-0.759) for both samples. In the final model, 51 variables (39 CCS categories) that were statistically significant were kept in the model, and the C statistics were 0.773 (95% CI, 0.762-0.784) and 0.763 (95% CI, 0.753-0.774) for the development and validation samples, respectively. The receiver operating characteristic curves of all 3 models based on the validation sample are reported in Figure 1.
To further examine the prediction accuracy, we compared the predicted and observed numbers of patients with ED revisits by 5 estimated risk categories (% = patients with ED revisits / patients in the risk group). As shown in Figure 2, in both the development and validation samples, the predicted and the observed numbers of revisits were very close except that the model overestimated the number of revisits for the low-risk group. In addition, we also estimated the shrinkage coefficient, which yielded a value of 0.86, indicating no overfit.28
EDs across the country are still overcrowded and the waiting lines remain long. The CDC reported that the average waiting time in EDs in the United States increased by 25%, from 46.5 to 58.1 minutes, from 2003 through 2009.31 In 2015, the American College of Emergency Physicians published a survey report showing that ED visits went up since implementation of the Affordable Care Act.32 Consequentially, waiting for hours before seeing a doctor in the ED is not uncommon.3,4,33 A systematic review showed that repeated ED visits are a key factor contributing to ED crowding, treatment delays, and excessive mortality.5,6
To free ED capacity for true emergencies, reducing unnecessary repeated visits is desired. However, unlike the rate of hospital readmissions, which has been used by CMS to penalize hospitals with excessive readmissions, ED revisits have not received much attention.8 Given limited resources, interventions to reduce ED revisits require accurate predictive modeling that can identify the patients at high risk for revisits, but only limited research focusing on predictive modeling is found in the literature.
In the present study, we developed a statistical model that predicts, with good accuracy, patients who are at high risk of returning to the ED within 30 days. The strength of the model for broad application lies in its 3 features: (1) It uses administrative data that are readily available in nearly all hospitals or health systems, (2) the comorbidity classification system is publicly available from AHRQ, and (3) the model is based on logistic regression, which can be readily implemented with most common software packages, including Microsoft Excel.
Unlike for predicting hospital readmissions,34 comorbidities added limited predictive power to demographics and prior-year utilization/cost: the C statistic increased by 0.19 from model 1 (only demographics were included) to model 2 (prior-year utilization/cost were added), while only improving by 0.015 from model 2 to model 3 (diagnoses were added). Similarly, our sensitivity analyses revealed that prior-year utilization/cost also did not add much predictive power if the patient demographics and comorbidities were entered into the model first, which indicates that prior-year utilization/cost and comorbidities are highly correlated.
It is worth noting that the stepwise procedure for selecting variables is popular but not perfect. In fact, it can easily produce misleading results if the objective is to assess the effect of an independent variable on the dependent variable. In this study, with prediction as the aim, the forward, backward, stepwise, and Lasso procedures produced nearly identical results and we reported the results based on the stepwise procedure.
Although the objective of this study is to forecast or predict ED revisits, rather than to assess the effect of each individual explanatory variable, some variables are of interest and further research is merited. In particular, the number of primary care visits in the prior year was inversely associated with the risk of ED revisits, which is clinically meaningful and consistent with previous findings.35 On the other hand, it appears to be paradoxical that the number of prior-year hospitalizations was also inversely correlated with ED revisits. Similarly, a few severe comorbidities (eg, CCS131, respiratory failure) were associated with low risk of ED revisits. Without knowing the exact reason(s), we speculate that these variables are indicators of disease severity that warrants inpatient rather than ED care,36 and higher mortality may also play a role. Obviously, further research is needed.
In addition, female patients in this study were less likely to experience ED revisits, holding other things constant, but the finding may not be generalizable to the general public because the study population is predominantly male. It is also interesting to see that a higher disability rating was associated with low risk of ED revisits. Although we do not know the exact reason(s), it could be due to the fact that medical needs for disability can be better anticipated and planned.
This study has several limitations. First, the study population included only veterans, and they were from only 1 region (upstate New York). VA patients could differ from the general population. Apart from seeing mostly male patients, EDs at VA hospitals see more patients with mental health issues (eg, posttraumatic stress disorder) than do EDs in the private sector.37 In addition, VA’s co-payment structure is also different from the private sector’s. No co-payment is required for low-income veterans or those with service-connected conditions, which may result in different ED visit patterns. Second, veterans in this study may have received care from non-VA providers, which may also have decreased the model’s predictive power. Third, the present study only analyzed whether or not a patient had any ED revisit(s) at all; future studies may consider multiple revisits within 30 days. Finally, for the approved study period, International Classification of Diseases, Tenth Edition, Clinical Modification (ICD-10-CM) codes were not available and therefore ICD-9-CM codes were used to identify patient comorbidities. However, the modeling process remains the same and the predictive power may be improved if ICD-10-CM codes are used.38
Curtailing unnecessary ED revisits will not only lower healthcare costs but also shorten wait time for those who critically need ED care. However, broad intervention targeting every ED visitor may not be feasible given limited resources. In the present study, we developed a predictive model based on administrative data, a publicly available patient classification system, and commonly used logistic regression, which yielded good predictive power with a C statistic of 0.76 compared with 0.70 for models previously reported in the literature. The model can be readily implemented by hospitals and healthcare systems with EHRs, and the resultant risk score can be used to identify high-risk patients for proactive interventions to reduce ED revisits.Author Affiliations: Stratton VA Medical Center, US Department of Veterans Affairs (KG, GP, LK), Albany, NY.
Source of Funding: This material is based upon work supported in part by the Office of Research and Development, Department of Veterans Affairs.
Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (KG, GP, LK); acquisition of data (KG, LK); analysis and interpretation of data (KG, LK); drafting of the manuscript (KG); critical revision of the manuscript for important intellectual content (GP, LK); statistical analysis (KG, LK); provision of patients or study materials (GP, LK); administrative, technical, or logistic support (GP, LK); and supervision (GP, LK).
Address Correspondence to: Laurence Kaminsky, PhD, Stratton VA Medical Center, US Department of Veterans Affairs, 113 Holland Ave, Albany, NY 12208. Email: firstname.lastname@example.org.REFERENCES
1. White D. Seventy-Two Hour Emergency Department Returns [master’s thesis]. Spokane, WA: Washington State University College of Nursing; 2010. research.wsulibs.wsu.edu/xmlui/bitstream/handle/2376/3447/D_White_010961816.pdf. Accessed October 8, 2018.
2. Hao S, Jin B, Shin AY, et al. Risk prediction of emergency department revisit 30 days post discharge: a prospective study [erratum in PLoS One. 2015;10(1):e0117633. doi: 10.1371/journal.pone.0117633]. PLoS One. 2014;9(11):e112944. doi: 10.1371/journal.pone.0112944.
3. Pitts SR, Niska RW, Xu J, Burt CW. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. Natl Health Stat Report. 2008;(7):1-38.
4. National Hospital Ambulatory Medical Care Survey: 2011 emergency department summary tables. CDC website. cdc.gov/nchs/data/ahcd/nhamcs_emergency/2011_ed_web_tables.pdf. Accessed October 8, 2018.
5. Gaieski DF, Agarwal AK, Mikkelsen ME, et al. The impact of ED crowding on early interventions and mortality in patients with severe sepsis. Am J Emerg Med. 2017;35(7):953-960. doi: 10.1016/j.ajem.2017.01.061.
6. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med. 2008;52(2):126-136. doi: 10.1016/j.annemergmed.2008.03.014.
7. Pizer SD. Should hospital readmissions be reduced through payment penalties? Med Care. 2013;51(1):20-22. doi: 10.1097/MLR.0b013e318277eb92.
8. Institute of Medicine. Hospital-Based Emergency Care: At the Breaking Point. Washington, DC: The National Academies Press; 2006.
9. Hastings SN, Oddone EZ, Fillenbaum G, Sloane RJ, Schmader KE. Frequency and predictors of adverse health outcomes in older Medicare beneficiaries discharged from the emergency department. Med Care. 2008;46(8):771-777. doi: 10.1097/MLR.0b013e3181791a2d.
10. Friedmann PD, Jin L, Karrison TG, et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med. 2001;19(2):125-129. doi: 10.1053/ajem.2001.21321.
11. Yee JL, Hasson NK, Schreiber DH. Drug-related emergency department visits in an elderly veteran population. Ann Pharmacother. 2005;39(12):1990-1995. doi: 10.1345/aph.1E541.
12. Hastings SN, Schmader KE, Sloane RJ, Weinberger M, Goldberg KC, Oddone EZ. Adverse health outcomes after discharge from the emergency department—incidence and risk factors in a veteran population. J Gen Intern Med. 2007;22(11):1527-1531. DOI: 10.1007/s11606-007-0343-9
13. Doran KM, Raven MC, Rosenheck RA. What drives frequent emergency department use in an integrated health system? national data from the Veterans Health Administration. Ann Emerg Med. 2013;62(2):151-159. doi: 10.1016/j.annemergmed.2013.02.016.
14. Goodman RM. Emergency department use associated with primary care office management. Am J Manag Care. 2013;19(5):e185-e196.
15. Rising KL, Padrez KA, O’Brien M, Hollander JE, Carr BG, Shea JA. Return visits to the emergency department: the patient perspective. Ann Emerg Med. 2015;65(4):377-386.e3. doi: 10.1016/j.annemergmed.2014.07.015.
16. Sauvin G, Freund Y, Saïdi K, Riou B, Hausfater P. Unscheduled return visits to the emergency department: consequences for triage [erratum in Acad Emerg Med. 2013;20(3):327. doi: 10.1111/acem.12124]. Acad Emerg Med. 2013;20(1):33-39. doi: 10.1111/acem.12052.
17. Hasegawa K, Tsugawa Y, Tsai CL, Brown DF, Camargo CA Jr. Frequent utilization of the emergency department for acute exacerbation of chronic obstructive pulmonary disease. Respir Res. 2014;15:40. doi: 10.1186/1465-9921-15-40.
18. Hastings SN, Smith VA, Weinberger M, Schmader KE, Olsen MK, Oddone EZ. Emergency department visits in Veterans Affairs medical facilities. Am J Manag Care. 2011;17(6):e215-e223.
19. Wu CL, Wang FT, Chiang YC, et al. Unplanned emergency department revisits within 72 hours to a secondary teaching referral hospital in Taiwan. J Emerg Med. 2010;38(4):512-517. doi: 10.1016/j.jemermed.2008.03.039.
20. Chan AH, Ho SF, Fook-Chong SM, Lian SW, Liu N, Ong ME. Characteristics of patients who made a return visit within 72 hours to the emergency department of a Singapore tertiary hospital. Singapore Med J. 2016;57(6):301-306. doi: 10.11622/smedj.2016104.
21. Tsai IT, Sun CK, Chang CS, Lee KH, Liang CY, Hsu CW. Characteristics and outcomes of patients with emergency department revisits within 72 hours and subsequent admission to the intensive care unit. Ci Ji Yi Xue Za Zhi. 2016;28(4):151-156. doi: 10.1016/j.tcmj.2016.07.002.
22. McCusker J, Cardin S, Bellavance F, Belzile E. Return to the emergency department among elders: patterns and predictors. Acad Emerg Med. 2000;7(3):249-259. doi: 10.1111/j.1553-2712.2000.tb01070.x.
23. Jin B, Zhao Y, Hao S, et al. Prospective stratification of patients at risk for emergency department revisit: resource utilization and population management strategy implications. BMC Emerg Med. 2016;16:10. doi: 10.1186/s12873-016-0074-5.
24. Stroupe KT, Tarlov E, Zhang Q, Haywood T, Owens A, Hynes DM. Use of Medicare and DoD data for improving VA race data quality. J Rehabil Res Dev. 2010;47(8):781-795.
25. Trivedi AN, Grebla RC, Wright SM, Washington DL. Despite improved quality of care in the Veterans Affairs health system, racial disparity persists for important clinical outcomes. Health Aff (Millwood). 2011;30(4):707-715. doi: 10.1377/hlthaff.2011.0074.
26. Clinical Classifications Software (CCS) for ICD-9-CM. Healthcare Cost and Utilization Project website. hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. Updated March 6, 2017. Accessed October 8, 2018.
27. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688-1698. doi: 10.1001/jama.2011.1515.
28. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361-387. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.
29. Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78(382):316-331.
30. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2012;51(4):368-373. doi: 10.1097/MLR.0b013e31827da95a.
31. Hing E, Bhuiya F. Wait time for treatment in hospital emergency departments: 2009 [NCHS data brief no. 102]. CDC website. cdc.gov/nchs/products/databriefs/db102.htm. Published August 2012. Accessed October 8, 2018.
32. ER visits continue to rise since implementation of Affordable Care Act [news release]. Washington, DC: American College of Emergency Physicians; May 4, 2015. newsroom.acep.org/2015-05-04-ER-Visits-Continue-to-Rise-Since-Implementation-of-Affordable-Care-Act. Accessed October 8, 2018.
33. Horwitz LI, Green J, Bradley EH. US emergency department performance on wait time and length of visit. Ann Emerg Med. 2010;55(2):133-141. doi: 10.1016/j.annemergmed.2009.07.023.
34. Shulan M, Gao K, Moore CD. Predicting 30-day all-cause hospital readmissions. Health Care Manag Sci. 2013;16(2):167-175. doi: 10.1007/s10729-013-9220-8.
35. Goodman RM. Emergency department use associated with primary care office management. Am J Manag Care. 2013;19(5):e185-e196.
36. Pellerin G, Gao K, Kaminsky L. Predicting 72-hour emergency department revisits. Am J Emerg Med. 2018;36(3):420-424. doi: 10.1016/j.ajem.2017.08.049.
37. Kessler CS, Bhandarkar S, Casey P, Tenner A. Predicting patient patterns in Veterans Administration emergency departments. West J Emerg Med. 2011;12(2):204-207.
38. Beta Clinical Classifications Software (CCS) for ICD-10-CM/PCS. Healthcare Cost and Utilization Project website. hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp. Updated March 23, 2018. Accessed October 8, 2018.