Joshua R. Vest, PhD, MPH

 Health care organizations are increasingly employing social workers to address patients’ social needs. However, social work (SW) activities in health care settings are largely captured as text data within electronic health records (EHRs), making measurement and analysis difficult. This study aims to extract and classify, from EHR notes, interventions intended to address patients’ social needs using natural language processing (NLP) and machine learning (ML) algorithms.

 Secondary data analysis of a longitudinal cohort.

 We extracted 815 SW encounter notes from the EHR system of a federally qualified health center. We reviewed the literature to derive a 10-category classification scheme for SW interventions. We applied NLP and ML algorithms to categorize the documented SW interventions in EHR notes according to the 10-category classification scheme.

Most of the SW notes (n = 598; 73.4%) contained at least 1 SW intervention. The most frequent interventions offered by social workers included care coordination (21.5%), education (21.0%), financial planning (18.5%), referral to community services and organizations (17.1%), and supportive counseling (15.3%). High-performing classification algorithms included the kernelized support vector machine (SVM) (accuracy, 0.97), logistic regression (accuracy, 0.96), linear SVM (accuracy, 0.95), and multinomial naive Bayes classifier (accuracy, 0.92).

NLP and ML can be utilized for automated identification and classification of SW interventions documented in EHRs. Health care administrators can leverage this automated approach to gain better insight into the most needed social interventions in the patient population served by their organizations. Such information can be applied in managerial decisions related to SW staffing, resource allocation, and patients’ social needs.

Social work (SW) interventions are increasingly important in health care.

Most of the SW notes examined in this study contained at least 1 SW intervention. In approximately 37% of SW notes, more than 1 intervention was identified, underscoring the need to address coexisting social needs among a sizable population.

The most frequent SW interventions offered included care coordination, education, financial planning, referral to community services and organizations, and supportive counseling.

Natural language processing and machine learning are inexpensive tools for automated identification and classification of SW interventions.

This automated identification can support managerial decisions related to social worker staffing and allocation of resources to address patients’ social needs.

Traditionally, efforts to improve health outcomes have primarily focused on medical care services.

 However, consistent evidence suggests that social needs and risk factors have a more profound effect on individual and population health than medical care.

 Given their professional training and workflows, social workers within health care settings are uniquely positioned to deliver interventions to address patients’ social needs.

 Moreover, social workers are increasingly becoming embedded in health care organizations to help address patients’ social needs.

 However, tracking and evaluating interventions instituted by social workers remains challenging because social work (SW) interventions are largely documented as unstructured text data within electronic health records (EHRs).

 Unstructured documentation makes it difficult to systematically monitor and study SW interventions and services, as manual chart reviews are expensive and time-consuming and often require expert reviewers.

 These limitations underscore the need for using novel information extraction methods, such as natural language processing (NLP) and machine learning (ML), to identify and classify interventions documented in unstructured EHR notes, such as SW notes.

Existing research has been successful in using NLP methods to identify and classify social needs with EHR data. Dorr et al

 demonstrated that NLP could be used to identify chronic stress, social isolation, financial insecurity, and housing insecurity in the clinical notes of primary care doctors. Conway et al

 developed the Moonstone system, which used rule-based classification to identify housing situations and social support within EHRs. Also, Cook et al

 used NLP and ML techniques to predict suicidal ideation from a text message intervention. These studies demonstrated the feasibility and applicability of using NLP approaches to detect social needs and risks in unstructured clinical data. Although a small number of studies have developed classification schemes for SW interventions using manual chart reviews, no study has sought to utilize NLP methods to identify and/or classify SW interventions.

The purpose of this study was to extract and categorize SW interventions aimed at addressing patients’ social needs by developing and applying NLP and ML algorithms. Development of such classification tools supports health care organizations’ assessment and measurement of a growing part of the nonmedical workforce. Additionally, this study highlights methodological approaches to examine SW interventions in situations in which only unstructured data are accessible. Health care organizations armed with data on SW interventions that have been instituted for their patient population will make more informed resource allocation, staffing, quality improvement, and program design decisions to address their patients’ social needs. Moreover, automation of the classification of interventions will offer stakeholders an enhanced ability to quantify the specific impact of social workers’ interventions on patients’ health outcomes.

We developed a classification scheme from the literature

 to categorize interventions instituted by social workers to address patients’ social needs at Eskenazi Health, an urban safety-net health system. Eskenazi Health employs in-house social workers to address patients’ social needs. These needs are identified through multiple pathways. For example, physicians and nurse practitioners may identify patients’ social needs during consultations and refer patients to social workers. Additionally, social workers may review records of patients with scheduled appointments to identify social needs.

We used NLP and ML algorithms for automated categorization of SW interventions based on the classification scheme we developed. The 

describes, in brief, the process and pipeline used for this classification (see 

]). This study was approved by the Indiana University Institutional Review Board.

We used patient record data from Eskenazi Health, a safety-net provider with a 300-bed hospital and a federally qualified health center serving the Indianapolis, Indiana, metropolitan area. The study sample included 408 patients with 815 SW encounters between October 1, 2016, and September 30, 2019.

The study data were derived from Eskenazi Health’s EHR (Epic). The EHR data included patients’ clinical notes and their demographic and clinical characteristics. We obtained unstructured data containing free-text descriptions of the reason(s) for patient visits, the intervention(s) instituted by the social worker, and future visits or engagement plans from social workers’ notes. We extracted SW notes, using an existing pipeline (nDepth

), by searching clinical notes for the following search terms: “social work” or “MSW” or “LCSW”, or “LSW”, or “LMSW”, or “LBSW.” nDepth, an NLP tool developed by the Regenstrief Institute, conducts information retrieval and extraction from textual documents by adapting its existing pipeline to customizable search queries. nDepth has been used in other clinical domains, including gastrointestinal diseases and sarcopenia.

 For this study, nDepth was used to extract a sample of SW notes from the EHR. Our search with nDepth retrieved 1289 clinical notes. Three abstractors (A.T.B., H.L.T., K.W.) manually reviewed the retrieved notes and categorized them into notes written by a social worker and notes written by other professionals. Notes were deemed as written by a social worker if they were signed off by a social worker or if the wordings and structure of the notes clearly indicated documentation of activities by a social worker. The final sample for this study included 815 SW encounters with notes that were determined to be written by social workers during distinct SW encounters (see 

Based on literature review and expert consultations, 3 members of the research team (A.T.B., H.L.T., K.W.) developed a classification scheme for SW interventions, with 10 non–mutually exclusive categories of SW interventions briefly defined as follows:

resources for funding medications, treatment, or other care.

emotional and mental health support services; addiction services.

coordination of care continuity, case management, and arranging and coordinating home visits.

 health/wellness programs, crisis intervention, and course planning.

referrals to community, spiritual, and peer advocacy organizations; translational services; food assistance; clothing assistance.

 help with filing applications; filing mandatory reports such as those to Department of Child Services and Adult Protective Services.

provision of transportation fares and passes; linkage with transportation services.

provision of medical equipment; repair, maintenance, and/or replacement of medical equipment.

link to attorneys, law enforcement, and other legal services.

 3 coders (A.T.B., H.L.T., K.W.) independently reviewed 100 randomly selected SW notes to identify verbs likely to indicate the presence of SW interventions in a sentence. The abstractors also identified contextual terms and negation terms associated with each verb, used to describe a social worker (social worker terms), as well as unique terms that map to each of the 10 intervention categories (see 

 for lexicon of terms). The abstractors manually coded the selected notes into their respective non–mutually exclusive SW intervention categories.We calculated the rate of agreement among the 3 coders using Fleiss’s kappa coefficients (see 

To categorize the notes based on our 10-category scheme, we used (1) rule-based classification algorithms using Python’s regular expressions

; (2) ML algorithms such as the multinomial naive Bayes classification algorithm

 and multilabel (one vs rest, binary relevance, classifier chains, and label powerset)

 classification algorithms using logistic regression

 and kernelized support vector machine (SVM) with radial basis function

; and (3) a deep learning algorithm for multilabel classification: the long short-term memory (LSTM) recurrent neural network with a sigmoid activation function and binary cross-entropy loss function.

 The LSTM model has a single input layer, an embedding layer, 1 LSTM layer with 128 nodes, and 1 output layer with 10 nodes, with each node in the output dense layer representing 1 of the SW intervention categories.

For the rule-based algorithm, we used Python’s

 in-built regular expressions to extract sentences that include a social worker term and one of the intervention verb terms, where the verb term is not preceded by one of the negative-lookbehind terms for intervention and not followed by one of the negative-lookahead terms. A review of SW notes and consultation with clinical social workers revealed that these sentences are likely to contain information about the type of intervention instituted. An SW note was deemed to indicate the presence of an intervention category if the extracted sentences in the note also contain at least 1 of the intervention key terms for that category.

For the implementation of the ML and deep learning algorithms, we randomly divided the data into training (68%), validation (12%), and test (20%) sets (see Figure). We preprocessed the text in each of these data sets as follows: We tokenized sentences in each SW note into individual words (tokens), removed stop words (eg, 1- and 2-letter words, compositions, prepositions), applied part of speech tagging to the tokens, and stemmed the tokens down to their root words. Next, for implementation of the ML algorithms, we created feature vectors using the term frequency–inverse document frequency vectorizer with single words (unigrams), 2 consecutive words (bigrams), and 3 consecutive words (trigrams). Relatedly, for the LSTM model, we created word embeddings using the GloVe word embeddings

 to convert text inputs to their numeric counterparts.

We used the preprocessed feature vectors (or word embeddings in case of the LSTM model) from the training data set to initially train each classifier and the validation data set to test the accuracy of the trained classifier. We implemented multiple iterations of training and validation for each classifier while tuning and optimizing the hyperparameters of the classification algorithm to attain higher accuracy. Upon achieving the highest accuracy score possible for each classifier, we coalesced the training and validation sets and trained a final classifier on them using the optimal hyperparameters. We also used 5-fold cross-validation to evaluate the performance of the logistic regression, kernelized SVM, linear SVM, and multinomial naive Bayes algorithms on the full training data (see 

). Finally, for each intervention category, we evaluated the performance of the rule-based, logistic regression, kernelized SVM, linear SVM, and multinomial naive Bayes algorithms on test data using accuracy; precision or positive predictive value (PPV); recall (sensitivity); F1-score, which is the harmonic mean of precision (PPV) and recall (sensitivity); specificity; and area under the curve (AUC).

Our final sample included 408 patients with 815 SW encounters. Descriptive information about the sample and categories of social worker interventions is available in 

. Briefly, 43% of the patients were Hispanic; their mean age was 38.7 years; and the majority (64%) were women.

Of the 815 SW encounters, almost three-fourths (n = 598; 73.4%) contained at least 1 SW intervention. More specifically, 217 (26.6%) did not include any description of a SW intervention, 295 (36.2%) included 1 SW intervention, 207 (25.4%) included 2 interventions, and 96 (11.8%) included 3 or more interventions. The highest number of interventions in a single SW note was 6, which was observed in only 5 of the 815 encounters. The most common SW interventions in the notes included care coordination (21.5%), education (21.0%), financial planning (18.5%), referral to community services and organizations (17.1%), and supportive counseling (15.3%) (

Models with the highest accuracy included multilabel (one vs rest) classifier with kernelized SVM (accuracy, 0.97), multilabel (one vs rest) classifier with logistic regression (accuracy, 0.96), linear SVC (accuracy, 0.95), and multinomial naive Bayes classifier (accuracy, 0.92) (see 

). Precision (PPV) score was generally higher than the recall (sensitivity) score in the kernelized SVM, logistic regression, and linear SVM for most of the intervention categories (

). However, for the multinomial naive Bayes classifier, the recall score was higher than the sensitivity score in the financial planning, care coordination, community service, and education intervention categories. Moreover, the multinomial naive Bayes classifier offered the highest recall scores for the financial planning (recall, 0.86) and community service (recall, 0.92) categories (Table 2 [B]). Linear SVM provided the best evaluation metrics for the financial planning category (accuracy, 0.96; precision, 0.91, recall, 0.91; F1-score, 0.91; specificity, 0.98; AUC, 0.94). The F1-scores for the logistic regression and kernelized SVM algorithms in the care coordination, community services, and transportation categories were high (0.82-0.94). The multinomial naive Bayes algorithms had the worst F1-score (0.00) in the legal intervention category. Linear SVM offered the best evaluation metrics for the housing (accuracy, 1.00; precision, 1.00; recall, 1.00; F1-score, 1.00; specificity, 1.00; AUC, 1.00) and durable medical equipment (accuracy, 0.99; precision, 0.80; recall, 1.00; F1-score, 0.89; specificity, 0.99; AUC, 0.90) categories (Table 2 [A-C]).

This study described the extent and nature of SW activities in primary care using NLP algorithms. These findings illustrate the key roles that social workers play in connecting patients to other aspects of the health care system, brokering connections with non–health care–related organizations, and providing patient education.

As brokers, social workers link patients to critically needed resources and services.

 This broker function was evident in our study, in which care coordination interventions and referrals to community-based organizations were frequently offered. SW interventions require brokering services for patients among different organizations/providers, maintaining communication, and identifying needed resources from referral partners. These efforts may lead to improved clinical outcomes. For instance, Bronstein et al

 found that social worker involvement in postdischarge care coordination, which involved directing patients to organizations that address food insecurity, clothing, and peer advocacy needs, was associated with fewer 30-day hospital readmissions. Pruitt et al

 showed that referrals to community-based organizations offered by social workers were associated with lower health care costs. Our study provided evidence that social workers in primary care offer interventions known to aid health care organizations’ efforts to improve quality of care, reduce costs, and effectively manage population health. We intend to investigate the value of SW interventions toward improving quality outcomes in our future studies.

In our study, social workers educated patients, in 21% of encounters, about the availability of resources and interpretation of medical information. Patient education contributes significantly toward better health outcomes.

 The role of social workers as providers of patient education thus highlights the significance of social workers in primary care teams. Also, financial planning, which involves assisting patients with medication, insurance, and benefits, was a common SW intervention in this study. Similar to previous studies,

 our study found that social workers primarily address the impact of finances on health care access by providing assistance with medication, insurance, and benefits, rather than directly providing monetary assistance.

Social workers are the largest group of providers of mental health services in the United States.

 In Indiana, social workers can offer a wide range of mental health services, including counseling and treatment for substance use, provided that they obtain a state license to practice mental health.

 However, some authors have opined that social workers employed in health care settings may not have the time to directly provide mental health services, given the need to frequently coordinate and allocate resources for these patients.

 For example, in a study on patients with diabetes, Rabovsky et al

 found that social workers did not directly address patients’ mental health issues. Rather, they referred patients to mental health providers. The practice pattern and/or expertise of the social workers in Rabovsky et al

 may explain why social workers outsourced counseling interventions to external organizations rather than providing the counseling services themselves. In our study, social workers addressed mental and behavioral health issues by directly providing supportive counseling interventions to patients. Thus, it is conceivable, given our findings, that social workers can multitask and interchange roles, thereby providing patients in health care settings with direct counseling and tangible resources or referrals. Finally, in previous studies, the nature of the advocacy role of social workers was not evident and underexplored.

 However, in our study, we found that social workers helped patients in completing and filing insurance coverage or housing applications and in reporting child and adult domestic issues to the appropriate social agencies. This highlights the role of social workers as patient advocates, particularly for those most vulnerable.

 we found that approximately two-fifths of the SW notes included more than 1 SW intervention, indicating that a substantial number of patients have co-occurring social needs.

 The presence of a sizable proportion of patients receiving multiple social interventions indicates the need for effective SW staffing and differential use of more experienced social workers to manage patients with co-occurring social needs.

Although unstructured free-text narratives within EHRs are conducive to NLP- and ML-based classification methods, previous research suggests that social needs interventions are rarely explored using these methods.

 Our study shows that NLP and ML can effectively be used to determine the types of SW interventions suggested by social workers operating within health care systems. Consistent with the findings of previous studies, ML- and deep learning–based classification algorithms performed better than rule-based classification methods.

 However, the multilabel LSTM model did not perform as well as some ML approaches. This finding may be due to the small sample size in this study, as deep learning algorithms are known to perform poorly when the sample size is small.

This study has several limitations. First, although we derived our SW intervention categories from consultation with experts and peer-reviewed literature, our classification scheme may not be exhaustive. Second, the small nature of our sample may limit the performance of our classification algorithms on new test data. However, for most of the intervention categories our evaluation metrics are satisfactory. In addition, our models were trained using data from a single health system, which weakens the generalizability of our findings to other hospital systems or other diverse populations. Lastly, the range of intervention categories and their relative frequencies in this study may be a reflection of the characteristics of the population under study and/or the practice pattern of the social workers in our study site, rather than the general primary care population.

Contextual details of interventions instituted by social workers, which are available in EHR notes, highlight how social needs are addressed by social workers. NLP and ML can be utilized for automated identification and classification of SW interventions documented in EHRs. These methods have the advantage of being scalable and can be automated and integrated into EHR systems. Thus, NLP and ML can be leveraged by health care administrators to gain better insight into the most needed social interventions in their patient populations, thereby helping organizations make better decisions related to SW staffing, resource allocation, and patients’ social needs.

The authors thank the Robert Wood Johnson Foundation for providing the financial support necessary for conducting this research through their Systems for Action National Coordinating Center (ID 73485). They also thank Data Core Services of the Regenstrief Institute for providing them with the data used in this study.

Research reported in this publication/press release was supported by the National Library of Medicine of the National Institutes of Health under award No. T15LM012502. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Library of Medicine.

 Department of Health Policy & Management, Indiana University Richard M. Fairbanks School of Public Health (ATB, HLT, KW, JRV), Indianapolis, IN; Center for Outcomes Research, Houston Methodist Research Institute (ATB), Houston, TX; Center for Biomedical Informatics, Regenstrief Institute (HLT, KW, SNK, JRV), Indianapolis, IN; School of Informatics and Computing, Indiana University–Purdue University Indianapolis (JZ), Indianapolis, IN; Wayne State University School of Social Work (HW-M), Detroit, MI; Wayne State University School of Law (HW-M), Detroit, MI; Department of Pediatrics, Indiana University School of Medicine (SNK), Indianapolis, IN.

 Robert Wood Johnson Foundation (ID 73485) and National Library of Medicine of the National Institutes of Health (T15LM012502).

 Dr Kasthurirathne is a founder of Uppstroms, LLC, and reports patents pending and stock ownership. Dr Vest is a founder and board member of Uppstroms, LLC. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (ATB, JZ, SNK, JRV); acquisition of data (HLT, SNK, JRV); analysis and interpretation of data (ATB, HLT, KW, HW-M, SNK); drafting of the manuscript (ATB, KW, SNK); critical revision of the manuscript for important intellectual content (ATB, HLT, KW, JZ, SNK, JRV); statistical analysis (ATB); provision of patients or study materials (JRV); administrative, technical, or logistic support (HW-M, SNK, JRV); and supervision (JZ, HW-M, SNK).

 Abdulaziz Tijjani Bako, MPH, Department of Health Policy and Management, Richard M. Fairbanks School of Public Health, Indiana University–Purdue University Indianapolis, 1050 Wishard Blvd, Indianapolis, IN 46202. Email: atbako@iu.edu.

1. Heiman HJ, Artiga S. Beyond health care: the role of social determinants in promoting health and health equity. Kaiser Family Foundation. November 2015. Accessed April 24, 2020. https://www.issuelab.org/resources/22899/22899.pdf

2. WHO Commission on Social Determinants of Health, World Health Organization. 

Closing the Gap in a Generation: Health Equity Through Action on the Social Determinants of Health

3. Seabrook JA, Avison WR. Socioeconomic status and cumulative disadvantage processes across the life course: implications for health outcomes. 

. 2012;49(1):50-68. doi:10.1111/j.1755-618x.2011.01280.x

4. Marmot MG, Shipley MJ, Rose G. Inequalities in death—specific explanations of a general pattern? 

. 1984;323(8384):1003-1006. doi:10.1016/s0140-6736(84)92337-7

5. Galea S, Tracy M, Hoggatt KJ, Dimaggio C, Karpati A. Estimated deaths attributable to social factors in the United States. 

. 2011;101(8):1456-1465. doi:10.2105/AJPH.2010.300086

6. Ruth BJ, Wachman MK, Marshall JW, et al. Health in all social work programs: findings from a US national analysis. 

. 2017;107(S3):S267-S273. doi:10.2105/AJPH.2017.304034

7. Occupational employment statistics. U.S. Bureau of Labor Statistics. March 31, 2020. Accessed May 19, 2020. https://www.bls.gov/oes/tables.htm

8. Vest JR, Grannis SJ, Haut DP, Halverson PK, Menachemi N. Using structured and unstructured data to identify patients’ need for services that address the social determinants of health. 

. 2017;107:101-106. doi:10.1016/j.ijmedinf.2017.09.008

9. National Association of Social Workers, Ohio Chapter. Clinical documentation. National Association of Social Workers. Accessed November 6, 2020. https://cdn.ymaws.com/www.naswoh.org/resource/resmgr/Private_Practice/Clinical_Documentation_NASW_.pdf

10. Kim EJ, Abrahams S, Uwemedimo O, Conigliaro J. Prevalence of social determinants of health and associations of social needs among United States adults, 2011-2014. 

. 2020;35(5):1608-1609. doi:10.1007/s11606-019-05362-3

11. Hatef E, Rouhizadeh M, Tia I, et al. Assessing the availability of data on social and behavioral determinants in structured and unstructured electronic health records: a retrospective analysis of a multilevel health care system. 

12. Kusnoor SV, Koonce TY, Hurley ST, et al. Collection of social determinants of health in the community clinic setting: a cross-sectional study. 

. 2018;18(1):550. doi:10.1186/s12889-018-5453-2

13. Dorr D, Bejan CA, Pizzimenti C, Singh S, Storer M, Quinones A. Identifying patients with significant problems related to social determinants of health with natural language processing. 

. 2019;264:1456-1457. doi:10.3233/SHTI190482

14. Conway M, Keyhani S, Christensen L, et al. Moonstone: a novel natural language processing system for inferring social risk from clinical narratives. 

. 2019;10(1):6. doi:10.1186/s13326-019-0198-0

15. Cook BL, Progovac AM, Chen P, Mullin B, Hou S, Baca-Garcia E. Novel use of natural language processing (NLP) to predict suicidal ideation and psychiatric symptoms in a text-based mental health intervention in Madrid. 

. 2016;2016:8708434. doi:10.1155/2016/8708434

16. Abeyta N, Freeman ES, Primack D, et al. SCIRehab Project series: the social work/case management taxonomy. 

. 2009;32(3):336-342. doi:10.1080/10790268.2009.11760787

17. Pockett R, Lord B, Dennis J. The development of an Australian national classification system for social work practice in health care. 

. 2001;34(1-2):177-193. doi:10.1080/00981380109517025

18. Anderson RL, Estle G. Predicting level of mental health care among children served in a delivery system in a rural state. 

. 2001;17(3):259-265. doi:10.1111/j.1748-0361.2001.tb00963.x

19. Viron M, Bello I, Freudenreich O, Shtasel D. Characteristics of homeless adults with serious mental illness served by a state mental health transitional shelter. 

. 2014;50(5):560-565. doi:10.1007/s10597-013-9607-5

20. Rosen A, Proctor EK, Staudt M. Targets of change and interventions in social work: an empirically based prototype for developing practice guidelines. 

. 2003;13(2):208-233. doi:10.1177/1049731502250496

21. Hammond FM, Gassaway J, Abeyta N, Freeman ES, Primack D. The SCIRehab project: social work and case management. social work and case management treatment time during inpatient spinal cord injury rehabilitation. 

. 2011;34(2):216-226. doi:10.1179/107902611X12971826988291

22. nDepth. Regenstrief Institute. Accessed September 22, 2019. https://www.regenstrief.org/implementation/ndepth/

23. Imler TD, Ring N, Crabb DW. Medical, social, and legal risks to predict alcoholic liver disease using natural language processing and advanced analytics. 

. 2015;148(4, suppl 1):S499. doi:10.1016/S0016-5085(15)31674-7

24. Moorthi RN, Liu Z, El-Azab SA, et al. Sarcopenia, frailty and cachexia patients detected in a multisystem electronic health record database. 

. 2020;21(1):508. doi:10.1186/s12891-020-03522-9

25. Social workers in hospitals & medical centers: occupational profile. National Association of Social Workers. 2011. Accessed April 24, 2020. https://www.socialworkers.org/LinkClick.aspx?fileticket=o7o0IXW1R2w%3D&portalid=0

26. Clinical social work: your guide in 2020. Online MSW Programs. Accessed December 2, 2020.
https://www.onlinemswprograms.com/careers/types-of-social-work/guide-to-clinical-social-work.html

27. What is discharge planning for social workers? Social Solutions. May 27, 2013. Accessed December 18, 2019. https://www.socialsolutions.com/blog/what-is-discharge-planning-for-social-workers/

28. Role of school social worker. School Social Work Association of America. Accessed December 18, 2019. https://www.sswaa.org/school-social-work

29. What types of community services are available? PediatricEducation.org. November 16, 2009. Accessed December 18, 2019. https://pediatriceducation.org/2009/11/16/what-types-of-community-services-are-available/

30. Harkema H, Dowling JN, Thornblade T, Chapman WW. ConText: an algorithm for determining negation, experiencer, and temporal status from clinical reports. 

. 2009;42(5):839-851. doi:10.1016/j.jbi.2009.05.002

32. Kibriya AM, Frank E, Pfahringer B, Holmes G. Multinomial naive Bayes for text categorization revisited. In: 

AI 2004: Advances in Artificial Intelligence

; 2004:488-499. doi:10.1007/978-3-540-30549-1_43

Articles

Natural language processing can be used for automated extraction of social work interventions from electronic health records, thereby supporting social work staffing and resource allocation decisions.

Using Natural Language Processing to Classify Social Work Interventions

 To measure the impact of hospital participation in Meaningful Use (MU) on disparities in 30-day readmissions associated with race.

 A retrospective cohort study that compared the likelihood of 30-day readmission for Medicare beneficiaries discharged from hospitals participating in Stage 1 of MU with the likelihood of readmission for beneficiaries concurrently discharged from hospitals that were not participating in the initiative.

Inpatient claims for 2,414,205 Medicare beneficiaries from Florida, New York, and Washington State were used as the primary data source. The study period (2009-2013) included at least 2 years of baseline data prior to each hospital initiating participation in MU. Estimates were derived with linear regression models that included hospital and time fixed effects. By including both hospital and time fixed effects, estimates were based on discharges from the same hospital in the same time period.

 MU participation among hospitals was not associated with a statistically significant change in readmissions for the broader Medicare population (percentage points [PP], 0.6; 95% CI, —0.2 to 1.4), but hospitals’ participation in the initiative was associated with a lower likelihood of readmission for African American beneficiaries (PP, –0.9; 95% CI, –1.5 to –0.4).

 Hospital participation in MU reduced disparities in 30-day readmissions for African American Medicare beneficiaries.

Hospital participation in Meaningful Use (MU) was not associated with declines in 30-day readmissions for the broader Medicare population.

Hospital participation in MU was associated with a decline in readmissions of 0.9 percentage points for African American Medicare beneficiaries.

Readmissions among Medicare beneficiaries are common and often reflect care of poor quality.

 Racial minorities are more likely to be readmitted than other Medicare beneficiaries.

 Policy makers have been optimistic that hospitals’ adoption of health information technology (IT) will lead to lower readmission rates,

 and reducing health disparities is a priority of the federal Meaningful Use (MU) initiative.

 A recent study evaluated the impact of hospitals’ participation in MU on readmission rates of Medicare fee-for-service (FFS) beneficiaries,

 but the relationship between hospitals’ participation in the initiative and racial disparities in readmissions has not been examined.

The MU initiative was created through the Health Information Technology for Economic and Clinical Health (HITECH) Act. Approximately $30 billion was made available through HITECH to promote the adoption and use of electronic health records (EHRs) certified by the Office of the National Coordinator for Health Information Technology.

 Hospitals have received MU incentive payments through the Medicare and Medicaid EHR Incentive Programs. For Stage 1 of the incentive programs, attestation requirements focused on capturing patients’ clinical data electronically and making health information available to patients electronically

 by requiring hospitals to meet 11 core objectives and 5 additional objectives selected from a menu of 10.

 Hospitals were required to meet performance thresholds for the objectives and report their performance on them for 90 continuous days in order to receive incentive payments. Ongoing financial penalties imposed on hospitals that do not attest to MU requirements have further incentivized participation. 

Hospitals have been responsive to the MU incentive payments and financial penalties, with 98% of eligible hospitals participating in the initiative as of 2016.

 MU participation among hospitals has not been associated with reductions in readmissions occurring within 30 days of discharge.

 However, given that racial minorities have higher readmission rates

 and that reducing disparities is a priority of the MU initiative,

 it is possible that hospitals’ participation in the initiative has had a greater impact on readmissions of racial minorities. The objective of this study was to measure the impact of hospital participation in MU on disparities in 30-day readmissions associated with race.

The likelihood of 30-day readmission for Medicare beneficiaries discharged from hospitals participating in MU was compared with the likelihood of readmission for concurrent beneficiaries discharged from hospitals that were not participating in the initiative. The study period covered 2009 through 2013. Hospitals in our sample initiated MU participation at different times following the implementation of the initiative in 2011. The differential timing of hospitals’ participation in MU was used to estimate its impact on readmissions.

It is possible that hospitals that were early participants in MU differed from later participants and nonparticipants in ways that affected the likelihood of readmission. Two steps were taken to address this issue. First, regression models included hospital fixed effects to control for permanent differences among hospitals. Second, only hospitals that began MU participation by the end of the study period were included in the sample.

The primary data sources for the study were State Inpatient Databases for Florida, New York, and Washington State from the Healthcare Cost and Utilization Program (HCUP) within the Agency for Healthcare Research and Quality. HCUP State Inpatient Databases include all discharges, regardless of payer, from community hospitals in a given state. The American Hospital Association Annual Survey provided information on hospitals’ use of EHRs, and Stage 1 MU payment records from the Medicare and Medicaid EHR Incentive Programs were used to identify participants. The unit of analysis was the hospitalization. Therefore, some patients were observed multiple times over the study period.

The study sample included 2,414,205 unique Medicare beneficiaries 65 years and older, in both FFS and Medicare Advantage, who were discharged to home or postacute care from 291 nonfederal hospitals. The MU incentive programs were implemented in 2011. Therefore, the study period (2009-2013) included at least 2 years of baseline data before each hospital began MU participation.

The outcome was an indicator of all-cause 30-day readmission derived from the HCUP algorithm for hospital revisits.

 Patient identifiers in the State Inpatient Databases only allowed a readmission to be identified if it took place in the same state as the index admission. For Washington, patient identifiers were not consistent from year to year and, therefore, index admissions occurring in December were excluded for this state. Control variables included indicators for age (65-69, 70-74, 75-79, 80-84, 85 years and older), gender, race, quartiles of median household income for the patient’s zip code, Deyo-Charlson Comorbidity Index scores (1, 2, 3, 4, 5 or higher),

 weekend admission, and a continuous measure of length of stay.

 Race categories for white and African American beneficiaries have shown reasonable reliability in administrative claims, but other race and ethnicity categories are less reliable.

 Therefore, race was categorized as white, African American, and other nonwhite race. Additionally, a binary indicator for hospital participation in MU was included in the model. Once a hospital received a Stage 1 payment from either the Medicare or Medicaid EHR incentive program, the measure identified the hospital as a MU participant from that year forward through the rest of the study period. The initial Stage 1 incentive payment was made to a hospital up to 8 weeks after it completed the 90-day attestation period.

 Therefore, each hospital had been meeting MU objectives for up to 5 months when the initial Stage 1 payment was received. The determinants of interest were interaction terms for a hospital’s participation in MU and patient race categories. Estimates associated with the interaction terms indicated whether hospital MU participation differentially impacted the likelihood of readmission for racial minorities.

Linear probability models with hospital and year fixed effects were used for estimation. This approach controls for permanent differences among hospitals by including a dummy variable for each. By including both hospital and time fixed effects, estimates were based on discharges from the same hospital in the same time period. However, the inclusion of fixed effects in the regression model results in wider confidence intervals for the estimated coefficients. 

In a secondary analysis, an indicator for EHR use in the year prior to MU participation was added to the specification. This was done to examine whether any changes in the likelihood of readmission were due to hospitals’ MU participation or simply use of an EHR. The standard errors were adjusted for clustering at the level of the hospital in both models.

 SAS version 9.2 (SAS Institute; Cary, North Carolina) and Stata MP version 14 (StataCorp; College Station, Texas) were used for analyses.

The study was approved by the Human Protections Office of Weill Cornell Medical College.

The mean age of Medicare beneficiaries in our sample was 78.2 years, and 56.6% were female (

). The racial composition of the sample was 77.2% white, 8.4% African American, and 12.9% other nonwhite race; 1.4% of the sample was of unknown race. Slightly more beneficiaries resided in middle-income zip codes than low- or high-income zip codes, with 22% in the lowest quartile, 26% each in the second and third quartiles, and 25% in the highest quartile. For hospital discharges, the mean score on the Deyo-Charlson index was 2.3, mean length of stay was 5.4 days, and 19.9% were weekend admissions. Among the hospitals in our sample, the mean number of licensed beds was 341.2, the majority (71.9%) were not-for-profit, 23.8% were government hospitals, and 39.5% were affiliated with a medical school. In 2011, the first year of the Medicare and Medicaid EHR Incentive Programs, 39.6% of the hospitals in our sample had an EHR but did not participate in MU and 33.5% initiated participation in the initiative. In 2012, the proportion of hospitals with EHRs and no MU participation fell to 7.1% and the proportion of hospitals participating in the initiative grew to 92.4%.

Unadjusted 30-day readmission rates for all Medicare beneficiaries rose in the first 2 years of the study period (

), climbing from 20.6% in 2009 to 21.1% in 2011, when the MU initiative was first implemented. Readmission rates then declined to 19.6% in 2013, the last year of the study period. A similar trend in unadjusted 30-day readmissions was observed for white beneficiaries, although the annual rates were lower in magnitude than rates for all Medicare beneficiaries in each year of the study period. For African American beneficiaries, readmission rates were higher throughout the study period compared with the rates for all beneficiaries. However, after remaining at 24.3% in 2009 and 2010, the readmission rate for African American beneficiaries fell to 23.8% in 2011 and 22.1% in 2013. Unadjusted readmission rates for beneficiaries who were not white or African American followed the same trend as those for all Medicare beneficiaries, but were higher in each year.

) did not indicate reductions in readmissions associated with hospital participation in MU for the overall Medicare population in our sample (percentage points [PP], 0.6; 95% CI, —0.2 to 1.4). The estimate for the interaction between MU participation and African American race (PP, –0.9; 95% CI, –1.5 to –0.4) indicated statistically significant declines in readmissions compared with white beneficiaries, but the interaction of MU participation and other nonwhite race did not indicate a reduction in these events compared with white beneficiaries (PP, –0.4; 95% CI, –0.8 to 0.0). The decline of 0.9 PP represents a 3.7% relative decrease in readmissions for African American beneficiaries using the unadjusted rate prior to MU (24.3%) as the baseline. Similar to our primary analysis, results from our secondary analysis that controlled for prior EHR use did not suggest a relationship between hospitals’ participation in the initiative and readmissions among the broader Medicare population (PP, 1.5; 95% CI, –0.1 to 3.1), but indicated a decline in the likelihood of readmission among African American beneficiaries (PP, –0.9; 95% CI, –1.5 to –0.4). Additionally, a statistically significant decline in readmissions for beneficiaries of other nonwhite race was associated with hospitals’ participation in MU in the secondary analysis (PP, –0.7; 95% CI, –1.1 to –0.3).

Hospital participation in MU was not associated with a reduced likelihood of readmission for the overall Medicare population in our sample, but it was associated with reductions in these events for African American beneficiaries. The results of a secondary analysis that controlled for use of an EHR prior to a hospital’s MU participation were consistent with these findings.

Our study represents an important addition to the existing literature because, despite the nation’s $30-billion investment in health IT through the MU initiative,

 evaluations of provider participation have shown little evidence of improvements in the quality of care.

 It is important to note that previous studies of provider participation in MU, including this one, have only examined Stage 1 of the initiative. Use of more advanced functions, such as interoperability of EHRs, in addition to improving patient outcomes were not emphasized until later stages of the initiative. However, better documentation of patients’ clinical information and improved access to that information through Stage 1 attestation requirements may have affected patients at higher risk of readmission,

 including racial minorities, more than other patients. Additionally, MU participation may have led to reductions in readmissions among broader groups of patients after the study period as hospitals gained more experience with EHRs, including increased use of health information exchange, and learned to use these systems to improve patient outcomes.

Our study has limitations to consider. Due to our reliance on administrative claims, it is possible that unobserved differences among hospitals influenced our results. Our inclusion of hospital fixed effects and limiting the sample to hospitals that initiated participation in MU prior to the end of the study period mitigated the potential for bias, but we cannot rule out this possibility. For example, these approaches would not have controlled for unobserved differences among hospitals or patients that changed over time. Additionally, records for incentive payments were used to identify the timing of MU participation, which likely introduced some measurement error. This would introduce bias toward the null leading to estimates that understated the impact of hospitals’ MU participation on readmissions. Differential 30-day mortality rates may also have influenced estimates of readmission. With the HCUP State Inpatient Databases, it was not possible to identify beneficiaries who died after hospital discharge. Although 30-day mortality is a competing risk, previous studies of Medicare readmissions have not shown mortality rates to be a considerable influence on estimates of hospitalization rates, including readmissions.

 We also included only 3 race categories: white, African American, and other nonwhite race. Combining multiple groups into “other nonwhite race” could have masked differences among these groups and introduced measurement error, biasing the estimate toward no effect. It was also not possible to determine whether a patient’s primary care physician was notified of a hospitalization or if the physician had an interoperable EHR that was capable of receiving the patient’s information electronically from the hospital. Timely follow-up care after hospital discharge and better coordinated care enabled through the electronic exchange of a patient’s health information could reduce the likelihood of readmission. However, findings of a previous study indicated that physician participation in MU did not lower the likelihood of 30-day readmissions for Medicare beneficiaries.

 Lastly, our sample only included hospitalized Medicare beneficiaries from Florida, New York, and Washington. However, these states are located in different regions of the country and have diverse populations.

In addition to the limitations listed above, other policy initiatives that may have been implemented during the study period with the goal of reducing readmissions may have affected our estimates. These include Medicare’s Hospital Readmissions Reduction Program (HRRP) and the Medicare Shared Savings Program (MSSP) for accountable care organizations (ACOs). Hospitals could also potentially have used EHRs adopted through MU participation to improve their performance in these initiatives. The HRRP was implemented in October 2012 and the first MSSP ACOs began in April 2012. If early participants in MU were more likely to participate in these programs, which led to more resources being focused on patients with a higher likelihood of readmission, such as racial minorities, our estimates may overstate the impact of hospitals’ participation in MU.

To our knowledge, this is the first study of the relationship between hospital participation in MU and racial disparities in readmissions. We found a statistically significant relationship between hospitals’ participation in the initiative and reductions in racial disparities associated with 30-day readmissions among Medicare beneficiaries. Additional study is needed to determine if disparities were further reduced as hospitals gained more experience with EHRs and outcomes were emphasized in later stages of MU.&ensp;

The authors are grateful for programming and data management support from Phyllis Johnson, MBA.

 Department of Healthcare Policy and Research, Weill Cornell Medical College (MAU, H-YJ, RK), New York, NY; Department of Health Policy and Management, Richard M. Fairbanks School of Public Health, Indiana University (JRV), Indianapolis, IN; Regenstrief Institute, Inc (JRV), Indianapolis, IN.

Hospital participation in Meaningful Use was associated with reduced disparities in 30-day readmissions for African American Medicare beneficiaries.

Joshua R. Vest, PhD, MPH

Articles

Using Natural Language Processing to Classify Social Work Interventions

Hospital Participation in Meaningful Use and Racial Disparities in Readmissions

Getting From Here to There: Health IT Needs for Population Health

The Health Information Technology Special Issue: Current Trends and Future Directions