Higher levels of the omega-3 fatty acid eicosapentaenoic acid (EPA) found in the blood appear to explain why icosapent ethyl drives down the risk of a cardiovascular (CV) event.

REDUCE-IT: Serum EPA Levels Key to Vascepa's CV Benefits

We interview experts on investigational treatments such as liso-cel, and ask how to bring them to community practice.

Spotlight on CAR T-Cell Therapy

The American Journal of Accountable Care® features research and commentary on new payment models.

New Members for AJAC Board

Even before the coronavirus disease 2019 (COVID-19) pandemic prompted a sweeping shift toward telemedicine instead of in-person care, technological innovations were not reaching all patient populations evenly, creating a “digital divide” between those with access and those without. Understanding these disparities and ensuring more equitable use of telemedicine has become even more imperative during the pandemic.

On this episode of Managed Care Cast, we’re talking with 2 coauthors of an Original Research article published in our 

Differences in the Use of Telephone and Video Telemedicine Visits During the COVID-19 Pandemic

,” describes patterns of telemedicine visits by patients’ race, ethnicity, age, and language amid the pandemic in Spring 2020.

The authors joining us are Dr Jorge A. Rodriguez and Dr Ishani Ganguli, of Harvard Medical School and Brigham and Women’s Hospital in Boston.

Listen above or through one of these podcast services:

Barriers to Accessing Online Medical Records in the United States

Addressing Evolving Patient Concerns Around Telehealth in the COVID-19 Era

Patient Experience After Modifying Visit Delivery During the COVID-19 Pandemic

Podcasts

The authors of a Clinical research article in the January 2021 Health IT issue of The American Journal of Managed Care® discuss their findings and suggestions for making telehealth access more equitable for all.

Promoting Equitable Access to Telemedicine During the COVID-19 Crisis

 diagnosis may increase the risk of both all-cause and cancer-specific mortality among women with Medicaid coverage, according to 

By linking Medicaid diagnosis claims for preexisting depression or severe mental illness (SMI; schizophrenia, bipolar disorder, other psychotic disorders) with New York State Cancer Registry data on breast cancer diagnoses between 2004 and 2016, the researchers compared survival outcomes with those in patients without a history of mental illness. They noted that Medicaid is the largest payer of mental health–related services in the United States.

“Evidence from epidemiological studies indicates that mental illnesses are associated with elevated cancer risk and poorer cancer outcomes compared with the general population,” the authors stated. “However, differences in survival and cause of death by severity of mental illnesses among women diagnosed with breast cancer remains poorly understood.”

For their retrospective cohort study, included patients (N = 10,444) had to have at least 3 claims for mental illness overall (n = 3203; SMI, n = 1430; depression, n = 1773), with 1 being in the 3 years before their breast cancer diagnosis, as classified by 

International Classification of Diseases, Ninth Revision, Clinical Modification

 codes for depression or SMI. The women were divided into premenopausal (< 50 years) and postmenopausal (≥ 50 years) age groups.

Of note, according to the findings, all-cause mortality was higher than cancer-specific mortality, despite both being elevated among women with comorbid SMI and breast cancer:

All-cause mortality had an HR of 1.36 (95% CI, 1.18-1.57).

Cancer-specific mortality had an HR of 1.27 (95% CI, 1.03-1.44).

The risk for all-cause mortality rose even higher when analyzing outcomes according to race/ethnicity, with non-Hispanic Asians/Pacific Islanders having a 76% higher risk of death compared with non-Hispanic Whites:

Non-Hispanic Asians/Pacific Islanders had an HR of 2.59 (95% CI, 1.15-5.87).

Non-Hispanic Whites had an HR of 1.47 (95% CI, 1.19-1.83; 

In addition, obesity status, postmenopausal age, and documented tobacco use among the women with preexisting SMI and breast cancer increased their risk of death:

Obesity had an HR of 1.58 (95% CI, 1.26-1.98;

Postmenopausal age had an HR of 1.49 (95% CI, 1.25-1.78; 

Documented tobacco use had an HR of 1.42; 95% CI, 1.13-1.78; 

Adjusted 10-year survival, too, showed lowered survival rates among the women with comorbid SMI vs no mental illness, with overall survival being the lowest (78.3% and 76.9%, respectively), followed by cancer-specific (80.7% and 80.0%) survival, and cardiovascular-specific (97.7% and 97.6%) survival being the highest.

And when the investigators stratified outcome by breast cancer characteristics, the women with localized disease (HR, 1.53; 95% CI, 1.19-1.96; 

.01) and hormone receptor–negative tumors (HR, 1.56; 95% CI, 1.22-1.99; 

Not found were associations of increased mortality and preexisting depression for women with diagnosed breast cancer.

Suggested underlying factors contributing to the higher mortality rate in women with comorbid breast cancer and SMI include that schizophrenia tends to worsen and become treatment resistant after menopause; SMI treatment regimens need to change due to interactions with cancer therapies; antipsychotic drugs can have adverse cardiovascular effects for patients considered obese; and smoking cessation rates are lower among persons with SMI. Cancer treatment delays and nonadherence are also more likely among those with mental disorders.

“The present study suggests that Medicaid-insured women with preexisting SMI at breast cancer diagnosis have a greater mortality hazard than women without preexisting mental illnesses,” the authors concluded. “These results highlight the importance of collaboration between oncologists, psychiatrists, and other health care providers to establish a coordinated care plan for breast cancer patients with preexisting SMI.”

How to best integrate breast cancer treatment and SMI care requires further research, they noted.

Lawrence WR, Kuliszewski MG, Hosler AS, et al. Association between preexisting mental illnesses and mortality among Medicaid-insured women diagnosed with breast cancer. 

Published online December 23, 2020. doi:10.1016/j.socscimed.2020.113643

Articles

Severe mental illnesses that precede a breast cancer diagnosis may increase the risk of both all-cause and cancer-specific mortality among women with Medicaid coverage. 

Mental Illness May Predispose Women With Breast Cancer to Higher Mortality

 since 2019 and has a BA in philosophy and journalism & professional writing from The College of New Jersey. 

As Rochelle Walensky, MD, MPH, prepares to assume the role of CDC director on January 20, the former professor of medicine at Harvard Medical School and infectious disease physician at Massachusetts General Hospital and Brigham Women’s Hospital faces a myriad of challenges wrought by the ongoing coronavirus disease 2019 (COVID-19) pandemic.

 since the first case of COVID-19 was reported in the United States, while current data indicate the country has surpassed 

. In comparison, the 1918 flu pandemic took 675,000 American lives, while the US reported a total of 405,000 fatalities during World War II.

Even at the unprecedented speed with which pharmaceutical companies have developed vaccines for COVID-19, rollout has been fragmented at the state level while 

racial disparities in administration rates

In an effort to improve the national rollout of COVID-19 vaccines, Walensky plans to increase the CDC’s communication to combat any 

 in receiving the vaccine, and indicated she wanted to increase media appearances above those made by current director Robert Redfield, MD, who departs with any remaining Trump administration officials Wednesday. She said making sure science-based communication is effectively disseminated to the public in layman’s terms is a top priority.

“Science is now conveyed through Twitter. Science is conveyed on social media, on podcasts, and in many different ways. And I think that's critical,” Walensky said during a livestreamed interview with 

Howard Bauchner, MD, the journal's editor-in-chief. When confronting vaccine hesitancy or anti-vaxxer sentiment on social media, “There's just this massive void and the right information, I think, is not getting out there… I want to make sure that the science is conveyed. We have to say it to one another. We have to say it to the public. And then we have to say it in other forms.”

Internally, Walensky hopes to bolster the voices of scientists already employed by the CDC. Under the Trump presidency, “they have been diminished. I think they've been muzzled,” Walensky said. “This top tier agency—world renowned—hasn't really been appreciated over the last 4 years, and really markedly over the last year. So I have to fix that.”

 in administering the allotment of COVID-19 vaccines they were given, many have reported 

. Part of the Biden administration’s plan to enhance rollout is to expand vaccine allocation to 4 key locations: federally qualified health centers, community vaccination centers (ie, stadiums), mobile units, and pharmacies.

“Part of the challenge with COVID-19 was that we had a frail public health infrastructure to start. It wasn't ready to tackle what it was given,” Walensky said. As director, she hopes to bring this reality to Congress’ attention. “We're in this because we had warnings for many, many other public health scares in the last 20 years and we didn't fix our public health infrastructure and our data infrastructure,” in response to those tests.

In order to meet President-elect Biden’s goal of 100 million vaccinations in 100 days, the constraints currently faced by federal and state governments need to be mitigated. “We have to titrate our supply and our eligibility so that we somehow hit the sweet spot, wherever it is we are, with how much supply we have and how many people are eligible,” Walensky said. 

While the CDC set the initial guidelines for vaccine eligibility and 

 this month, the Trump administration left actual rules and distribution processes to states, resulting in wide variation across the country. Some states adopted stricter standards that led to 

, while loose adherence has led to long lines and 

Expanding the population of those eligible to administer the vaccine can also help alleviate these roadblocks. These individuals can include retirees, the 

, medical military, upper level medical and nursing students, dentists and veterinarians.

Increasing both the number of vaccination sites and vaccinators will also help address the 

 brought to light by the pandemic. “We want to make sure that we can deliver volume, but also volume to the people in places that might be harder to reach.” In a collaborative approach, the federal government will step in at a state-by-state level and offer help based on each state’s unique challenges, Walensky said.

As Rochelle Walensky, MD, MPH, prepares to assume the role of CDC director on January 20, the  infectious disease specialist faces a myriad of challenges wrought by the ongoing coronavirus disease 2019 (COVID-19) pandemic. 

Incoming CDC Director to Prioritize Communication, COVID-19 Vaccine Rollout

Heart Failure Clinical and Economic Burden

The Progressive Nature of Heart Failure

Link Between LVEF, NT-ProBNP and Cardiac Remodeling

Improved Understanding of Heart Failure

Treatment Goals for Patients With Heart Failure

The HFrEF Standard of Care

Heart Failure Management Unmet Needs

Utilization Management Strategies for Appropriate Treatment

Challenges in the Management of HFpEF

Differences in Standard of Care: HFpEF vs HFrEF

Ensuring Timely and Appropriate Access to Therapy

Differences in Economic Burden: HFpEF and HFrEF

PROVE-HF Trial Overview

PARAGON-HF Trial Overview

Recent Findings from the PARALLAX Study

Promising Findings From the TOPCAT Study

Differences in Leading Treatments for Heart Failure

When to Add an SGLT2 Inhibitor to Treatment Regimen

Videos

Key opinion leaders discuss the key differences and unmet needs in disease management for heart failure with preserved ejection fraction and heart failure with reduction ejection fraction. 

Heart Failure With Preserved Ejection Fraction: Current Status and Future Opportunities

Results from a prospective observational study showed aqueous transforming growth factor beta (TGF-β) and autotaxin (ATX) exhibited high diagnostic performance in detecting 

 subtypes and could serve as promising biomarkers for glaucoma. 

Progression of glaucoma depends on the disease’s subtype, the authors explained. Although high intraocular pressure (IOP) is characteristic in some types of glaucoma, it can vary and does not always give an exact diagnosis of subtype. “Therefore, there is an urgent and unmet need for biomarkers that can differentiate glaucoma subtypes and presume disease progression,” they wrote.

Elevations in IOP are thought to be the result of increased aqueous humor (AH) outflow resistance, primarily in the conventional pathway which consists of the trabecular meshwork (TM) and Schlemm’s canal (SC) tissues.

However, “in open-angle glaucoma (OAG), primary open-angle glaucoma (POAG), secondary open-angle glaucoma (SOAG), and exfoliation glaucoma (XFG), an abnormal accumulation of extracellular matrix (ECM) is found along the conventional outflow pathway, leading to the disorganization and degeneration of the TM and SC as well as IOP elevation,” researchers explained.

Previous studies have found that upregulated or downregulated levels of TGF-β and ATX are present in patients with various forms of glaucoma. Based on these findings, the researchers hypothesized that “crosstalk may exist between ATX and TGF-βs, suggesting that concurrent levels of these mediators may be promising diagnostic biomarkers.”

To investigate the correlation between ATX and TGF-βs in different glaucoma subtypes, the investigators collected AH samples from patients with cataracts or glaucoma who were 20 years or older and who underwent surgery for these complications between March 2014 and December 2019. Participants presented at the University of Tokyo Hospital and 3 affiliated eye clinics. Machine learning was used to classify eyes into 3 groups depending on OAG diagnosis: POAG, SOAG, and XFG.

IOP was measured using Goldmann applanation tonometry, and maximum preoperative IOP was evaluated within 3 months prior to AH collection. In addition, the authors wrote, “[l]evels of ATX in the AH were determined via a two-site immunoenzymatic assay with an ATX assay reagent using a Tosoh AIA system. TGF-β levels in the AH were measured using a Bio-Plex Pro TGF-β assay.”

Two hundred and eighty-one eyes from as many patients were included in the study. Eighty-eight eyes were normal (without any ocular complications) while 97 had POAG, 48 had SOAG, and 48 had XFG. Researchers measured areas under the curve (AUC), sensitivity, and specify for 4 glaucoma indicators (ATX, TGF-β1, TGF-β2, and TGF-β3).

ATX, TGF-β1, and TGF-β3 were positively correlated with IOP.

ATX was significantly and negatively correlated with the mean deviation.

From least absolute shrinkage and selection operator regression analysis, the AUC values to distinguish each subgroup (normal, POAG, SOAG, and XFG) ranged between 0.675 (POAG vs normal) and 0.966 (XFG vs normal) when the 4 variables were used.

High AUC values were obtained with ATX for discriminating XFG from normal eyes and with TGF-β3 for discriminating XFG from normal eyes, POAG, or SOAG.

“Collectively, ATX and TGF-βs can be used as biomarkers to differentiate glaucoma subtypes; specifically, TGF-β3 and ATX have the ability to discriminate XFG from other subtypes or normal eyes…and ATX is effective in presuming severity in glaucoma,” the authors wrote. “If we can measure the levels of ATX and TGF-β1–β3 in the AH, it would provide information in addition to clinical findings to differentiate glaucoma subtypes.”

The lack of baseline IOP data, as most subjects had already been administered glaucoma eyedrops during their first visit to an outpatient facility, marks a limitation to the study. AH samples for SOAG included those from patients with elevated IOP due to various causes, meaning future studies could benefit from more discrimination. In addition, further in vitro or in vivo studies ought to be conducted to confirm the mechanisms underlying the changes in aqueous ATX and TGF-βs induced in glaucoma, the researchers concluded.

Igarashi N, Honjo M, Asaoka R, et al. Aqueous autotaxin and TGF-βs are promising diagnostic biomarkers for distinguishing open-angle glaucoma subtypes. 

. Published online January 14, 2021. doi:10.1038/s41598-021-81048-3

Results from a prospective observational study showed aqueous transforming growth factor beta and autotaxin exhibited high diagnostic performance in detecting glaucoma subtypes and could serve as promising biomarkers for glaucoma. 

Study Finds Potential Biomarkers for Distinguishing Glaucoma Subtypes 

 Palliative care has been demonstrated to have positive effects for patients, families, health care providers, and health systems. Early identification of patients who are likely to benefit from palliative care would increase opportunities to provide these services to those most in need. This study predicted all-cause mortality of patients as a surrogate for patients who could benefit from palliative care.

Claims and electronic health record (EHR) data for 59,639 patients from a large integrated health care system were utilized.

A deep learning algorithm—a long short-term memory (LSTM) model—was compared with other machine learning models: deep neural networks, random forest, and logistic regression. We conducted prediction analyses using combined claims data and EHR data, only claims data, and only EHR data, respectively. In each case, the data were randomly split into training (80%), validation (10%), and testing (10%) data sets. The models with different hyperparameters were trained using the training data, and the model with the best performance on the validation data was selected as the final model. The testing data were used to provide an unbiased performance evaluation of the final model.

 In all modeling scenarios, LSTM models outperformed the other 3 models, and using combined claims and EHR data yielded the best performance.

 LSTM models can effectively predict mortality by using a combination of EHR data and administrative claims data. The model could be used as a promising clinical tool to aid clinicians in early identification of appropriate patients for palliative care consultations.

We conclude that the long short-term memory model can be used as a promising clinical tool to aid clinicians in early identification of patients who are appropriate for palliative care consultations.

It can update predicted probability of mortality daily for each patient.

It does not require expert knowledge or hand selection to design features.

The predicted mortality probabilities may assist physicians in identifying patients for palliative care consultations.

Although most seriously ill patients wish to avoid aggressive and burdensome care, many receive invasive procedures and therapies in the final months of their lives with limited benefit.

 Such aggressive care near the end of life is associated with reduced quality of life for patients, excessive end-of-life medical spending, and a high proportion of patients dying in the hospital or other medical facility rather than at home.

Palliative care is a holistic approach to care administered at any point in a serious illness that offers patients support with intensive symptom management and complex medical decision-making, whereas hospice care utilizes a palliative interdisciplinary approach for patients with a prognosis of having less than 6 months to live; these patients typically desire less aggressive care.

 Palliative care is associated with improved patient quality of life,

 Despite multiple studies demonstrating that the majority of patients in the last of year of life experience a high and escalating symptom burden and considerable need for support with advanced care planning,

 providers and health care systems have struggled to proactively identify and support these high-risk patients and their families.

Improving the use of palliative care has been identified as a key driver of success for value-based reimbursement models such as accountable care organizations (ACOs), Medicare Advantage plans, and capitated insurance plans.

 One study of managed care patients provided with home palliative care support found a mean savings of $3908 per member per month in the last 3 months of life compared with propensity-matched controls, with hospice participation increasing from 25% to 70% and median hospice length of stay increasing from 9 to 34 days.

 Currently, the majority of patients who would benefit from palliative care services are either referred late in their disease course or not at all,

 so there has been intense interest in developing and implementing prognostic tools to proactively identify patients near the end of life who may benefit from palliative care services.

Prognostic models are commonly used in clinical practice to identify high-risk populations who may or may not benefit from specific interventions.

 highlighted the limitations of such models, including the use of small data sets for derivation, limitations in number and complexity of variables considered, oversimplified models, and constraints on the population included in the models. To overcome these limitations, the authors applied a deep learning model on electronic health records (EHRs) for a large number of patients.

 However, a remaining limitation of the study was that it employed only EHR data, which may be incomplete for patients who receive care in multiple medical systems, as most patients do.

In this paper, we integrated administrative claims and EHR data into our models. Claims data provide a richer picture for each patient, although the information can lag by 3 to 6 months. We hypothesized that the integration of claims and EHR data would provide more complete and accurate data than either source alone. We searched the PubMed and Google Scholar databases for literature prior to August 2020. To our best knowledge, no studies have compared EHR and claims data for predicting mortality for patients in the United States.

We implemented a deep learning method to predict mortality of inpatients and outpatients as a proxy for identifying patients to be considered for palliative care. Our long-term goal is to provide a clinical referral system for daily use to assist physicians in identifying patients who could benefit from palliative care.

We hypothesized that combining claims data with EHR data could achieve better performance for mortality prediction than using EHR or claims data alone. To test this hypothesis, we applied and compared mortality prediction performance across 4 types of machine learning and deep learning models—long short-term memory (LSTM), deep neural networks (DNN), random forest (RF), and logistic regression (LR)—using EHR data, claims data, and a combination of EHR and claims data.

In this study, we used EHR data and administrative claims data from the Medicare and Medicare Advantage ACOs of a large integrated health care system. The system operates in 2 states and uses EpicCare EHR software. We included all 59,639 patients with encounter records from January 2017 to February 2020.

A patient was considered a positive case if the patient had a recorded date of death. All other patients were then considered negative cases. We first included 59,639 total unique patients based on the patient having at least 1 EHR record or claim record (“claims + EHR”). Among these 59,639 patients, when we extracted only the claims data of patients, there were 56,598 unique patients (“claims only”). When we extracted only the EHR data of patients, there were 55,316 unique patients (“EHR only”).

For the above 3 cases, we followed a similar process of feature extraction, data splits, model training, and evaluation as described below.

For each patient, we extracted all available encounter records. The features included diagnosis codes, procedure codes, medication codes, problem lists, demographics, and social history. For the social history, we combined the questions and answers as a column of “codes.” For example, if a patient had a question of “drug use” and the corresponding answer is “no,” then we combined the question and answer as “DrugUseNo.” There were 84,315 unique codes included in the analyses.

To better capture the pattern and nature of our longitudinal data, all codes were sorted in a time-increasing order and then mapped to 32-dimensional vector space by the technique of Word2Vec.

 The Python Genism Word2Vec model was used with the following hyperparameters: size (embedding dimension) was 32, window (the maximum distance between a target word and all words around it) was 5, min_count (the minimum number of words counted when training the model) was 1, and sg (the training algorithm) was CBOW, or the continuous bag of words. Each feature was then represented by a 32-dimensional numerical vector and also associated with a time point (age at event), which was calculated by the difference in years between the corresponding visit date and the birth date of the patient. We also included race as a feature in our study. Ultimately, each individual patient had their own numerical vector to represent their codes.

The number of events for different patients may vary. We selected 500 events for all patients; this empirical choice was based on proximity to the mean number (mean = 466 for case of claims + EHR) for all patients. These 500 events were sorted in a time order. If a patient had more than 500 events, only their most recent 500 were included. If fewer than 500 were available, zero events padding to 500 was utilized.

 was the number of samples/patients, 500 was the number of codes included, and 34 was the dimensionality of features (32-dimensional vector by Word2Vec, plus age and race). The input of DNN, RF, and LR had a dimension of (n, 500 × 34).

Data Splits for Training, Validation, and Testing

We included 59,639 patients for claims + EHR, 56,598 patients for claims only, and 55,316 patients for EHR only analyses.

The selected patients were randomly split into training (80%), validation (10%), and testing (10%) data sets. The number of patients in each data set is shown in 

The models were trained on the training data set and then validated on the validation data set. The model with the best performance on the validation data set was selected as the final model to be tested in the testing data set.

The Training of Machine Learning and Deep Learning Models

The embedded vectors were the inputs of machine learning and deep learning models. Our model is an LSTM neural network, which is composed of an input layer, 1 hidden layer (128 dimensions), and a scalar output layer. A binary cross-entropy loss function was employed as the output layer and a Sigmoid function was used as the activation function for the hidden layer. The Adam optimizer

 was used to optimize the model with a minibatch size of 256 samples. We did an extensive hyperparameter search for activation functions (Sigmoid, tanh, SeLU, and ReLU), as well as the embedding dimensions (32, 64). We did not extensively search other hyperparameters, such as number of LSTM layers, number of recurrent units, and batch size, as these hyperparameters were of minor importance.

We also compared the LSTM with other models: DNN,

 The DNN model was composed of an input layer, 4 hidden layers (with 256, 128, 64, and 32 dimensions, respectively), and a scalar output layer. We used the Sigmoid function

 at each hidden layer. Binary cross-entropy was used as loss function, and the Adam optimizer was used to optimize the models with a minibatch size of 256 samples. The RF and LR models were configured by the default options in the Scikit-learn package in Python 3.

We implemented a variety of metrics to evaluate the model performance in order to identify the best models. The metrics included area under the receiver operator characteristic (ROC) curves (AUCs), precision recall curves, overall accuracy, sensitivity, specificity, precision, and F1-score.

We also tested the model under different discrimination thresholds or cutoffs. The values of cutoffs used were 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 0.99.

Separate Model Testing for Inpatients and Outpatients

In the study design, we included all inpatients and outpatients. Inpatients were defined as patients who had at least 1 hospital admission. Patients without hospital admissions were defined as outpatients. We investigated inpatients and outpatients separately in the testing data set by our trained LSTM models for claims + EHR analyses.

In the testing data set, 3143 patients had an admission and 2821 patients had only outpatient visits recorded. Among these, 509 inpatients and 107 outpatients were deceased and 2634 inpatients and 2714 outpatients were alive at the end of follow-up.

is the flowchart of the overall study. Analyses were conducted in 2020 using the libraries of Scikit-learn, Keras, Scipy, and Matplotlib with Python version 3.6.5.

The study found 4 main results. In this section, we list an overview of the main results. First, we showed the characteristics of the overall study population (

) and observed that the majority of patients were female and White, with a mean age of 74 years. Second, we evaluated model performance by ROC curves, precision recall curves, and other performance metrics (

). In all modeling scenarios, LSTM models outperformed the other 3 models, and using claims + EHR data yielded the best performance. Third, we showed the distribution of predicted probabilities for the case of claims + EHR by LSTM model to intuitively understand prediction performance results and potential thresholds for a desired accuracy (

). The distribution of predicted probabilities of LSTM on patients in the testing data set showed a high true-negative rate (specificity) and a low false-positive rate. Finally, we showed the LSTM model performance on inpatients and outpatients to understand the difference in performance among different populations (

). The trained LSTM models performed better on the inpatient sample compared with the outpatient sample (AUC, 0.97 vs 0.92; average precision [AP], 0.92 vs 0.72).

Characteristics of the Overall Study Population

The Table details the characteristics of our overall study population. Approximately 10% of patients died over the course of follow-up. The mean age of patients who died was 78 years, and the mean age was 73 years for those who were alive at the end of follow-up. The majority of the patients were female and White. Approximately 76% patients had a diagnosis of essential (primary) hypertension, and deceased patients had higher proportion (84%) of this diagnosis compared with the patients who were alive (75%).

Model Performance by AUC and Precision Recall Curves

The LSTM model outperformed the other 3 models in terms of AUC values and precision recall curves. All the models achieved the best performance by using a combination of EHR and claims data compared with using each data type alone.

Figure 2 shows the model performance by AUC and precision recall curves. The AUC for the LSTM model was 0.97 (0.95 for DNN, 0.92 for RF, and 0.88 for LR) for claims + EHR, 0.96 (0.93, 0.90, and 0.88, respectively) for claims only, and 0.94 (0.90, 0.84, and 0.77) for EHR only.

Meanwhile, the LSTM model achieved an AP score of 0.89 (0.82 for DNN, 0.71 for RF, and 0.60 for LR) for claims + EHR, 0.87 (0.76, 0.66, and 0.60, respectively) for claims only, and 0.77 (0.64, 0.47, and 0.37) for EHR only.

Both the AUC values and precision recall curves indicated that LSTM outperformed the other 3 models. The curves also demonstrated that LSTM models had a strong early recall behavior.

LSTM models outperformed the other 3 models using additional performance metrics, and using claims + EHR data yielded the best performance.

eAppendix Table 2 lists other metrics we used to evaluate model performance. The metrics include overall accuracy, sensitivity, specificity, precision, and F1-score.

In all cases, LSTM demonstrated the best performance. For claims + EHR, it achieved an accuracy of 0.97 (vs 0.96 for claims only and 0.95 for EHR only), a sensitivity of 0.77 (vs 0.72 and 0.60, respectively), a specificity of 0.99 (vs 0.99 and 0.98), a precision of 0.91 (vs 0.91 and 0.79), and an F1-score of 0.83 (vs 0.81 and 0.68).

The LSTM model had high sensitivity and specificity. The eAppendix Figure shows the distribution of predicted probabilities for claims + EHR by the LSTM model on the testing data set. A majority of deceased patients were predicted by the model to have high probabilities of mortality, and a small proportion of them had lower predicted probabilities of mortality, which means the model had a high true-positive rate (sensitivity) and a low false-negative rate. Of the patients who were alive at the end of follow-up, only a few were predicted to have a high probability of mortality and most of them had very low predicted probabilities of mortality, which means the model had a high true-negative rate (specificity) and a low false-positive rate.

Figure 3 shows the model performance of LSTM under different cutoffs of predicted probabilities. When using lower cutoffs (eg, 0.1), the model had a higher sensitivity (88%) with lower precision (58%) and F1-score (70%). When using higher cutoffs, the sensitivity of the model decreased and specificity increased. The F1-score first increased and then decreased as the cutoffs became higher.

Performance on Separate Inpatient and Outpatient Cohorts by the Trained LSTM Model

The LSTM models achieved better performance on inpatient compared with outpatient samples.We also tested the trained LSTM model on the inpatients and outpatients separately. The performance results are summarized in eAppendix Table 3 and Figure 4.

The trained LSTM models performed better on inpatients compared with outpatients. The AUC value was 0.97 for inpatients (0.92 for outpatients), and the AP score was 0.92 for inpatients (0.72 for outpatients). The other metrics followed a similar trend.

In this study, we utilized EHR data and administrative claims data from a large health system to predict mortality of patients to better identify patients who might benefit from palliative care. We investigated 3 data sets, using both EHR data and claims data, only claims data, and only EHR data. We compared LSTM models with DNN, RF, and LR models.

Our results indicated that deep learning LSTM models can most effectively predict the mortality of patients compared with other models using all data sets. One possible reason for observing better performance of the LSTM model was that it can learn from experience based on the time-series data when the time steps are of arbitrary size.

 LSTM models can also handle vanishing gradient problems by retaining information (via memory unit) for an arbitrary time period.

 The memory unit can determine which information needs to be retained, forgotten, and output to the next computational unit.

In addition, our framework for using a LSTM model for mortality prediction has additional advantages, as it (1) is suitable for multiple data format sources, (2) does not require expert knowledge or hand selection to design features, (3) can dynamically update predicted mortality probabilities based on newly added features of patients, and (4) is a more general framework that can be used in both outpatient and inpatient systems. Understanding the relative performance of these models for identifying high-risk patients may help health care systems and managed care organizations select the optimal model for their organization.

Our results also indicated that models achieved the best performance by using both EHR and claims data, the next best by using claims data only, and the worst by using only EHR data. This was not surprising, as the combined EHR data and claims data provided more information for the models to discover and identify more potential patterns, which may contribute to the accurate predictions. One possible reason for only claims data performing better than only EHR data is that claims data usually contain more complete information for patients than EHR data; most patients receive care in multiple medical systems, which can result in incomplete EHR data. A study by Zeltzer et al of 118,510 Israeli patients found that over longer time durations, the relative benefit of claims data appeared to increase compared with EHR data, which were associated with greater prognostic accuracy in short-term outcomes.

There is a consensus that effective use of palliative care is critical to the success of value-based care organizations by improving quality and reducing costs for patients near the end of life, where spending is concentrated. One of the key barriers to widespread adoption of palliative care to date has been correctly identifying which patients to target. Smith et al report that “[s]ystematic identification of patients for whom specialist and/or primary palliative care interventions are likely to improve quality of life and reduce cost is essential in setting up a successful population management program. Unfortunately, widely accepted criteria for identifying patients who would most benefit from palliative care have not been developed.”

 Current approaches to identifying patients for palliative care often focus on individual disease states (eg, congestive heart failure, chronic obstructive pulmonary disease) or historical utilization (eg, hospitalizations in the past 12 months), which do not identify all patients at risk and often identify patients too late in the disease course. Typically, these programs focus on either inpatients or outpatients but do not include both.

The advances in this article demonstrate a more accurate, holistic approach to identifying patients at high risk of mortality using all available sources of patient information (claims and EHR) for both inpatients and outpatients. If this approach is validated in additional populations, it can be used to test the impact of palliative care for patients in different phases of care on quality, spending, and utilization. This would allow value-based care organizations to more accurately measure the added value of palliative care for various populations of patients.

To better understand the differing performance for inpatients and outpatients, we investigated the number of records (source of knowledge) for each group. The inpatients had more information, with a mean number of 848 records, compared with the outpatients, with a mean number of 136 records. Thus, we hypothesize that the larger amount of information provided by inpatients to the LSTM model enabled better prediction of mortality compared with outpatients. Moreover, we tested the LSTM model for the case of claims + EHR in 4 patient groups with different numbers of records: fewer than 150 (n = 1680 patients), between 150 and 300 (n = 1309 patients), between 301 and 500 (n = 1104 patients), and more than 500 (n = 1871 patients). We observed that the number of records had an impact on model prediction results. Specifically, prediction results became less accurate for patients with fewer records. Future studies will also investigate the model performance on other patient groups, including different disease groups.

Our study has several limitations. First, our models were trained based on data from 1 site, and additional research is needed to determine their scalability to other health care settings. Second, we included only a subset of the population (Medicare/Medicare Advantage) who is older and has more comorbidities than the general population, so prediction for other age groups with fewer comorbidities may not be as accurate.

 Unknown model “fairness” with regard to protected attributes might be caused by machine/deep learning models.

 Third, we predicted the overall status of death rather than the timing of death from a clinical event, although more than 96% of deaths occurred over a 1-year follow-up. Future studies will use similar methods to predict time to death, in-hospital mortality, and 30-day mortality at admission, which may help clinicians decide on the timing of palliative care interventions. These models used comprehensive data sets that were assembled after the last follow-up for these patients. Using these models prospectively may be challenged by our ability to assemble, analyze, and interpret them in real time. Future studies will focus on the implementation of this proof-of-concept study.

We demonstrate that deep learning LSTM models can effectively predict mortality by using a combination of EHR data and administrative claims data. The promising predictions of all-cause mortality could assist physicians in suggesting and providing palliative care consultations to appropriate patients.

The authors would like thank Karen Shakiba, Jacob Heidbrink, Ly Mettlach, and Ryan Soluade at BJC Medical Group for their research assistance.


 Institute for Informatics (AG, RF) and Department of Internal Medicine (RF, PW), Washington University School of Medicine, St Louis, MO; University of Pennsylvania Health System (CC), Philadelphia, PA; Division of Pulmonary, Allergy, and Critical Care, and Palliative and Advanced Illness Research Center, Perelman School of Medicine at the University of Pennsylvania (KC), Philadelphia, PA; BJC Medical Group and BJC Accountable Care Organization (NM), St Louis, MO.

 Work included in this document was produced by the research team. This work was produced with the support of the Big Ideas Program, a BJC HealthCare and Washington University internal grant program, hosted by the Healthcare Innovation Lab and the Institute for Informatics.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (AG, RF, PW, CC, KC, NM); acquisition of data (NM); analysis and interpretation of data (AG, RF, PW, CC, KC, NM); drafting of the manuscript (AG, PW, NM); critical revision of the manuscript for important intellectual content (RF, PW, CC, KC, NM); statistical analysis (AG); provision of patients or study materials (NM); obtaining funding (PW, NM); administrative, technical, or logistic support (CC, NM); and supervision (RF, PW, CC).

 Aixia Guo, PhD, Washington University School of Medicine, 600 S Taylor Ave, Ste 102, St Louis, MO 63110. Email: aixia.guo@wustl.edu.

Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life

2. Heyland DK, Dodek P, Rocker G, et al; Canadian Researchers End-of-Life Network (CARENET). What matters most in end-of-life care: perceptions of seriously ill patients and their family members. 

Deep learning algorithms could improve palliative care by predicting mortality from electronic health records and claims data.