Medicare, Medicaid, and commercial plans have all explored ways to improve outcomes for patients with high costs and complex medical and social needs. The purpose of this study was to test the effectiveness of a high-intensity care management program that the Rutgers University Center for State Health Policy (CSHP) implemented as an adaptation of a promising model developed by the Camden Coalition of Healthcare Providers.

We estimated the impact of the program on 6 utilization and spending outcomes for a subgroup of beneficiaries enrolled in Medicare fee-for-service (n = 149) and a matched comparison group (n = 1130).

 We used Medicare claims for all analyses. We used propensity score matching to construct a comparison group of beneficiaries with baseline characteristics similar to those of program participants. We employed regression models to test the relationship between program enrollment and outcomes over a 12-month period while controlling for baseline characteristics.

 A test of joint significance across all outcomes showed that the CSHP program reduced service use and spending in aggregate (

 = .012), although estimates for most of the individual measures were not statistically significant. Participants had 37% fewer unplanned readmissions (

 = .086) than did comparison beneficiaries. Although we did not find statistically significant results for the other 5 outcomes, the CIs for these outcomes spanned substantively large effects.

 Although these findings are mixed, they suggest that adaptations of the Camden model hold promise for reducing short-term service use and spending for Medicare super-utilizers.

An independent evaluation of a high-intensity care management program implemented by provider groups in 4 states shows that the model holds promise for efforts to reduce short-term service use and spending based on an analysis of 149 Medicare patients with complex medical and social needs. These findings may help managed care decision-makers to: 

Reduce spending among super-utilizers through the use of mobile interdisciplinary care teams

Address social determinants of health that contribute to high spending

Adapt community-based care management models to suit local contexts

Team-based care models designed to address the needs of super-utilizers—defined as patients with frequent hospital use and complex health issues&mdash;are proliferating in response to the cost of caring for such patients using traditional approaches.

 Medicare, Medicaid, and commercial insurers have all explored ways to improve quality and cost outcomes for super-utilizers.

 Arguably, the most recognized is the “hotspotting” model pioneered by the Camden Coalition of Healthcare Providers in New Jersey, first brought to national attention by an article in 

 In the Camden model, interdisciplinary mobile care teams provide high-intensity care management and care coordination to individuals with multiple chronic conditions. The model differs from many other care management models in its focus on both medical and social determinants of health among people with high service use.

So far, interest in programs to help super-utilizers has outpaced the available evidence on their effectiveness. Results from an early evaluation of the Camden model were promising: The Camden Coalition reported reductions in emergency department (ED) visits, inpatient admissions, and average charges per month among participants in a pilot program.

 However, the study included only 36 participants and used a pre-post design that did not control for regression to the mean or for confounding factors that might influence outcomes. Regression to the mean is particularly important in interventions in which patients are selected because of their recent service use; some decline in service use post enrollment would be expected without any intervention.

 No independent rigorous evaluation of the Camden model has been published to date, although a randomized controlled trial is ongoing.

 Evaluations of similar care management models have yielded mixed findings over time.

Recognizing the need for evidence, the Center for Medicare & Medicaid Innovation awarded a $14.3-million Health Care Innovation Award to the Rutgers University Center for State Health Policy (CSHP) to support adoption of the Camden model by provider groups in 4 states: (1) an independent physician association in San Diego, California; (2) a nonprofit community health center in Aurora, Colorado; (3) a nonprofit health system with 2 hospitals in Kansas City, Missouri; and (4) a nonprofit operator of 2 federally qualified health centers in Allentown, Pennsylvania. These provider groups targeted super-utilizers living in poor neighborhoods within their service areas.

Researchers affiliated with the Aurora site provided suggestive evidence on the program’s impacts on a group of primarily Medicaid-eligible beneficiaries. They found that it reduced ED visits and hospitalizations and increased primary care use among program participants in Colorado.

 However, the study’s comparison group included patients who declined to participate in the program, potentially biasing results.

In this article, we present evidence from an independent evaluation of the CSHP program across all 4 sites. Our evaluation included prespecified hypotheses that the program would (1) reduce total admissions, particularly readmissions, and (2) reduce Medicare parts A and B spending.

 We used rigorous tests that account for regression to the mean with a well-matched comparison group. This study provides a unique perspective on how the original Camden model can be adapted to a variety of settings and provider groups and adds to the available evidence on care management programs for super-utilizers.

The 4 CSHP sites enrolled 1068 participants between January 2013 and June 2015. Initially, all 4 sites used the same utilization-based criteria as the Camden model—2 or more hospital admissions in the past 6 months&mdash;to identify potential participants. Two sites amended these thresholds early in the award period to expand the pool of potential participants: One site targeted individuals with 2 or more hospital admissions in the prior 6 months or 3 or more admissions in the prior 12 months, and the other targeted individuals with 3 or more hospital events (admissions or ED visits) in the prior 6 months. Among those who met the utilization-based criteria, the sites excluded patients whose conditions, such as cancer or serious behavioral issues, could not be managed with existing program resources.

All 4 sites received technical assistance from the Camden Coalition to guide implementation of the intervention. The sites implemented the same basic set of activities, including development of individualized care plans, integrated care management services through mobile care teams, and education to improve patients’ ability to manage their medical and social needs. Care teams, which included some combination of nurses, community health workers, social workers, medical assistants, and/or behavioral health providers, provided education about the importance of using primary and specialty care instead of, or as a follow-up to, emergency and hospital care. They also addressed participants’ nonmedical needs, including enrollment in social services (such as housing and Social Security disability benefits) and behavioral health service programs.

On average, teams contacted participants 10 times per month, spending roughly 6 hours with each participant per month. The majority of that time (87%) was spent in person, at the participants’ homes and medical appointments, and the remainder was on the phone. Participants remained in the program for an average of 4.2 months, ideally until their health and social circumstances had stabilized and the program could graduate them. By June 2015, the program had graduated 673 participants, although some continued to receive occasional support from the care teams. Additional detail on the intervention is available in a CMS evaluation report.

Although the program’s 1068 participants had public, private, or no insurance, our study population is limited to 149 participants with Medicare fee-for-service (FFS) coverage for whom we could obtain service use and spending data for a 12-month period prior to program enrollment. At the time that we conducted the evaluation, these data were only available for Medicare FFS beneficiaries. Using Medicare claims, we selected 1130 comparison Medicare FFS beneficiaries who met the health and utilization criteria for CSHP program eligibility and were similar to the treatment group beneficiaries on other characteristics. For each potential comparison beneficiary, we created a pseudo-enrollment date to approximate the date that the beneficiary would have enrolled in the intervention if he or she had been in the treatment group. We used propensity scores to match potential comparison beneficiaries to treatment group beneficiaries using geography, baseline (preintervention) measures of service use and spending, chronic conditions, and other demographic and health-related variables (

We used data from the Medicare enrollment database and claims. We defined outcomes quarterly for the 12 months before and after each beneficiary’s enrollment date, so each beneficiary had outcomes in 4 baseline quarters and up to 4 postenrollment quarters in the analysis sample. We estimated impacts on 6 outcomes that the program was designed to address, which we specified prior to the evaluation: all-cause inpatient admissions, inpatient admissions for ambulatory care—sensitive conditions (ACSCs), 30-day unplanned readmissions, outpatient ED visits, Medicare parts A and B spending, and inpatient spending.

We estimated impacts as the regression-adjusted difference between comparison and treatment group beneficiaries in each of the outcomes during the year after enrollment. We used an intent-to-treat design and thus included some patients who dropped out of the program prior to graduation. The regressions adjusted for the same variables used in matching (Table 1

). We used 2-sided tests of significance with a 

 <.05 threshold because we were concerned about both type II and type I errors given the small size of the treatment population. This threshold has been applied in other evaluations of care management programs when policy makers needed to identify promising care models even if additional evidence was necessary before those models could be deemed to have significant impacts at the standard threshold of 

 To examine the aggregate impact of the program on service use and spending, we also conducted a test of joint significance across all outcomes. We used bootstrapping to estimate standard errors in all tests; this method accounted for clustering of outcomes at the individual level.

The characteristics of the treatment group were consistent with CSHP’s target population: individuals with multiple chronic conditions (7.7 on average) who were frequent users of hospital care (Table 1

). Service use and spending were roughly 10 times and 7 times the Medicare FFS average, respectively. The treatment group was also distinct from the average Medicare beneficiary in terms of demographic characteristics and reason for Medicare eligibility. They were younger and more likely to live in high-poverty areas, be entitled to Medicare due to disability or end-stage renal disease, and have dual Medicare and Medicaid coverage.

The observable characteristics of the matched comparison group were very similar to those of the treatment group at the start of the intervention. Groups were well balanced on all variables (Table 1

) and had similar trends in the outcome variables over the 4 quarters leading up to the intervention (

The treatment group had 0.126 (37%) fewer unplanned readmissions on average per quarter than comparison beneficiaries (

 = .086; 90% CI, —0.247 to –0.004]). The point estimates for admissions (–16%), ACSC admissions (–14%), and total spending (–9%) were all large and in the favorable direction, whereas the estimates for ED visits (5%) and inpatient spending (–1%) were close to zero. None of these 5 estimates was individually statistically significant (

). A joint test indicated that, as a group, the estimates for all 6 prespecified outcomes were significantly different from zero (

CSHP’s intervention contributes to a growing body of evidence on the use of care management programs to improve patient outcomes. Our results indicate that the program substantially reduced readmissions (by an estimated 37%) and suggest that it may have reduced all-cause admissions, ACSC admissions, and Medicare parts A and B spending. Many other care management interventions have not achieved this type of success.

 However, the CSHP program shares several distinctive features with the relatively few other models that have succeeded in reducing hospital admissions and/or readmissions, including targeting very high-risk patients and providing frequent in-person contact.

Another distinguishing feature of the CSHP program was its attention to social determinants of health. There is a growing understanding that social determinants affect health and service use, but payers and providers need more evidence on interventions that effectively address these issues, as they are particularly difficult to attend to during medical encounters.

 The results of this study suggest that adaptations of the Camden model are promising for these purposes and are feasible to implement in various settings.

Similar to other evaluations of comprehensive care management programs, our study’s main limitation was low sample size, in part because we were limited to studying Medicare beneficiaries, who accounted for only 26% of all enrollees in the CSHP program. The low sample size reduced the likelihood of finding a statistically significant impact on outcomes and prevented further investigation into whether effects might have differed for certain subpopulations (eg, by implementation site). Findings by Capp et al of significant reductions in ED visits at the Colorado site, which primarily enrolled Medicaid beneficiaries, suggest that this could be the case.

 There may also be some selection criteria for program enrollment that we could not fully replicate in claims data when selecting the comparison group. Finally, we could not compare program savings with costs because we estimated savings for only a fraction of participants and the costs were spread over all participants. Although it is possible that the costs of the program exceed the short-term savings to Medicare, any future cost-benefit analysis should take into account long-term costs and benefits of the program.

This study assessed whether, under CMS’s innovation award, the CSHP community-based care management program could improve the outcomes of super-utilizers across 4 implementation sites. Findings on this adaptation of a well-known super-utilizer program provide insight into the ability to replicate program impacts in different settings. In aggregate, there is sufficient evidence to suggest that the CSHP program reduced service use and parts A and B spending among Medicare beneficiaries. Although these results are promising, the Camden model will need further testing at larger scales before it is possible to make stronger conclusions about its impacts.&ensp;

The authors are grateful for programming and technical support from Mark Ezzo, Andrew McGuirk, Sandi Nelson, and Frank Yoon. They also thank the Center for State Health Policy staff (Margaret Koller, Joel Cantor, Jolene Chou) and staff at all of the implementation sites for giving them detailed program data and valuable perspectives on their implementation experience.

Mathematica Policy Research, Princeton, NJ (PS, CNS, KK, LM), Ann Arbor, MI (KWVB), Washington, DC (GP, LB), Cambridge, MA (BG), and Chicago, IL (KS); Centers for Medicare & Medicaid Services (TD), Baltimore, MD.

 Evaluation funded by CMS, Center for Medicare & Medicaid Innovation, contract number HHSM-500-2010-000261/HHSM-500-T0015. The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of HHS or any of its agencies.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (PS, KWVB, TD, GP, BG, LB, KK, KS, LM); acquisition of data (PS, CNS, KWVB, GP, LM); analysis and interpretation of data (PS, CNS, KWVB, GP, BG, LB, KK, KS, LM); drafting of the manuscript (PS, CNS, KWVB, TD, GP); critical revision of the manuscript for important intellectual content (PS, CNS, KWVB, TD, GP, LB, KK, KS, LM); statistical analysis (PS, GP, KK); provision of patients or study materials (GP); obtaining funding (GP, LM); administrative, technical, or logistic support (TD, KS, LM); and supervision (GP, BG, LM).

 Purvi Sevak, PhD, Mathematica Policy Research, 600 Alexander Park, Princeton, NJ 08540. Email: psevak@mathematica-mpr.com.

1. Vaida B. For super-utilizers, integrated care offers a new path. 

. 2017;36(3):394-397. doi: 10.1377/hlthaff.2017.0112.

2. Mann C. Targeting Medicaid super-utilizers to decrease costs and improve quality. Medicaid website. medicaid.gov/federal-policy-guidance/downloads/cib-07-24-2013.pdf. Published July 24, 2013. Accessed December 13, 2016.

3. Bodenheimer T. Strategies to reduce costs and improve care for high-utilizing Medicaid patients: reflections on pioneering programs. Center for Health Care Strategies, Inc website. chcs.org/media/HighUtilizerReport_102413_Final3.pdf. Published October 2013. Accessed December 13, 2016.

4. Hong CS, Siegel AL, Ferris TG. Caring for high-need, high-cost patients: what makes for a successful care management program? The Commonwealth Fund website. commonwealthfund.org/~/media/files/publications/issue-brief/2014/aug/1764_hong_caring_for_high_need_high_cost_patients_ccm_ib.pdf. Published August 7, 2014. Accessed December 13, 2016.

5. Programs focusing on high-need, high-cost populations. Center for Health Care Strategies, Inc website. chcs.org/resource/programs-focusing-high-need-high-cost-populations. Published April 2016. Accessed December 13, 2016.

Articles

WEB EXCLUSIVE

A community-based care management program for high-risk patients reduced hospital readmissions and also likely reduced admissions and Medicare parts A and B spending.

Effects of a Community-Based Care Management Model for Super-Utilizers

 To develop a predictive model that hospitals or healthcare systems can use to identify patients at high risk of revisiting the emergency department (ED) within 30 days and thus reduce unnecessary ED use through proactive interventions.

 A retrospective analysis of fiscal years (FYs) 2013 and 2014 data from 4 Veterans Affairs hospitals in upstate New York.

 This study developed a predictive model based on administrative data, a publicly available patient classification system, and logistic regression. The study data were from 4 Veterans Affairs hospitals in upstate New York; FY 2013 data were used to predict 30-day revisits in FY 2014. All 22,734 patients with ED visits were included in the analysis. The predictive variables were patient demographics, prior-year utilization, and comorbidities. To prevent overfitting, we validated the model by the split-sample method. The predictive power of the model is measured by C statistics.

 In the first model using only patient demographics, the C statistics were 0.568 (95% CI, 0.555-0.580) and 0.556 (95% CI, 0.543-0.568) for the development and validation samples, respectively. In the second model with prior-year utilization added, the C statistics were 0.748 (95% CI, 0.737-0.759) for both samples. In the final model with comorbidities added, the C statistics reached 0.773 (95% CI, 0.762-0.784) and 0.763 (95% CI, 0.753-0.774) for the development and validation samples, respectively.

 The predictive model we developed in this study is straightforward to implement and offers significantly higher predictive power than other models reported in the literature. Hospitals and healthcare systems can use it to identify high-risk “frequent flyers” for early interventions to reduce ED revisits.

Emergency department (ED) crowding results in treatment delays and excessive mortality.

ED revisits constitute a key factor of ED crowding.

Given limited resources, an effective predictive model is needed to identify high-risk patients for proactive interventions to reduce ED revisits.

The model developed in this study is straightforward to implement by hospitals and payers (eg, health plans, Medicaid, and governmental programs abroad) and offers higher predictive powers compared with other models reported in the literature.

Providing care in the emergency department (ED) is expensive. The average cost of an ED visit is estimated at $1038 compared with $176 for a primary care visit in the United States.

 Moreover, the number of ED visits grew at roughly twice the population growth rate between 2001 and 2008.

 The CDC reported that the number of ED visits increased from 119.2 million (40.5 visits per 100 persons) in 2006 to 136.3 million (44.5 visits per 100 persons) in 2011.

 Furthermore, studies have revealed that “frequent-flyer” patients constitute a key factor of ED crowding, which results in treatment delays and excessive mortality.

 Thus, given limited resources, reducing repeated visits is warranted to improve ED effectiveness and timeliness for those truly in need.

To develop interventions to reduce ED revisits, reliable predictive modeling that can identify high-risk patients is desired. However, compared with inpatient 30-day readmissions, which have been measured by CMS since 2012 to adjust payments to hospitals,

 ED revisits have received less attention.

 Although substantial published literature has examined factors influencing ED revisits, research on predictive models is limited.

 As a result, the predictive models for ED revisits published in the literature have only moderate predictive power: The highest C statistics reached 0.70 for 30-day revisits

In the present study, we developed a statistical model that predicts the risk of revisiting the ED within 30 days of discharge. The model can be used to identify high-risk frequent flyers for proactive intervention. With rapid adoption and use of health information technology and especially electronic health records (EHRs), administrative data are coming increasingly closer to real time and offer greater potential for improving ED care. Our model, based on administrative data and a publicly available patient classification system, can be readily implemented in health systems to reduce unnecessary ED revisits.

ED visit data from fiscal years (FYs) 2013 and 2014 in Veterans Healthcare Network Upstate New York (VISN 2 Upstate) were analyzed in this study. VISN 2 is 1 of the 18 networks through which the US Department of Veterans Affairs (VA) delivers care to more than 6 million patients annually. VISN 2 Upstate, with 5 medical centers and 31 outpatient clinics across upstate New York, serves approximately 140,000 patients with an annual budget of $1 billion (starting in FY 2016, VISN 2 was restructured to include New York downstate VA hospitals). In FY 2014 for VISN 2 Upstate, a total of 21,141 patients had ED visits in the 4 medical centers that provided ED services.

We used the VA National Patient Care Database (NPCD) hosted at the VA Information and Computing Infrastructure as the primary data source for this study. The Outpatient Care File (OPC) and clinical stop code 130 were used to identify index ED visits and revisits. In addition to encounter information, such as visit dates and 

International Classification of Diseases, Ninth Revision, Clinical Modification

) codes, OPC also contains patient demographic and socioeconomic information, such as age, gender, race, and income. Data in NPCD, including OPC, are routinely used in VA operational analysis and research. Most of the data fields, such as visit dates and clinical information like 

 codes, are regularly and rigorously validated with strict business rules. The patient income information is means-tested. One exception is that its race information is often incomplete because the VA health system does not mandate veterans to report race. However, for the last several years, the VA has systematically gathered racial and ethnic information from other data sources, such as Medicare and the Department of Defense; as a result, the updated racial and ethnic data are deemed accurate and reliable.

We also used Managerial Cost Accounting (MCA; formerly Decision Support System) files that contain actual patient care costs, rather than amounts claimed or paid as in private health plans. MCA costs are the primary financial data for internal operations and Congressional inquiries. For case mix or comorbidities, we used a publicly available and widely used system, Clinical Classifications Software (CCS), developed by the Agency for Healthcare Research and Quality (AHRQ),

 which classifies patients into 285 homogeneous groups based on 

The dependent variable in this study was dichotomous (yes = 1, no = 0), indicating whether a patient had any ED revisit(s) within 30 days after being discharged from the ED in FY 2014. The explanatory or predictive variables used in this study were from FY 2013 and can be grouped into 4 categories: (1) demographics (age, sex, marital status, race, disability rating, and period of military service); (2) socioeconomic variables (patient income, homeless [yes = 1, no = 0], and patient insurance status [ie, not covered by any insurance (yes = 1, no = 0), enrolled in Medicare (yes = 1, no = 0), enrolled in Medicaid (yes = 1, no = 0), and covered by private insurance (yes = 1, no = 0)]); (3) prior-year utilization and cost (ED revisit within 30 days [yes = 1, no = 0], number of ED revisits within 30 days, total number of ED visits, number of primary care visits, number of telehealth encounters, total outpatient visits, number of hospitalizations, and total cost); and (4) patient risk or comorbidities (285 clinically homogeneous groups produced by CCS, which is developed by AHRQ

The present study did not need or use any identifiable patient private information and therefore had expedited institutional review board review.

We employed logistic regression to predict the probability or risk of 30-day ED revisits. Logistic regression has been the most extensively used model in predicting binary outcomes where the dependent variable equals 1 if the event happened and otherwise equals 0. The model’s predictive power or discriminative ability is measured by the C statistic, which is defined as the proportion of times that the model correctly discriminates a random pair of individuals with or without an event. It is also equivalent to the area under the receiver operating characteristic curve. A C statistic of 0.5 indicates that the model is no better than flipping a coin, a C statistic of 0.7 to 0.8 suggests that the model has good discriminative ability, and a C statistic of 0.8 or higher indicates great discriminative ability or predictive power.

To prevent model overfit, we only included variables with 

 values smaller than .05 (based on the stepwise procedure) in the final regression analysis, and we also calculated shrinkage coefficient, an indicator of overfit.

 Further, we fitted the model by the split-sample method.

 With this method, the full sample (after merging the dependent variable from FY 2014 and the independent variables from FY 2013) was randomly split into a development sample (50%) and a validation sample (50%).

 The model was fitted on the development sample and then the estimated coefficients were applied to the validation sample to produce the risk score (probability) and the goodness-of-fit statistics. The split-sample method has been extensively used to prevent predictive models from fitting random noises rather than a true trend. In addition, we also visually examined the prediction accuracy by comparing the predicted and observed numbers of patients with any 30-day revisit(s) in 5 estimated risk categories, which equally divided all of the patients into 5 groups based on their estimated risk. The analyses were conducted using PROC LOGISTIC of SAS 9.3 (SAS Institute Inc; Cary, North Carolina).

To demonstrate the predictive power of different explanatory variables, we configured and fitted 3 models, from simple to comprehensive. For model 1, only demographic and socioeconomic variables were included in the model as the independent variables (ie, age, sex, marital status, race, income, enrolled in Medicaid, enrolled in Medicare, or covered by other private insurance [no insurance status was omitted in the regression as reference]). We also used 3 dummy variables (1 was omitted as reference) as the fixed effect to take into account the potentially different practice patterns among the 4 medical centers. Model 2 included the variables in model 1 and prior-year utilizations and cost (ie, ED revisit within 30 days [yes/no], number of ED revisits within 30 days, total number of ED visits, number of primary care visits, number of telehealth encounters, number of all outpatient visits, number of hospitalizations, and total cost). Model 3 included the variables in model 2 and patient comorbidities based on CCS. The inclusion or exclusion of the variables in the final regression depended on the 

 values produced by the stepwise procedure.

All 22,734 patients who had ED visits in FY 2014 were included in this study. Among the 22,734 patients, 4937 returned to the EDs within 30 days; the overall 30-day revisit rate was 22%. The total number of ED visits in FY 2014 was 42,192. The independent variables (except the 285 CCS indicator variables) and their descriptive statistics are reported in 

 shows the top 20 most frequent diagnoses of patients with 30-day ED revisits. All the statistically significant variables (

 <.05) in the final model are reported along with their coefficient estimates, 

. Note that the age groups were kept in the model as a convention regardless of their statistical significance (the group aged 75 years or older was omitted as the baseline).

In predicting 30-day ED revisits, the first model included only demographics, socioeconomic characteristics, and the fixed effect of the medical centers, in which 10 variables were statistically significant (

 <.05) and kept in the model. The C statistics were 0.568 (95% CI, 0.555-0.580) and 0.556 (95% CI, 0.543-0.568) for the development and validation samples, respectively. In the second model, 13 variables were statistically significant and kept in the model. The C statistics were 0.748 (95% CI, 0.737-0.759) for both samples. In the final model, 51 variables (39 CCS categories) that were statistically significant were kept in the model, and the C statistics were 0.773 (95% CI, 0.762-0.784) and 0.763 (95% CI, 0.753-0.774) for the development and validation samples, respectively. The receiver operating characteristic curves of all 3 models based on the validation sample are reported in 

To further examine the prediction accuracy, we compared the predicted and observed numbers of patients with ED revisits by 5 estimated risk categories (% = patients with ED revisits / patients in the risk group). As shown in 

, in both the development and validation samples, the predicted and the observed numbers of revisits were very close except that the model overestimated the number of revisits for the low-risk group. In addition, we also estimated the shrinkage coefficient, which yielded a value of 0.86, indicating no overfit.

EDs across the country are still overcrowded and the waiting lines remain long. The CDC reported that the average waiting time in EDs in the United States increased by 25%, from 46.5 to 58.1 minutes, from 2003 through 2009.

 In 2015, the American College of Emergency Physicians published a survey report showing that ED visits went up since implementation of the Affordable Care Act.

 Consequentially, waiting for hours before seeing a doctor in the ED is not uncommon.

 A systematic review showed that repeated ED visits are a key factor contributing to ED crowding, treatment delays, and excessive mortality.

To free ED capacity for true emergencies, reducing unnecessary repeated visits is desired. However, unlike the rate of hospital readmissions, which has been used by CMS to penalize hospitals with excessive readmissions, ED revisits have not received much attention.

 Given limited resources, interventions to reduce ED revisits require accurate predictive modeling that can identify the patients at high risk for revisits, but only limited research focusing on predictive modeling is found in the literature.

In the present study, we developed a statistical model that predicts, with good accuracy, patients who are at high risk of returning to the ED within 30 days. The strength of the model for broad application lies in its 3 features: (1) It uses administrative data that are readily available in nearly all hospitals or health systems, (2) the comorbidity classification system is publicly available from AHRQ, and (3) the model is based on logistic regression, which can be readily implemented with most common software packages, including Microsoft Excel.

Unlike for predicting hospital readmissions,

 comorbidities added limited predictive power to demographics and prior-year utilization/cost: the C statistic increased by 0.19 from model 1 (only demographics were included) to model 2 (prior-year utilization/cost were added), while only improving by 0.015 from model 2 to model 3 (diagnoses were added). Similarly, our sensitivity analyses revealed that prior-year utilization/cost also did not add much predictive power if the patient demographics and comorbidities were entered into the model first, which indicates that prior-year utilization/cost and comorbidities are highly correlated.

It is worth noting that the stepwise procedure for selecting variables is popular but not perfect. In fact, it can easily produce misleading results if the objective is to assess the effect of an independent variable on the dependent variable. In this study, with prediction as the aim, the forward, backward, stepwise, and Lasso procedures produced nearly identical results and we reported the results based on the stepwise procedure.

Although the objective of this study is to forecast or predict ED revisits, rather than to assess the effect of each individual explanatory variable, some variables are of interest and further research is merited. In particular, the number of primary care visits in the prior year was inversely associated with the risk of ED revisits, which is clinically meaningful and consistent with previous findings.

 On the other hand, it appears to be paradoxical that the number of prior-year hospitalizations was also inversely correlated with ED revisits. Similarly, a few severe comorbidities (eg, CCS131, respiratory failure) were associated with low risk of ED revisits. Without knowing the exact reason(s), we speculate that these variables are indicators of disease severity that warrants inpatient rather than ED care,

 and higher mortality may also play a role. Obviously, further research is needed.

In addition, female patients in this study were less likely to experience ED revisits, holding other things constant, but the finding may not be generalizable to the general public because the study population is predominantly male. It is also interesting to see that a higher disability rating was associated with low risk of ED revisits. Although we do not know the exact reason(s), it could be due to the fact that medical needs for disability can be better anticipated and planned.

This study has several limitations. First, the study population included only veterans, and they were from only 1 region (upstate New York). VA patients could differ from the general population. Apart from seeing mostly male patients, EDs at VA hospitals see more patients with mental health issues (eg, posttraumatic stress disorder) than do EDs in the private sector.

 In addition, VA’s co-payment structure is also different from the private sector’s. No co-payment is required for low-income veterans or those with service-connected conditions, which may result in different ED visit patterns. Second, veterans in this study may have received care from non-VA providers, which may also have decreased the model’s predictive power. Third, the present study only analyzed whether or not a patient had any ED revisit(s) at all; future studies may consider multiple revisits within 30 days. Finally, for the approved study period, 

International Classification of Diseases, Tenth Edition, Clinical Modification

) codes were not available and therefore 

 codes were used to identify patient comorbidities. However, the modeling process remains the same and the predictive power may be improved if 

Curtailing unnecessary ED revisits will not only lower healthcare costs but also shorten wait time for those who critically need ED care. However, broad intervention targeting every ED visitor may not be feasible given limited resources. In the present study, we developed a predictive model based on administrative data, a publicly available patient classification system, and commonly used logistic regression, which yielded good predictive power with a C statistic of 0.76 compared with 0.70 for models previously reported in the literature. The model can be readily implemented by hospitals and healthcare systems with EHRs, and the resultant risk score can be used to identify high-risk patients for proactive interventions to reduce ED revisits.

Stratton VA Medical Center, US Department of Veterans Affairs (KG, GP, LK), Albany, NY.

 This material is based upon work supported in part by the Office of Research and Development, Department of Veterans Affairs.

 Concept and design (KG, GP, LK); acquisition of data (KG, LK); analysis and interpretation of data (KG, LK); drafting of the manuscript (KG); critical revision of the manuscript for important intellectual content (GP, LK); statistical analysis (KG, LK); provision of patients or study materials (GP, LK); administrative, technical, or logistic support (GP, LK); and supervision (GP, LK).

 Laurence Kaminsky, PhD, Stratton VA Medical Center, US Department of Veterans Affairs, 113 Holland Ave, Albany, NY 12208. Email: laurence.kaminsky@va.gov.

Seventy-Two Hour Emergency Department Returns

 [master’s thesis]. Spokane, WA: Washington State University College of Nursing; 2010. research.wsulibs.wsu.edu/xmlui/bitstream/handle/2376/3447/D_White_010961816.pdf. Accessed October 8, 2018.

2. Hao S, Jin B, Shin AY, et al. Risk prediction of emergency department revisit 30 days post discharge: a prospective study [erratum in 

. 2015;10(1):e0117633. doi: 10.1371/journal.pone.0117633]. 

. 2014;9(11):e112944. doi: 10.1371/journal.pone.0112944.

3. Pitts SR, Niska RW, Xu J, Burt CW. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. 

4. National Hospital Ambulatory Medical Care Survey: 2011 emergency department summary tables. CDC website. cdc.gov/nchs/data/ahcd/nhamcs_emergency/2011_ed_web_tables.pdf. Accessed October 8, 2018.

5. Gaieski DF, Agarwal AK, Mikkelsen ME, et al. The impact of ED crowding on early interventions and mortality in patients with severe sepsis. 

. 2017;35(7):953-960. doi: 10.1016/j.ajem.2017.01.061.

6. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. 

. 2008;52(2):126-136. doi: 10.1016/j.annemergmed.2008.03.014.

7. Pizer SD. Should hospital readmissions be reduced through payment penalties? 

. 2013;51(1):20-22. doi: 10.1097/MLR.0b013e318277eb92.

Hospital-Based Emergency Care: At the Breaking Point

. Washington, DC: The National Academies Press; 2006.

9. Hastings SN, Oddone EZ, Fillenbaum G, Sloane RJ, Schmader KE. Frequency and predictors of adverse health outcomes in older Medicare beneficiaries discharged from the emergency department. 

. 2008;46(8):771-777. doi: 10.1097/MLR.0b013e3181791a2d.

“Frequent flyers” significantly contribute to emergency department (ED) crowding. This study developed a predictive model that can be used to identify high-risk patients and reduce ED revisits.

Predicting 30-Day Emergency Department Revisits

 We describe online portal account adoption and feature access among subgroups of patients who traditionally have been disadvantaged or represent those with high healthcare needs.

 Retrospective cohort study of insured primary care patients 18 years and older (N = 20,282) receiving care from an integrated health system.

 Using data from an electronic health record repository, portal adoption was defined by 1 or more online sessions. Feature access (ie, messaging, appointment management, visit/admission summaries, and medical record access and management) was defined by user-initiated “clicks.” Multivariable regression methods were used to identify patient factors associated with portal adoption and feature access among adopters.

 One-third of patients were portal adopters, with African Americans (odds ratio [OR], 0.50; 95% CI, 0.46-0.56), Hispanics (OR, 0.63; 95% CI, 0.47-0.84), those 70 years and older (OR, 0.48; 95% CI, 0.44-0.52), and those preferring a language other than English (OR, 0.43; 95% CI, 0.31-0.59) less likely to be adopters. On the other hand, the likelihood of portal adoption increased with a higher number of comorbidities (OR, 1.04; 95% CI, 1.02-1.07). Among adopters, record access and management features (95.9%) were accessed most commonly. The majority of adopters also accessed appointment management (76.6%) and messaging (59.1%) features. Similar race and age disparities were found in feature access among adopters.

 The diversity of portal features accessed may bode well for the ability of portals to engage some patients, but without purposeful intervention, reliance on portals alone for patient engagement may exacerbate known social disparities—even among those with an activated portal account.

Healthcare organizations are increasingly using online portals to engage patients and enhance access to health information. Our findings illustrate that focusing on improving the diversity of the population that has an activated portal account alone may not be enough to prevent disparities. We found disparities not only in who adopted portal technology but also in which features were accessed by those with an activated portal account.

African Americans, Hispanics, those 70 years and older, and those preferring a non-English language were significantly less likely to be portal adopters.

Compared with white portal adopters, African American portal adopters were less likely to access each of the portal features evaluated.

Portal adopters who were 70 years and older were less likely to access most portal features, including those that were more interactive.

Since 2011, CMS has provided financial incentives to healthcare providers who implement and “meaningfully use” certified health information technology. Incentives to date have focused on providing patients the ability to view, download, and transfer data electronically. Although the incentives do not specify how organizations are to meet these criteria, most primary care practices and other healthcare delivery organizations have chosen to purchase electronic health records (EHRs) and accompanying patient portal software to meet these requirements.

 Neither how these online portals are being adopted nor their potential influence on known health disparities has been fully explored.

In most primary care practices, patient portals enable users to view personal health information, such as laboratory test results, discharge summaries, and notes from recent visits, thus fulfilling the CMS requirement to view, download, and transfer data electronically.

 In some practices, portal users can also schedule appointments and refill prescriptions or send other secure, unstructured messages to providers. Other practices enable users to record personal health information, such as allergies or preventive service use. Despite the array of functionalities that currently exist within commercially available patient portals, we know of just 2 studies that have examined which features adopters access once they have an activated portal account.

 Thus, although studies repeatedly have found traditionally vulnerable populations less likely to be portal adopters as defined by account activation,

 to our knowledge, just 1 prior study has reported findings specific to whether disparities exist in how those adopting portal technology engage with available health-related portal features once they have an activated account.

 It is important to understand whether disparities in health-related portal feature access exist among those who have an account, because portal technology is increasingly being used to target care delivery to those with specific chronic diseases.

We describe how a patient portal was initially used among an insured, sociodemographically diverse population receiving primary care within an integrated health system. This system was selected for its relatively early adoption of patient portal technology and for the comprehensiveness of portal features initially made available to patients. We report the extent to which members of traditionally disadvantaged subgroups of the population (ie, those from minority races, those for whom English is not a first language, and those of Hispanic ethnicity) and those known to have disproportionate health needs (ie, older patients and those with more comorbidities) had activated portal accounts (ie, were portal adopters). Among adopters, we further evaluate which health-related portal features were accessed. Of particular interest was the extent to which adopters accessed features that enable viewing test results and health history information versus accessing the more interactive features that allow adopters to manage and exchange health-related information. In doing so, we provide important information regarding the patient portal foundation upon which subsequent patient engagement capabilities are being built, and we identify subgroups at risk of being left behind as healthcare organizations increasingly rely on portal technology to enable patients to electronically view, download, and transmit their health information to a third party.

The study was conducted in an integrated health system serving metropolitan Detroit, Michigan. The health system’s institutional review board approved all aspects of the study protocol. In 2012, the health system purchased an enterprise license for EpicCare EMR, a commercial medical record software program, and its accompanying patient portal, MyChart.

 As originally implemented, patients seen in any of the health system’s 26 primary care clinics were able to use MyChart via a desktop computer, tablet, or mobile device to securely schedule appointments, receive appointment reminders, view test results, request prescription refills, view and manage information about their health, and send secure messages to care teams.

Portal implementation was a systemwide initiative, with patient- and clinician-targeted campaigns. Screen savers introducing MyChart were launched on examination room computers and other work&shy;stations. Posters were placed in clinic elevators, clinician work areas, examination rooms, and waiting rooms. Informational brochures were available in all clinics. Each component of the campaign instructed patients on how to request a MyChart activation code via the health system’s website or call center. Activation codes were also included on patient discharge summaries. All clinical staff received MyChart training, and front desk staff were instructed to inquire about patients’ account activation status. Work processes were adjusted to prioritize answering patient portal messages over telephone messages.

We identified patients 18 years or older enrolled in the health system’s affiliated health plan with at least 1 visit to their primary care physician between April 1, 2013, and March 31, 2014. Patients younger than 18 years, those without a primary care visit, and those with indeterminate portal activation status (eg, those with account information but no evidence of a session) were excluded.

Patient sociodemographic information, comorbidities, primary care visits, and portal activation information were obtained from the health system’s EHR data repository. These patient-level data were joined with clinic-level information available from health system administrative files.

“Portal adopters” activated their portal account and initiated at least 1 online session between December 1, 2012, and March 31, 2014. Among portal adopters, we examined which health-related portal features were accessed (ie, “clicked on”) between April 1, 2013, and March 31, 2014, in 4 categories: (1) messaging, (2) appointment management, (3) visit/admission summaries, and (4) medical record access and management. Adopters were considered to be users of the messaging feature if they (1) viewed electronic test result letters or other personal communications from their physician, (2) sent a message or viewed a response related to prescription refills, or (3) sent a message or viewed a response related to general medical advice. Those who reviewed their upcoming appointment information or accessed the appointment scheduling feature were considered to be users of the appointment management feature. Adopters who viewed outpatient visit summaries and/or inpatient admission summaries were deemed users of the visit/admission summaries feature. Finally, adopters who accessed the following features were considered to be users of the medical record access and management feature: test results, health summary, current health issues, medications, allergies, preventive care, medical history, immunizations, and/or home monitoring reports. We also constructed a binary variable (“interactive feature use”) that indicated whether the adopter had ever (vs never) clicked on a feature that enabled them to input information (eg, update health issues, update medications, update allergies, send a secure message to their doctor, or request a prescription renewal).

Of particular interest was whether portal adoption and feature access differed by patient race (African American vs white or other), ethnicity (Hispanic ethnicity vs not of Hispanic origin), or language preference (English vs other), or by health needs as defined by patient age or comorbidities. The latter was measured via the Deyo adaption of the Charlson Comorbidity Index (CCI),

a diagnosis-based measure of a patient’s case mix that has been shown to be associated with subsequent mortality and resource use.

We also obtained information on patient gender (female/male), marital status (currently married vs other), and patients’ realized access to office-based primary care. The last was measured by whether the patient had a health maintenance exam visit (ie, “checkup”) and by the total number of primary care office visits within the 12-month study period. In addition to the patient-level variables, we controlled for 3 clinic-level factors that may affect portal adoption and feature use: the number of primary care physicians practicing at the clinic (a proxy for clinic size), whether medical students and/or residents saw patients at the clinic (ie, designated teaching facility), and whether the clinic was located within the city of Detroit (urban) or a surrounding area (suburb).

A multivariable logistic regression model with random effects was fit using the PROC GLIMMIX procedure (SAS software, version 9.4 [SAS Institute Inc; Cary, North Carolina]) to test the role of the patient-level factors of interest in portal adoption status, controlling for other patient- and clinic-level factors. Patients were blocked by physician and nested within clinic, and the Laplace method was used for likelihood approximation. Among portal adopters, we report the percentage of patients accessing each of the 4 categories of portal features (messaging, appointment management, visit/admission summaries, and medical record access and management) as well as by whether or not an interactive feature was accessed. We used multivariable logistic regression to test the role of the same patient-level factors of interest in feature access, again controlling for other patient- and clinic-level factors. Because adopters’ feature access was not influenced by the clinic from which or physician from whom care was received (ie, interclass correlation coefficients <0.01), models were fit using PROC LOGISTIC (SAS version 9.4).

 describes the sociodemographic and other characteristics of the study population (N = 20,282). About one-third of study-eligible patients (32.6%) were portal adopters (n = 6621). The majority (92%) of adopters had at least 2 online sessions, and 86% had 3 or more sessions. Reflecting the relative newness of the portal’s availability, patients who were adopters had been using the portal for a mean (SD) of 8.9 (4.5) months, with 95% using their account for at least 1 month. Across the health system, patients’ portal adoption rates ranged from 6.8% to 64.9% among primary care clinics (interclass correlation, 0.067) with a mean (SD) of 32.7% (12.4%), and from 0% to 100% among primary care physicians (interclass correlation, 0.060) with a mean (SD) of 31.1% (20.2%). Prior to controlling for other factors, African Americans, those of Hispanic ethnicity, those preferring to use a non-English language, and those 70 years and older were substantially less likely to have an activated portal account (Table 1). It should be noted, however, that portal adoption among these vulnerable and relatively high—health-need populations varied substantially across physician panels and primary care clinics. For example, on some physician panels, as few as none (0.0%) of the African American patients had an activated portal account, whereas on others, all (100.0%) African American patients had a portal account. Similarly, at some clinics, as few as 4.5% of the African American patients receiving primary care from the clinic had an activated portal account, whereas at others, the percentage was as high as 52.9%. Similar variability at the physician panel and primary care clinic levels was seen for the other subgroups of interest.

Multivariable logistic regression model results comparing portal adopters with nonadopters are also presented in Table 1. After controlling for other factors, African American patients were half as likely as patients of other races to be portal adopters. Similarly, compared with patients aged 50 to 69 years, those 70 years and older were approximately half as likely to be portal adopters. Those of Hispanic ethnicity and with a preference for a non-English language were also significantly less likely to be portal adopters, whereas portal adoption was more likely among patients with higher CCI scores.

Most adopters accessed portal features related to medical record access and management (95.9%) (

), most often to obtain laboratory test results (91.7%). As depicted in Table 2, adopters also frequently accessed information on their current health issues (87.8%), medications (87.5%), immunizations (86.1%), allergies (86.0%), and health summaries (84.7%). The majority of adopters also accessed the appointment management features (76.6%), more frequently to review information on upcoming appointments (73.1%) than to schedule appointments (58.5%). Relatively fewer patients accessed the messaging features (59.1%), and within this category, access was evenly distributed across letters (35.5%), medical advice requests (31.7%), and medication renewal requests (31.3%). Fewer patients had accessed the visit/admission summary component (41.0%). Overall, 97.0% of portal adopters accessed at least 1 of the health-related features available within the portal, with 68.8% accessing at least 1 interactive feature.

 presents results from the 5 multivariable logistic regression models testing the association of the patient-level factors of interest with feature access. With the exception of accessing visit/admission summaries, patients of African American race were significantly and substantively less likely to have accessed each of the portal features. Similarly, with the exception of record access and management (which virtually all [95.9%] patients accessed), patients 70 years and older were also significantly and substantively less likely to have accessed each of the portal features. On the other hand, as a patient’s CCI score or number of primary care visits increased, so too did their likelihood of accessing the different portal features.

We found that approximately one-third of patients with a primary care visit were portal adopters, as defined by their having an activated portal account. Furthermore, the majority of portal adopters logged into their account multiple times and accessed a variety of features related to both viewing and inputting information. Together, these results illustrate the ability of patient portals to reach a large number of patients, particularly those who are already actively connected with a primary care provider.

Importantly, however, our results also underscore why efforts to enhance and ensure portal adoption as defined by account activation are likely to be insufficient: Our analyses point to disparities not only between those who activate and do not activate a portal account but also in how those with an active account access the features available within portals. Particularly concerning was the finding that African American and older patients were consistently least likely to have an activated portal account, and when they did, they were also least likely to access most portal features. The latter included being least likely to access the appointment management and messaging features—2 functionalities that are arguably critical to care access. The disparities in portal account activation that we observed are consistent with those now repeatedly found by others.

 The findings of disparities in portal adoption, combined with the additional disparities in feature access among portal adopters that we and another study have shown,

 highlight the importance of providing alternative channels for vulnerable and traditionally less engaged patient populations to view and share their health information. Such a need is echoed by findings from a recent qualitative study that highlight the basic computer-related and other barriers that vulnerable populations experience once logged into a portal

 and by a recent systematic review that highlights the role of health literacy in patients’ ability to accurately input and interpret information contained within portals.

We identified wide variability in portal account activation by primary care physician panel and clinic—both overall and among traditionally high-need and vulnerable populations. Understanding the factors that underlie such variability was beyond the scope of the present study, but evidence from elsewhere would point to a diverse array of multilevel factors that may include everything from patients’ health and technological literacy and internet access to the characteristics and supportiveness of physicians and other care team members to the programs and support offered within the broader community.

 Future studies should specifically explore the provider-, clinic-, system-, and community-level factors underlying this variability in portal use, as such studies may hold clues to innovative solutions for engaging and enabling the viewing and sharing of health information among hard-to-reach populations.

 What is clear is that there is likely not a universal patient portal solution for electronically capturing and sharing health information if the goal is the inclusion and engagement of all members of diverse primary care populations. Moving beyond this one-size-fits-all mentality is important in light of mounting evidence that patient adoption of and engagement with EHR-integrated portals can lead to improved outcomes among some patients, including treatment adherence and clinical outcomes among patients with diabetes and HIV.

Our results should be interpreted with a number of study limitations in mind. First, to determine the types of portal features used by patients, we measured user clicks on the links to access specific portal features. Thus, our results are based on the links that patients activated and not time spent per se. Capturing use in this way is consistent with other studies

 but may nonetheless result in overestimates of patient feature use because some clicks may have been due to curious exploration and not actual engaged use. Second, although we were able to control for some clinic-level factors, because of the nature of our study methods and data, we were unable to control for some potentially important patient-level factors, such as education and income. We were also unable to determine the reasons behind patients’ adoption or nonadoption of the portal and specific portal features accessed. Understanding the multilevel and modifiable factors that contribute to observed patterns of portal adoption and feature access is a fruitful avenue for future research. In addition, the context for this research was 1 integrated health system that serves an urban/suburban Midwestern locale. As such, findings may not reflect patterns of portal adoption and feature access that are faced over the long term or by health systems serving other types of populations or based in other locations.

Our results underscore the opportunities and challenges that patient portals present. On the one hand, when integrated within EHRs, portals represent an exciting new frontier for information exchange between healthcare organizations and patients. As such, portal technology may be able to support patients’ health-related decision making and adherence to evidence-based care (and thereby health outcomes) beyond what traditional office visits alone can accomplish. On the other hand, our findings caution that without purposeful planning and integration, reliance on patient portals alone for engaging patients outside of office visits could inadvertently serve to exacerbate existing racial and other social disparities. Our findings therefore highlight not only the need to continually assess the reach and impact of patient portals on diverse populations but also a need to consider alternative channels and mechanisms for sharing health information and engaging patients beyond the walls of traditional office visits.

The views expressed herein are solely those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or the US Government.

 UNC Eshelman School of Pharmacy (JEL) and Lineberger Comprehensive Cancer Center (JEL), University of North Carolina, Chapel Hill, NC; Henry Ford Health System (JEL, DAS, MS), Detroit, MI; Department of Health Behavior and Policy, School of Medicine, Virginia Commonwealth University (CAM), Richmond, VA; School of Social Work, Michigan State University (DAS), East Lansing, MI; Center for the Study of Healthcare Innovation, Implementation & Policy, VA Greater Los Angeles Healthcare System (KD), Los Angeles, CA; Department of Epidemiology and Population Health, UM School of Public Health, University of Michigan (SMR), Ann Arbor, MI.

An early version of this manuscript was presented as an oral paper at the 14th International Conference on Communication in Healthcare, Heidelberg, Germany, September 2016.

 Concept and design (JEL, KD, MS); acquisition of data (JEL, DAS, MS); analysis and interpretation of data (JEL, CAM, DAS, SMR); drafting of the manuscript (JEL, CAM, DAS, KD); critical revision of the manuscript for important intellectual content (JEL, DAS, KD, SMR, MS); statistical analysis (CAM, SMR); obtaining funding (JEL); administrative, technical, or logistic support (JEL); and supervision (JEL, MS).

 Jennifer Elston Lafata, PhD, UNC Eshelman School of Pharmacy, University of North Carolina, 301 Pharmacy Ln, 2214 Kerr Hall, CB #7573, Chapel Hill, NC 27599. Email: jel@mail.unc.edu.

1. Shay R. EHR adoption rates: 20 must-see stats. Practice Fusion website. practicefusion.com/blog/ehr-adoption-rates. Published March 1, 2017. Accessed November 16, 2016.

2. Monegain B. More than 80 percent of docs use EHRs. Healthcare IT News website. healthcareitnews.com/news/more-80-percent-docs-use-ehrs. Published September 18, 2015. Accessed November 16, 2016.

3. What is a patient portal? HealthIT.gov website. healthit.gov/providers-professionals/faqs/what-patient-portal. Updated September 29, 2017. Accessed April 4, 2017.

We found race and age disparities not only in who adopted patient portal technology but also in which features were accessed by those who were adopters.

November 2018

Effects of a Community-Based Care Management Model for Super-Utilizers

Predicting 30-Day Emergency Department Revisits

Patients' Adoption of and Feature Access Within Electronic Patient Portals

Impact of Dementia on Costs of Modifiable Comorbid Conditions

Hospital Cancer Pain Management by Electronic Health Record–Based Automatic Screening

A Narrow View of Choosing Wisely

Understanding and Improving Value Frameworks With Real-World Patient Outcomes

A Randomized, Pragmatic, Pharmacist-Led Intervention Reduced Opioids Following Orthopedic Surgery

Cost of Pharmacotherapy for Opioid Use Disorders Following Inpatient Detoxification

Overdose Risk for Veterans Receiving Opioids From Multiple Sources

From the Editorial Board: Glen D. Stettin, MD