Currently Viewing:
The American Journal of Managed Care January 2020
Using Applied Machine Learning to Predict Healthcare Utilization Based on Socioeconomic Determinants of Care
Soy Chen, MS; Danielle Bergman, BSN, RN; Kelly Miller, DNP, MPH, APRN, FNP-BC; Allison Kavanagh, MS; John Frownfelter, MD, MSIS; and John Showalter, MD
Eliminating Barriers to Virtual Care: Implementing Portable Medical Licensure
Pooja Chandrashekar, AB; and Sachin H. Jain, MD, MBA
Trust in Provider Care Teams and Health Information Technology–Mediated Communication
Minakshi Raj, MPH; Jodyn E. Platt, PhD, MPH; and Adam S. Wilk, PhD
The Health IT Special Issue: Enduring Barriers to Adoption and Innovative Predictive Methods
Ilana Graetz, PhD
What Accounts for the High Cost of Care? It’s the People: A Q&A With Eric Topol, MD
Interview by Allison Inserro
Does Machine Learning Improve Prediction of VA Primary Care Reliance?
Edwin S. Wong, PhD; Linnaea Schuttner, MD, MS; and Ashok Reddy, MD, MSc
Health Information Technology for Ambulatory Care in Health Systems
Yunfeng Shi, PhD; Alejandro Amill-Rosario, MPH; Robert S. Rudin, PhD; Shira H. Fischer, MD, PhD; Paul Shekelle, MD; Dennis Scanlon, PhD; and Cheryl L. Damberg, PhD
The Challenges of Consumerism for Primary Care Physicians
Timothy Hoff, PhD
Advancing the Learning Health System by Incorporating Social Determinants
Deepak Palakshappa, MD, MSHP; David P. Miller Jr, MD, MS; and Gary E. Rosenthal, MD
Predicting Hospitalizations From Electronic Health Record Data
Kyle Morawski, MD, MPH; Yoni Dvorkis, MPH; and Craig B. Monsen, MD, MS
Opt-In Consent Policies: Potential Barriers to Hospital Health Information Exchange
Nate C. Apathy, BS; and A. Jay Holmgren, MHI
Currently Reading
e-Consult Implementation Success: Lessons From 5 County-Based Delivery Systems
Margae Knox, MPH; Elizabeth J. Murphy, MD, DPhil; Timi Leslie, BS; Rachel Wick, MPH; and Delphine S. Tuot, MDCM, MAS

e-Consult Implementation Success: Lessons From 5 County-Based Delivery Systems

Margae Knox, MPH; Elizabeth J. Murphy, MD, DPhil; Timi Leslie, BS; Rachel Wick, MPH; and Delphine S. Tuot, MDCM, MAS
This case study of 5 county-based delivery systems finds that existing specialty care relationships and information technology integration are important differentiating factors for e-consult implementation success.

Objectives: Electronic consultation, or e-consult, systems improve specialty care access by conveying specialist expertise to primary care clinicians (PCCs) without requiring specialist visits. Our study evaluates organizational factors for e-consult implementation across 5 publicly financed, county-based health systems in California. Each system serves 40,000 to 180,000 culturally and linguistically diverse patients across 4 to 19 primary care locations.

Study Design: We interviewed leaders whose systems received grant funding between 2015 and 2017 to plan and implement e-consult. Interviews discussed platform selection, electronic health record (EHR) compatibility, PCC and specialist opinions, and project governance. We also collected implementing systems’ platform operations metrics.

Methods: Mixed methods, including semistructured interviews and quantitative platform metrics. Interviews were analyzed in alignment with the Consolidated Framework for Implementation Research inner setting domain.

Results: Three of the 5 systems successfully implemented e-consults. System 1 sustained implementation across 27 specialties, system 2 achieved fragmented implementation, and system 3 reported early-stage implementation. Existing PCC-specialist relationships emerged as the strongest facilitator. E-consult–EHR technology integration was also important, although an add-on platform enabled e-consult expansion in system 2. Although all systems faced challenges, such as project management resourcing, systems 4 and 5 abandoned implementation amid compound climate and readiness barriers.

Conclusions: Successful e-consult implementations in public delivery systems leveraged (1) prior primary care and specialty care clinician relationships and (2) integrated EHR and e-consult platforms. This contrasts with common expectations that new technology will overcome care delivery gaps. Findings add to existing e-consult implementation literature that emphasizes reimbursement and leadership champions.

Am J Manag Care. 2020;26(1):e21-e27.
Takeaway Points
  • Electronic consultation, or e-consult, improves specialty care access and equips primary care clinicians to manage more complex cases by bridging specialty expertise and primary care delivery. Despite immense benefits, systems have varying success in implementing e-consult technology.
  • This case study of 5 county-based health systems finds that amid similar patient populations and resource constraints, systems with prior specialist relationships and more closely integrated information technology (IT) infrastructure established more robust e-consult implementation.
  • Other managed care systems can benefit from this takeaway by focusing on specialty relationship building and IT integration as prerequisites for e-consult system development.
Electronic consultation, or e-consult, systems link primary care clinicians (PCCs) with specialist expertise, supporting PCCs’ ability to manage complex conditions. PCCs using e-consults report both high satisfaction and high value for themselves and their patients.1,2 E-consults provide specialist guidance that can either eliminate the need for in-person specialist appointments or ensure that in-person appointments are more useful by identifying diagnostics or tests that patients should complete in advance. One e-consult platform found that approximately 1 in 4 e-consults in a national sample either avoided an unnecessary referral or avoided referral to the wrong specialty.3 Health systems similarly report reduced specialist visit wait times after implementing e-consult as a result of fewer unnecessary or inappropriate referrals.4,5 These outcomes are particularly salient for patients in public delivery systems, who disproportionately experience fragmented care and long wait times.

Past research identifies that specialty care access and appointment wait times commonly drive e-consult implementation. Facilitators include executive and clinician leadership. Common barriers include specialist reimbursement, technology funding, and administrative support.6,7 However, no studies have examined whether these factors similarly influence e-consult implementation across publicly financed, county-based health systems.



We explored e-consult implementation factors across 5 California county-based public health delivery systems. Each system serves 40,000 to 180,000 patients who are culturally and linguistically diverse. Each system provides primary and specialty care to predominantly publicly insured or uninsured patients at 4 to 19 primary care locations per system. Between 2015 and 2017, the Blue Shield of California Foundation solicited proposals from public hospital systems through competitive and by-invitation processes to advance e-consult adoption in the California safety net. Grant amounts and terms were designed to match stages of planning and implementation, from short-term feasibility grants ($50,000-$100,000) to multiyear implementation or spread grants ($250,000-$300,000). Foundation funding also supported access to technical assistance with e-consult implementation experts (eg, consultative phone calls, meetings, collaborative webinars). Funding recipients were expected to share operational metrics and provide leadership interviews.

Study Design

We used a mixed-method case study approach with qualitative and quantitative data.8 We conducted semistructured interviews with executive leaders approximately 1 year after systems began implementation. Interviewees included chief medical informatics officers (systems 1, 3, and 4), a chief medical officer (system 4), medical directors of ambulatory care (systems 1, 2, 3, and 5), a medical director of community health partnerships (system 1), and project managers (system 2). In contrast with other studies that focus on end users,2, 9,10 we focused on executive leaders with knowledge of the implementation trajectory from early-stage planning to launch and expansion. Interviews lasted 30 to 60 minutes and were audio recorded and comprehensively summarized by the study team. All interviewees provided informed consent. Interviewees did not receive individual compensation but represented health systems that received implementation grant support. We also analyzed standard operational metrics to differentiate those systems that successfully implemented e-consult from those that did not. The University of California, San Francisco Institutional Review Board reviewed and approved this project (IRB #14-15193).


A standardized interview guide (eAppendix [available at]) explored e-consult platform selection criteria, electronic health record (EHR) compatibility, primary and specialty care clinician leader opinions, and project leadership and management. Two coauthors (M.K., D.S.T.) conducted thematic analysis using a preliminary set of analytic codes. Initial codes encompassed 6 essential implementation factors from e-consult implementation literature: executive leadership, clinical champions, efficient workflows, funding, EHR integration, and specialty access.6 Coding was refined during analysis based on interviewee responses. For example, interviewees rarely discussed “efficient workflows,” but a “resources to support operations” code was added because of repeated mention.

Project codes were subsequently mapped with the Consolidated Framework for Implementation Research (CFIR), a framework that unifies multiple implementation theories to guide evaluation across different studies and settings.11 CFIR authors recommend that “CFIR not be applied wholesale to every problem” but rather “those concepts that will be most fruitful to study.”11 Thus, we concentrated on the inner setting domain to identify features unique to each system that may have influenced implementation. In contrast, we expected that other domains, such as intervention characteristic and outer setting, would yield less variation because we were evaluating the same intervention (e-consult) across similar organizational settings (county-based delivery systems in California).

Quantitative, descriptive operations data for the period January to June 2018 were obtained from the 3 systems that successfully implemented e-consult programs. Data included number of specialties offering e-consults and volume of e-consults completed (to identify breadth and depth of the service), average specialist e-consult response time (a measure of specialty care access), and percentage of e-consult requests that were virtually comanaged versus converted to an in-person specialist visit (a measure of e-consult appropriateness and effectiveness to maintain patients in their primary care homes). These have been previously identified as important core measures of e-consult implementation to achieve cost-effective, high-quality specialty care delivery while maintaining high care team and patient satisfaction.12 We triangulated the operations data and leadership interviews to determine e-consult implementation success: “no, not implemented”; “no, limited pilot”; “yes, early implementation”; “yes, fragmented”; and “yes, sustained.”

Copyright AJMC 2006-2020 Clinical Care Targeted Communications Group, LLC. All Rights Reserved.
Welcome the the new and improved, the premier managed market network. Tell us about yourself so that we can serve you better.
Sign Up