P. Michael Ho, MD, PhD

 Electronic consultations, or e-consults, between primary care providers and specialists have been shown to improve access to specialty care, shorten wait times, and reduce outpatient visits. The objective of this study was to evaluate differences in health care costs between patients who received an electronic specialty consultation and patients who received a face-to-face specialty consultation.

 Retrospective cohort evaluation of patients who received a specialty consultation in the Veterans Health Administration during 2016.

 Patients who received an e-consult were matched 1:1 to patients who received a face-to-face consultation using propensity scores. Total, outpatient, and inpatient health care costs over 3 and 6 months following the specialty consultation were compared using a generalized linear model with a gamma distribution and log link.

 e-Consults accounted for 1.8% (urology) to 9.6% (hematology) of specialty consultations, on average. Across 11 specialties, patients receiving an e-consult had significantly lower health care costs compared with patients receiving a face-to-face consultation, ranging from 3.6% (cardiology) to 30.7% (hematology) lower. This was largely driven by differences in outpatient costs. Patients receiving an e-consult had significantly lower outpatient costs for all specialties except cardiology, ranging from 6.9% (endocrinology) to 31.2% (hematology) lower. Three-month inpatient costs among those who received an e-consult were significantly lower only in cardiology (5.2%), nephrology (9.3%), pulmonary (13.0%), and gastroenterology (14.3%).

 Electronic specialty consultations are a potential mechanism to reduce health care costs and promote the efficient use of health care resources.

Patients receiving electronic rather than face-to-face specialty consultations had lower 3-month health care costs across 11 specialties, ranging from 3.6% lower in cardiology to 30.7% lower in hematology.

The cost differences between patients receiving electronic vs face-to-face specialty consultations were largely driven by differences in outpatient costs.

Electronic consultations between primary care providers and specialists may be a promising mechanism to reduce health care costs.

Electronic consultations, or e-consults, are non–face-to-face consultations conducted between clinicians through a shared electronic health record or web-based platform.

 They are most commonly used by primary care providers (PCPs) to solicit specialty input for their patients. e-Consults have been adopted across several health care systems to increase access to specialty care services, reduce burden on patients, and potentially reduce health care costs. Recent research results have shown that e-consults can reduce outpatient specialist utilization without reducing access to specialty expertise.

 However, it is unknown whether e-consults influence health care spending.

e-Consults could potentially reduce health care costs by facilitating efficient access to specialty care services and reducing unnecessary face-to-face visits. However, e-consults may increase health care costs if they lead to unnecessary diagnostic testing, provide lower-quality care, or delay (not supplant) the need for face-to-face visits. A 2018 publication by Anderson and colleagues examined the association between cardiology e-consults and health care costs for 369 Medicaid recipients cared for at a federally qualified health care center.

 They reported an approximate $466 cost savings per patient receiving a cardiology e-consult.

 However, evidence is lacking that examines the association between e-consults and health care costs for other specialties, among larger samples, and for payer types besides Medicaid.

The Veterans Health Administration (VHA) is the largest integrated health care system in the United States and provides primary and specialty care for more than 9 million veterans annually.

 An e-consult program was piloted among several VHA medical centers in 2009 and expanded nationally in 2011. More than 200,000 e-consults across 11 specialties were completed in the VHA in 2016. The objective of this retrospective cohort evaluation was to compare health care costs between patients who received a specialty care e-consult and patients who received a face-to-face specialty consult.

The population for this evaluation included patients who had a specialty care consultation, either electronically or face to face, in the VHA during calendar year 2016. Specialty care services are typically housed at one of the VHA medical centers, whereas primary care can be obtained from the medical center or community-based outpatient clinics. VHA PCPs request face-to-face or electronic specialty consultations by placing an order through the electronic health record. For e-consults, the reason for the consult, and any accompanying information, is recorded as free text. The specialist reviews this information and responds to the PCP electronically by providing their recommendation or by requesting additional information or testing. In some cases, the specialist may request a face-to-face visit with the patient.

A specialty consultation, which was further categorized as an e-consult or a face-to-face consultation, defined the exposure. Health care utilization data from the VHA’s Corporate Data Warehouse were used to identify e-consults and their associated patients and providers. We first identified patients who received an e-consult for 1 of the following 11 specialties: cardiology, diabetes, endocrinology, gastroenterology, hematology, neurology, oncology, pain medicine, pulmonary, nephrology, and urology. These 11 specialties were selected to align with prior e-consultation literature,

 are all uniquely identified from stop codes within VHA data, and collectively represent very different clinical care, with some potentially being more ideal for e-consults than others.

Patients in the e-consult cohort were then matched 1:1 to patients who received a face-to-face specialty consult. To identify a similar population to the e-consult cohort, propensity score matching was used to select patients for the face-to-face cohort. Patients were matched on individual characteristics (age, gender, marital status, and race), Medicaid eligibility (as a proxy for income and availability of secondary insurance), specialty of consultation, and severity (as measured by comorbidities, survival, and total health care costs prior to the index date). The date of the e-consult or face-to-face specialty visit served as the index date.

The primary outcomes were total health care costs at 3 and 6 months after the index specialty consultation. Secondary outcomes included inpatient and outpatient costs at 3 and 6 months. Data from the VHA’s Managerial Cost Accounting (MCA) system were used to calculate the health care costs for patients in each cohort.

 Cost data in the MCA system are based on the allocation of actual resources to an encounter.

 Total health care costs, total outpatient health care costs, and total inpatient health care costs were compared between the e-consult and face-to-face cohorts for 3 and 6 months after the index date. This time frame aligned with that in prior literature

 and was informed by clinician input on the duration of expected influence from a single e-consult. Patients with outlier 6-month costs were identified using the interquartile rule for outliers

 and were removed from the analysis. For patients in the analytic data set, the health care costs for 1 month prior to the index visit and for 6 months after the index visit were calculated at the monthly level.

We first quantified e-consult use across specialties by dividing the number of e-consults by the sum of e-consults and face-to-face specialty visits for each clinic. Our economic evaluation took the perspective of the VHA health care system and focused on direct health care costs accrued to the VHA health care system. For the cost outcomes, we calculated medians and 95% CIs for those receiving e-consults and those receiving face-to-face specialty consultations. Differences between the 2 cohorts were tested through Mann-Whitney U tests. This nonparametric test was used due to the positive skew observed with the cost data. Analyses were stratified by specialty of care and thus results were presented separately for each specialty. Total, outpatient, and inpatient health care costs were also evaluated using a generalized linear model with a gamma distribution and log link to account for the skewed cost data.

 The multivariable regression models controlled for patient characteristics (age, gender, and race), patient severity (as measured by the Charlson Comorbidity Index and dichotomous indicator variables for the individual comorbidities within the index), location (urban/rural location and distance to the nearest VHA facility), and VHA facility characteristics (facility complexity and a wage index

 to account for geographic variation in costs). Standard errors from the models were clustered at the facility level to account for potential correlation within a VHA facility.

 The dependent variables included the outcomes of interest detailed above. The primary independent variable was specialty visit categorization (ie, electronic or face to face). All models were estimated separately for each of the 11 specialties.

During the period of the study, veterans enrolled in VHA health care were eligible to receive specialty care from non-VHA providers and health systems through the Veterans Choice Program if they resided more than 40 miles away from a VHA provider or had to wait more than 30 days for an appointment.

 In an effort to identify a subset of the population who did not receive community care, a sensitivity analysis was conducted that restricted the patients in the analysis to those residing within 40 miles of a VHA facility.

The number of e-consults conducted in calendar year 2016 are provided in 

; they ranged from 4892 in the diabetes specialty to 42,141 in the cardiology specialty. Across sites, the average uptake of e-consults ranged from 1.8% of total visits in urology to 9.6% of total visits in hematology. Table 1 also reports the characteristics of patients receiving an e-consult vs face-to-face specialty consultation following propensity score matching. After matching, there are still statistically significant but overall small differences between the 2 patient populations. For example, among those who received cardiology specialty visit consultation, the mean age of patients was 67.2 years and 67.8 years in the e-consult and face-to-face cohorts, respectively (

Median 3- and 6-month total health care costs, stratified by specialty, are provided in 

. Patients receiving consultation for the cardiology specialty had the highest costs among the e-consult cohort, with median 3- and 6-month health care costs of $3312 and $6156, respectively. Patients receiving consultation for the diabetes specialty had the lowest costs among the e-consult cohort, with median 3- and 6-month health care costs of $2264 and $4150, respectively. In the face-to-face cohort, 3-month health care costs were highest for patients receiving consultation for the diabetes specialty ($3431); 6-month health care costs were highest for patients receiving consultation for the oncology specialty ($7109). Three-month health care costs were lowest for patients receiving face-to-face consultations for the nephrology specialty ($3196); 6-month health care costs were lowest for patients receiving face-to-face consultation for the urology specialty ($5281). After 3 and 6 months, health care costs were significantly higher in the face-to-face cohort compared with the e-consult cohort for all specialties except cardiology. Within the cardiology specialty, there was no significant difference in total costs at 3 months, but costs were significantly lower among patients who received e-consults at 6 months.

Results were similar when using generalized linear models and adjusting for covariates. 

 reports the multivariable regression model findings for 3- and 6-month total health care costs. Patients in the e-consult cohort had significantly lower 3-month health care costs across all specialties. Total cost differences ranged from 3.6% lower among patients receiving cardiology e-consults to 30.7% lower among those receiving hematology e-consults. Patients in the e-consult cohort had significantly lower 6-month health care costs across all specialties, except in cardiology, where no significant cost differences were observed between cohorts. Among the remaining 10 specialties, total cost differences ranged from 10.0% lower among patients receiving endocrinology e-consults to 39.7% lower among those receiving hematology e-consults.

The majority of the cost differences were due to differences in outpatient costs rather than inpatient costs. 

 presents the 3- and 6-month outpatient cost differences for the e-consult and face-to-face cohorts, stratified by specialty. Patients in the e-consult cohort had significantly lower 3-month outpatient costs for all specialties, except cardiology, ranging from 6.9% lower in endocrinology to 31.2% lower in hematology. At 6 months, costs were significantly lower for patients in the e-consult cohort for all specialties, except cardiology. Hematology once again had the largest comparative difference in outpatient costs.

 presents the 3- and 6-month inpatient cost differences for the e-consult and face-to-face cohorts, stratified by specialty. There were few significant differences in inpatient costs between the 2 cohorts. In comparison with those of the face-to-face cohort, the 3-month inpatient costs among those who received an e-consult were significantly lower in cardiology (5.2% lower), gastroenterology (14.3% lower), pulmonary (13.0% lower), and nephrology (9.3% lower). At 6 months, inpatient costs were 15.7% lower among patients who received hematology e-consults and higher among those receiving urology (16.7%) and cardiology (20.6%) e-consults.

More than 98% of the patients in the analysis lived within 40 miles of a VHA medical center. Results of the sensitivity analysis including only those patients were similar to the findings presented above, suggesting that the results were robust even without the direct inclusion of community care cost data.

The purpose of this evaluation was to examine the influence of e-consults on subsequent health care costs. This evaluation found that patients who received specialty e-consultation rather than face-to-face specialty consultation had significantly lower total health care costs following the consultation. This finding was observed within each of the 11 studied specialties at 3 months. This translates to 3-month cost savings ranging from $120 (cardiology) to $1100 (oncology) for each patient referred for an e-consult rather than a face-to-face consultation. Multiplying the number of e-consults by the difference in total cost for each specialty, this equates to an annual savings of approximately $110 million. This is the first large-scale evaluation that assesses health care costs associated with e-consults across a variety of specialties among a nationwide sample.

Cost savings among the e-consult cohort were largely driven by differences in outpatient rather than inpatient costs. This is likely because outpatient costs accounted for the majority of total costs at 3 and 6 months. Unfortunately, the cost data used to inform this analysis were aggregated to the patient-month and therefore we did not have specific health care utilization for each patient. Future work will evaluate specific health care utilization differences between e-consults and face-to-face specialty visits to more clearly identify what is driving these cost differences. We also observed large variation in e-consult utilization and cost savings between specialties. Differences in utilization across specialties may be attributable to differing levels of support and infrastructure provided to implement e-consults across facilities and specialties. It is also likely that some specialties are simply more aligned to provide care without the need for a face-to-face visit. For example, specialties that are heavily reliant on physical examination of patients, taking detailed and condition-specific clinical histories, or performing clinical procedures may be less likely to rely solely on electronic communication with other providers to make clinical decisions.

When interpreting these findings, it is important to also consider the operational cost of providing the e-consult, which primarily equates to PCP and specialist time. In the VHA, providers receive 1 of 3 levels of workload credit based on the time spent providing the e-consult.

 The workload accounting is then figured into how much money each VHA facility is allocated. Outside of the VHA, multiple potential reimbursement mechanisms exist for the operational cost of e-consults. For example, some hospitals provide salary support to providers to compensate their effort on e-consults. At the Mayo Clinic, e-consults are scheduled as 15-minute appointments and the providers then receive visit credit. Insurance companies could reimburse as traditional fee-for-service, and some state Medicaid programs provide a transactional payment for either the PCP, the specialist, or both.

 As e-consults are integrated and expanded in the health care system, the operational cost and potential reimbursement must be considered.

The findings from this analysis expand on a publication by Anderson and colleagues

 that evaluated differences in 6-month total health care costs between 369 Medicaid patients who were randomized to receive either a cardiology e-consult or a face-to-face visit. They found that 6-month health care costs were $466 lower for patients who had an e-consult, with the majority of the cost savings occurring in residual claims.

 Our evaluation found that 3-month health care costs were approximately $120 lower for veterans who had a cardiology e-consult, with the majority of the cost savings occurring in outpatient care. Although this number seems small, this translates to potentially substantial cost savings when considered within the context of an entire health care system, as more than 40,000 cardiology e-consults occurred within the VHA in 2016. Further, e-consults represented only approximately 4% of cardiology specialty care. With the potential for e-consult use to increase, savings could reach the millions of dollars for the cardiology specialty alone.

This work also adds to prior literature on e-consults and increasing access to specialty care by providing evidence that increasing access to specialty care through e-consults is also likely an efficient use of resources. This has unique implications for the VHA with the implementation of the MISSION Act in June 2019.

 The intent of the MISSION Act is to provide veterans with more health care options. The current system is structured so that it is easier for PCPs to refer eligible patients to the community, rather than referring patients to a specialist within the VHA. e-Consults could be an option to facilitate timeliness of access and specialty care recommendations within the VHA system. The speed of an e-consult is likely much faster than if the patient were to seek care outside of the VA. In addition to increasing the timeliness of specialty care access, e-consults would also decrease coordination efforts that are required when a veteran obtains care outside of the VHA system, and they would promote patient-centricity by precluding the patient’s need to drive to (or be transported to) additional health care services.

Given the coronavirus disease 2019 (COVID-19) global pandemic, the findings from this study are particularly timely and clinically relevant, as the importance of decreasing in-person face-to-face contact for patients and providers continues to increase. e-Consults are likely one mechanism that can help reduce face-to-face contact between patients and providers to reduce potential transmission of the virus that causes COVID-19. Given the unknown time course of the current COVID-19 pandemic, and that systems are continuing to encourage the adoption of non–face-to-face modalities, this may result in sustained and increased use of e-consults.

Although e-consultation may be a promising mechanism to deliver efficient specialty care, reduce health care costs, and reduce the need for face-to-face contact, a number of questions should be evaluated with further adoption. We have demonstrated significant cost savings at 3 and 6 months, but future work will be important to verify that the cost savings extend to the long term, to ensure that e-consultation is not simply delaying inevitable care. One limitation of our study is that we were unable to address the possibility of indication bias, in that patients might simply be referred for an e-consult when providers have simpler (ie, less costly) clinical questions. Future research should aim to compare the costs between consultation mechanisms among patients referred to specialty consultation for the same reasons (eg, atrial fibrillation medication management). Although prior work has shown that providers have reported favorably on the quality of care delivered to patients through e-consults, formal assessments of quality of care across a range of conditions have yet to be conducted within the VHA or elsewhere. It will be important to assess the impact of delivering e-consults for PCPs, specialists, and other clinical staff. Preliminary qualitative evaluations made by VHA PCPs have indicated that the e-consult process can be burdensome and may increase their clinical workload. Given the high level of burnout among VHA PCPs and the relationship between burnout and workforce turnover, potential unintended consequences of e-consult adoption should be considered. This would be consistent with other research showing that technological interventions, such as e-consults, may contribute to burnout among physicians.

 Lastly, we know little about patient perceptions regarding e-consults and how this mechanism of care may affect patient satisfaction and their perceptions of care quality. Some published research results suggest that patients, PCPs, and specialty physicians report being satisfied with e-consults due to the improved communication and timeliness of care; however, this work was limited in sample size, with data from 1 health system, and it included providers who had chosen to use e-consults voluntarily.

 This is a ripe and necessary area for additional research, especially as providers experience an increase in pressure to pursue e-consults and other non–face-to-face modalities.

Our evaluation has limitations, primarily related to the potential for residual confounding within our results. It is possible that the patients in the e-consult cohort had less severe (and thus less costly) conditions, and thus the PCP felt comfortable ordering an e-consult rather than a face-to-face specialty consultation. We matched patients in the e-consult cohort to patients in the face-to-face cohort through propensity score matching that adjusted for important clinical and sociodemographic characteristics that could affect cost. For example, we matched on health care costs prior to the index visit to balance the 2 cohorts on health care utilization. If patients in the face-to-face cohort had more severe conditions than patients in the e-consult cohort, we would expect their prior month costs to be higher; however, in 9 of the 11 specialties, the prior month health care costs were significantly equivalent or lower in the face-to-face cohort. We also matched on comorbidities and survival to promote groups that were similar in condition severity. Before matching, 1-year mortality was similar between the 2 groups (9.9% among patients who received a face-to-face consultation and 8.6% among patients who received an e-consult), which thus does not suggest that patients who had more severe conditions were reserved for face-to-face consultations. Further, this evaluation included only costs of health care provided by the VHA. During the evaluation time period, veterans could receive care from a community, non-VHA provider through the Veterans Choice Program if they satisfied select criteria, including living further than 40 miles from a VHA provider.

 Our analysis accounted for this by controlling for distance in all adjusted regression models and conducting a sensitivity analysis that included only patients who lived within 40 miles of a VHA facility and who thus would not be eligible for reimbursed community care based on the distance criteria. The results were consistent among our sensitivity analysis and complete analysis, suggesting that the cost savings would likely persist even if data on cost of community care were included. Similarly, the economic perspective of our evaluation was that of the VHA health care system and thus did not include potential costs to the patient or other members of society. There could be other tangible cost savings with e-consults, including patient travel expenses to a face-to-face visit and patient and/or caregiver time missed from work to attend a face-to-face visit. These potential cost savings were not included in this analysis and should be considered additive to the savings reported in this evaluation. Accordingly, the VHA may accrue additional cost savings if its reimbursements to patients for eligible patient travel expenses were reduced due to fewer face-to-face visits.

In this study of more than 200,000 specialty consultations occurring within the VHA in 2016, patients referred to specialty care through an e-consult rather than face to face had lower 3- and 6-month costs across nearly every specialty studied. Given that prior study results have shown that e-consults also represent a way to improve access to specialty health care, reduce appointment wait times, and reduce face-to-face consultations, this study provides additional, compelling support for the continued investigation into e-consults as a potential mechanism to deliver high-value and lower-cost care. Future work is needed to estimate longer-term costs and then compare the quality of care between e-consult and face-to-face mechanisms.

This work was funded by the VA Office of Rural Health and sponsored by the VA Office of Veterans Access to Care, Department of Veterans Affairs, Washington, DC, through a MyVA Access Improvement Project Grant: “VISN19 VA Denver Developing best practices for subspecialty e-consultation procedures.” The activities were undertaken in support of a Veterans Health Administration (VHA) operational project and did not constitute research, in whole or in part, in compliance with VHA Handbook 1058.05. Therefore, institutional review board approval was not required. The views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Veterans Affairs. The authors have no conflicts of interest related to this content to report.

 Center of Innovation for Veteran-Centered and Value-Driven Care, VA Eastern Colorado Healthcare System (MDW, PMH, RRK, JS), Aurora, CO; Department of Medicine, University of Colorado Anschutz Medical Campus (PMH, JS), Aurora, CO; School of Medicine, Case Western Reserve University (SRK), Cleveland, OH; Veterans Affairs Central Office (SRK), Washington, DC; Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System (JT-S, DHA), Seattle, WA.

 This work was funded by the Veterans Health Administration Office of Rural Health and sponsored by the Veterans Health Administration Office of Veterans Access to Care through a MyVA Access Improvement Project Grant.

 The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (MDW, SRK, JS); acquisition of data (JT-S, DHA); analysis and interpretation of data (MDW, PMH, SRK, JS); drafting of the manuscript (MDW, RRK, JT-S, DHA, JS); critical revision of the manuscript for important intellectual content (MDW, PMH, SRK, JS); statistical analysis (MDW); obtaining funding (RRK, DHA, JS); administrative, technical, or logistic support (PMH, RRK, JT-S); and supervision (DHA, JS).

 Melanie D. Whittington, PhD, Center of Innovation for Veteran-Centered and Value-Driven Care, VA Eastern Colorado Healthcare System, 1700 N Wheeling St, Aurora, CO 80045. Email: Melanie.whittington3@gmail.com.

1. Anderson D, Villagra V, Coman EN, et al. A cost-effectiveness analysis of cardiology eConsults for Medicaid patients. 

2. Chen AH, Kushel MB, Grumbach K, Yee HF Jr. Practice profile: a safety-net system gains efficiencies through ‘eReferrals’ to specialists. 

. 2010;29(5):969-971. doi:10.1377/hlthaff.2010.0027

3. Kim EJ, Orlander JD, Afable M, et al. Cardiology electronic consultation (e-consult) use by primary care providers at VA medical centres in New England. 

. 2019;25(6):370-377. doi:10.1177/1357633X18774468

4. Kim-Hwang JE, Chen AH, Bell DS, Guzman D, Yee HF Jr, Kushel MB. Evaluating electronic referrals for specialty care at a public hospital. 

. 2010;25(10):1123-1128. doi:10.1007/s11606-010-1402-1

5. Barnett ML, Yee HF Jr, Mehrotra A, Giboney P. Los Angeles safety-net program eConsult system was rapidly adopted and decreased wait times to see specialists. 

. 2017;36(3):492-499. doi:10.1377/hlthaff.2016.1283

6. Vimalananda VG, Gupte G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. 

. 2015;21(6):323-330. doi:10.1177/1357633X15582108

7. Kirsh S, Carey E, Aron DC, et al. Impact of a national specialty e-consultation implementation project on access. 

8. Managerial cost accounting (MCA). US Department of Veterans Affairs. Accessed August 29, 2019. https://www.herc.research.va.gov/include/page.asp?id=managerial-cost-accounting

9. Pirson M, Dramaix M, Leclercq P, Jackson T. Analysis of cost outliers within APR-DRGs in a Belgian general hospital: two complementary approaches. 

. 2006;76(1):13-25. doi:10.1016/j.healthpol.2005.04.008

10. Berkowitz SA, Seligman HK, Rigdon J, Meigs JB, Basu S. Supplemental Nutrition Assistance Program (SNAP) participation and health care expenditures among low-income adults. 

. 2017;177(11):1642-1649. doi:10.1001/jamainternmed.2017.4841

11. Medicare wage index for VA facilities. US Department of Veterans Affairs. Accessed August 29, 2019. https://www.herc.research.va.gov/include/page.asp?id=guidebook-wage-index

12. Desai M, Bryson SW, Robinson T. On the use of robust estimators for standard errors in the presence of clustering when clustering membership is misspecified. 

. 2013;34(2):248-256. doi:10.1016/j.cct.2012.11.006

13. 10 things about the Veterans Choice Program. US Department of Veterans Affairs. July 25, 2017. Accessed August 29, 2019. https://www.va.gov/health/newsfeatures/2017/july/things-to-know-about-the-veteran-choice-program.asp

14. VA launches new health care options under MISSION Act. News release. US Department of Veterans Affairs.June 6, 2019. Accessed June 6, 2019. https://www.blogs.va.gov/VAntage/61286/va-launches-new-health-care-
options-mission-act/

15. Gleason N, Ackerman S, Shipman SA. eConsult—transforming primary care or exacerbating clinician burnout? 

. 2018;178(6):790-791. doi:10.1001/jamainternmed.2018.0762

16. Rodriguez KL, Burkitt KH, Bayliss NK, et al. Veteran, primary care provider, and specialist satisfaction with electronic consultation. 

. 2015;3(1):e5. doi:10.2196/medinform.3725

Articles

This retrospective cohort evaluation found that patients receiving electronic, compared with face-to-face, specialty consultation had significantly lower health care costs for at least 3 months.

Cost Savings Associated With Electronic Specialty Consultations

To assess the early impact of implementation of the electronic consults (e-consults) initiative by the Veterans Health Administration (VHA), designed to improve specialty care access.

Observational cohort study exploiting a natural experiment begun in May 2011 at 12 VHA medical centers and expanded to 122 medical centers by December 2013.

The following were assessed: 1) growth of e-consults by VHA regional networks, medical centers, and specialty; 2) location of patient’s primary care provider (medical center vs community-based outpatient clinic [CBOC]); 3) potential patient miles needed to travel for a specialty care face-to-face consult in place of the observed e-consults using estimated geodesic distance; 4) use of specialty care subsequent to the e-consult.

Of 11,270,638 consults completed in 13 clinics of interest, 217,014 were e-consults (adjusted rate, 1.93 e-consults per 100 consults). The e-consult rate was highest in endocrinology (5.0 per 100), hematology (3.0 per 100), and gastroenterology (3.0 per 100). The percentage of e-consult patients with CBOC-based primary care grew from 28.5% to 44.4% in the first year of implementation and to 45.6% at year 3. Of those e-consult patients from community clinics, the average potential miles needed to travel was 72.1 miles per patient (SD = 72.6; median = 54.6; interquartile range = 17.1-108), translating to a potential savings of 6,875,631 total miles and travel reimbursement costs of $2,853,387.

E-consult volume increased significantly since inception within many medical and surgical specialties. For patients receiving primary care at one of more than 800 CBOCs, e-consults may decrease travel burden and direct travel costs for patients.

This article discusses a broad-scale implementation of outpatient specialty care e-consults within a patient-centered medical home model for a large integrated health system. 

This can be a patient-centered model that saves patients travel time and money. 

E-consults may obviate the need for some specialty care visits. 

Policy to financially support this model may be prudent. 

Increased uptake may occur in specific medical specialties.

The Veterans Health Administration (VHA) is the largest integrated healthcare delivery system in the United States. Like an accountable care organization (ACO), the VHA provides care to a defined group of enrollees with the goal of delivering coordinated, high-quality, patient-centered care.

 Approximately 50% of the 8 million veterans cared for in the VHA receive specialty care. Efforts to bring primary care closer to veterans’ homes have led to more than 800 community-based outpatient clinics (CBOCs). In contrast, specialty care remains largely concentrated in urban medical centers, although 41% of veterans enrolled in VHA live in rural communities. These and other barriers to access present challenges for the delivery of specialty care.

To improve access, efficiency, and coordination between specialty care and primary care, the VHA’s Specialty Care Transformation Program Office implemented electronic consults (e-consults) in 2011. Primary care clinicians request clinical guidance with the expectation of an expeditious, albeit asynchronous, response. E-consults and responses are integrated into the VHA’s electronic health record (EHR). Through e-consults, primary care clinicians can incorporate specialty care advice into veteran care and better prepare patients for specialty care visits if needed. Although e-consults have been implemented on smaller scales in other settings, the implementation of a national e-consult program has not been previously described.

The study’s objective was to describe the spread and impact of the VHA national e-consult program. We assessed the growth of e-consults by VHA regional networks, medical centers, and clinical specialty, and evaluated the use of e-consults based on the location of the patient’s primary care provider (PCP) (medical center vs CBOC). Finally, we estimated the number of miles patients may have potentially traveled for an in-person specialty care visit in the absence of an e-consult program. E-consultation implementation across the VHA system can serve as a model for technology-based interventions to improve access to specialist consultation and provide patient-centered specialty care.

The VHA has 152 medical centers and over 800 CBOCs within 21 regional networks (Veterans Integrated Service Networks [VISNs]). Each regional VISN has 4 to 8 medical centers and generally covers 1 to 4 states. In addition to medical centers, each VISN has CBOCs located 10 to 300 miles from a medical center, providing primary care to patients closer to their homes. To support veterans living far from medical centers (41% of the veteran population lives in rural areas), patients are frequently eligible for financial reimbursement for miles traveled for care. In general, veterans are assigned to a PCP at a CBOC or medical center closest to them.

At the discretion of the PCP, e-consults can be entered into the EHR for any specialty in which an e-consult mechanism has been implemented. E-consults about specific questions are generally entered as free text or into templates with pre-specified questions. Responding specialty clinicians have access to the medical records, including progress notes, laboratory data, radiology tests, and medications. E-consults generally focus on narrow questions that help address questions of diagnosis, testing, or management of a condition. Either specialist or PCP can convert an e-consult into a request for a face-to-face consultation. Follow-up communication with patients about recommendations, if any, is the responsibility of the PCP, unless otherwise identified in the EHR.

In May 2011, the e-consult program for certain specialties began at 12 VHA medical centers in 7 VISNs. Through a competitive process, 12 pilot-site medical centers received limited funding and national guidance for business rules. Oncology, diabetes, endocrinology, dementia, neurosurgery, cardiology, hematology, liver transplant, pain medicine, and rheumatology were the included specialties. Following initial rollout at pilot sites, Specialty Care Transformation worked with VISN leadership to expand the use of e-consults nationally. Further funding was provided until September 30, 2014, to all VISNs; local facilities could implement e-consults in other clinical specialties based on facility needs. Subsequently, e-consults became a national priority and use of e-consults was tracked for each medical center as a component of VHA goals for patient-centered care delivered virtually.

We focused on the following clinics of interest to Specialty Care Transformation: endocrinology, hematology, neurology, pulmonary, nephrology, urology, cardiology, gastroenterology, oncology, pain medicine, geriatric, dementia, and diabetes. Consults were identified in administrative outpatient visit data that included specialty clinic category (ie, clinic stop codes) and a designated Current Procedural Terminology code to distinguish e-consults. We focused on medical centers in the continental United States and patients with assigned PCPs.

E-consult volume adjusted for clinic volume. 

Many medical centers implemented e-consults in multiple specialties. For every specialty/site combination, we calculated daily rolling rates of e-consult volume adjusted for total clinic volume. For a given 30-day window, we summed the number of e-consults and divided by the number of total consults.

 Travel distance was estimated by the geodesic (“as the crow flies”) distance from patients’ residence to their “home station” (ie, facility where they received primary care). The adjusted distance was considered the difference between the distance from the patient’s residence to the specialty care facility and the distance from the patient’s residence to their primary care facility. Distances were considered erroneous and set to missing if the patient distance to primary care exceeded 150 miles or the adjusted distance was lower than —25 miles. (Negative values may occur if the patient’s primary care location is farther away than the specialty clinic completing the e-consult.) Primary care visits outside the specialty care administrative region didn’t contribute to home station identification; patients receiving primary care outside that region were considered to have missing home stations. Patients with home locations outside the contiguous 48 states were excluded from all analyses.

We aggregated the number of completed e-consults by specialty, regional network, and location where patients received primary care, either at a CBOC or medical center. We reported the quarterly frequency of e-consults from May 2011 to December 2013, stratified by VISN and specialty. Quarterly e-consults were adjusted by total consults and expressed as a rate. We calculated the e-consult rate to facilitate comparisons across VISNs, facilities, and clinic specialties.

We calculated the proportion of patient consults receiving primary care at a CBOC for each yearly quarter, stratified by type of consult (e-consult or not). We used χ

 tests to compare binary outcomes (patient receiving primary care at a CBOC; subsequent face-to-face consult after the first consult; and primary care visit in the 3 months following the first consult) between e-consults and face-to-face consults. Among patients receiving primary care at a CBOC, we used Wilcoxon rank-sum tests to compare the distribution of miles needed to travel between e-consults and face-to-face consults.

Statistical tests were performed using R software, version 3.1.0, (R Foundation for Statistical Computing, Vienna, Austria). Initial data aggregation was performed with SAS version 9.3 (SAS Institute, Cary, North Carolina).

We identified 14,182,136 specialty care visits between May 1, 2011, and December 31, 2013, of which 262,143 were e-consults. We excluded visits outside of the continental United States (n = 275,611), visits occurring outside of VHA medical centers (n = 1,265,169), visits by patients without an assigned primary care clinic (n = 692,726), visits by patients whose assigned primary care clinic was not a CBOC or VHA medical center (eg, residents of long-term care facilities) (n = 986,759), sites that had less than 1 e-consult per 10,000 visits (n = 1,127,251), and sites with less than 1000 total visits across the time period (n = 87,973). These categories had some overlap, and 2,911,498 visits in total were excluded.

There were 217,014 completed e-consults that met the inclusion criteria in the 13 clinics of interest in all 21 VISNs. During the same time period, there were 11,270,638 total consults that met the inclusion criteria, giving an adjusted rate of 1.93 e-consults per 100 consults. At the VISN level, the rate of e-consults per 100 total consults varied (mean = 2.0; SD 

0.8; median = 1.9; interquartile range [IQR] = 1.4-2.4). Although there has been some variability in uptake of e-consults across VISNs, it has generally increased with time (

). At the individual medical centers, the range of e-consults has varied from 0.02 to 26 per 100 consults.

E-consults were completed most frequently in cardiology (n = 44,322), gastroenterology (n = 29,043) and endocrinology (n = 23,972). Adjusting for total consult volume, the rate of e-consults were highest in endocrinology (5.0 per 100 total consults), hematology (3.0 per 100 total consults), and gastroenterology (3.0 per 100 total consults) (

Comparing e-consults to face-to-face consults, there were differences in both: 1) the percent of patients coming from outside the specialty care medical center, and 2) the distance from their primary care center to the specialty care center. During the study period, comparing the patients participating in e-consults with those participating in face-to-face consults, the e-consult population had a higher percentage of patients who received primary care at CBOCs (45.3% vs 36%; 

 <.0001). In the first year of implementation, the percent of e-consult patients with CBOC-based primary care grew from 28.5% to 44.4%. This increase slowed in the subsequent 2 years, growing from 44.4% to only 45.6%. In the same 3-year period, the percent of face-to-face consult patients with CBOC-based primary care grew slightly, from 34.7% to 37% (see 

). Among the patients receiving primary care at CBOCs outside of a medical center, the distance from their PCP to the medical center was farther for patients receiving e-consults (38 vs 32 miles; 

 <.0001) compared with patients receiving face-to-face consults. These results suggest that compared with face-to-face consults, e-consults are used more frequently for patients with lower spatial access to specialty care.

Geographic distributions of “hub” medical centers and “spoke” community clinics demonstrate wide variation in number of spoke connections for each hub, as well as number of consults between each hub and spoke combination (

). The Figure 3 map highlights the geographic reach of e-consults to patients with potentially less access to specialty care.

In the 3 months following a specialty care consult, patients who had received e-consults were less likely to have subsequent face-to-face visits with the same specialty compared with patients with a face-to-face consult (18.6% vs 43.2%; 

 <.001). In contrast, e-consult patients were more likely to have a subsequent primary care visit compared with patients with a face-to-face consult (78.8% vs 67.4%; 

 <.001), suggesting that patients receiving an e-consult have a higher probability of follow-up with their PCP for subsequent care compared with patients with a face-to-face specialty care visit.

Finally, we calculated the potential miles needed to travel to receive a face-to-face consult in lieu of an e-consult. Among the 217,014 e-consults, 93% (201,735) had acceptable distance calculations. Among those patients, 52.7% (n = 106,310) received primary care from the same facility; thus there were no potential mileage-traveled differences between an in-person consult and an e-consult. For the 47.3% (n = 95,425) of e-consults for patients from CBOCs, the average potential miles needed to travel was 72.1 miles per patient (SD = 72.6; median = 54.6; IQR = 17.1-108) (see 

). This distance translates into 6,875,631 total potential miles needed to travel. During the time period, the VHA reimbursed patient mileage at a rate of 41.5 cents per mile; thus, the total potential miles needed to travel would translate into potential direct costs of $2,853,387. These mileage estimates are conservative because they are based on “as the crow flies” rather than true distance necessary to be driven.

Our study describes the implementation, spread, and impact of the VHA national e-consult program. Since VHA implementation in 2011, nearly 2% of all consults were e-consults without evidence for a plateau in their use. In addition, receipt of e-consults for patients with primary care at CBOCs was associated with reductions in the total potential miles traveled and travel costs. These findings have important implications for the VHA for addressing both spatial and nonspatial barriers to access for specialty care services, especially for veterans who live in rural communities. Furthermore, our results provide potential approaches to specialty care access for ACOs and other integrated healthcare delivery systems.

 Like other healthcare institutions, VHA has adopted the “triple aim” framework for optimizing healthcare delivery to improve the patient healthcare experience and the health of the patient population while balancing per capita cost.

 The implementation of e-consults is a potential mechanism by which VHA can strive to achieve these aims.

E-consults may improve the overall patient experience of care by reducing inconvenience and waiting times for specialist consultation/input. In addition, patients have the opportunity to avoid long travel distances to see specialists and to obtain specialty advice in a timely manner compared with an in-person visit. For patients who either prefer or require an in-person specialist visit, the initial e-consult may also provide an opportunity to be more prepared for the visit. For example, by arranging for diagnostic or laboratory tests in advance, some visits to the specialist could be avoided and time to needed services potentially decreased. We found that for the majority of consults, an in-person visit did not occur following the e-consult. This finding reinforces the study of Keely et al, who found that specialists were able to answer the e-consult without needing further information 89% of the time.

E-consults have the potential to improve quality of care for both individuals and the general patient population. First, e-consults improve patient access to specialist expertise, especially for the large number of veterans who live in rural communities. We found higher utilization of e-consults for those in CBOCs and more follow-up appointments with primary care after e-consults than for those patients not receiving e-consults. Second, e-consults promote the use of a standardized referral process with iterative communication that can lead to increased effectiveness of care delivered. In fact, preliminary results of interviews with clinicians have found that both PCPs and specialists value the benefits e-consults have for patients. In addition, e-consults provide an opportunity to improve care coordination for patients as a consequence of better PCP-specialist communication. Third, when combined with the capabilities of an EHR, it is possible to identify populations who would benefit from additional specialty care expertise. “Consultation” may be pre-emptive, thereby avoiding preventable morbidity.

 This population management approach will need to be evaluated prospectively.

E-consults may reduce the per capita cost of healthcare from the healthcare system perspective. Reductions may be accomplished by improved efficiency of use of high-cost specialists. On the other hand, increased access to specialists may result in increased testing and consequent expense; healthcare value may improve, albeit without reducing costs.

 However, we believe that the overall costs will be less with use of e-consults over time due to primary care coordination. VHA cost structure also has some unique aspects. The VHA outsources specialty care—particularly for rural veterans&mdash;because of difficulties for these veterans to access specialists. This so-called “fee-basis” care consumes funds that are used for outside care and do not return to the system. In addition, specific populations of veterans are reimbursed for their travel to clinic visits; generally, this reimbursement is provided to veterans with at least a 30% service-connected disability. The elimination of unnecessary visits subsequently reduces travel reimbursement costs. Finally, efficiency may be improved by better coordination of care and less fragmentation of care.

 We are not suggesting that e-consults will obviate the need for face-to-face visits, but this technology-supported platform can optimize efficient use of specialist expertise, which may be limited in differing geographic locations and health systems.

E-consult programs have been established in a variety of healthcare systems. However, in order for e-consults to be more widely adopted, changes to the payment systems are needed. The VHA’s model of salaried physicians and capitated reimbursement provides a more favorable environment for e-consult implementation. In contrast, under the fee-for-service model, specialists must physically see the patient and bill for a separate visit in order to receive payment. Thus, although the reach of technology expands and allows the extension of care delivery into areas where few specialists reside, payment models lag behind and may hinder the potential spread of such programs. It will be important to assess how the different models for reimbursement and shared savings in ACOs under the Affordable Care Act evolve; their effects on different types of specialist care delivery and coordination of care remain to be seen.

Our study has several limitations. First, the rate of increase of e-consults may reflect changes in documentation; formalized coding of e-consults was part of the initiative because of its relevance to tracking patient care and the necessity to capture the work of specialty care providers. Second, the e-consult initiative was conducted within an integrated healthcare delivery system and a single EHR enabling ready access to information for both PCPs and specialists. Nevertheless, trends in use of EHRs and efforts to achieve interoperability may render implementation of e-consults in other settings less difficult in the future. The logistical issues for implementing such a system in the fee-for-service sector are likely to be different. This may include recognition that some healthcare systems may need to contract out for specialist care; contracted specialists may not be interested in supporting large-scale e-consult initiatives. However, as the country moves toward ACOs, the lessons learned in implementing this initiative in the VHA may be applicable to ACOs. Finally, although the number of e-consults has increased, we do not know if these e-consults have resulted in improved patient outcomes. Future studies must be done to address any potential unintended consequences for patients, clinicians, and healthcare systems, as well as other unanswered questions. Last, the patients’ perspectives need to be considered and to that end, we will be conducting interviews with patients in the near future to obtain their feedback and experience with e-consults.

Since the e-consult initiative rolled out in 2011, there has been a steady increase in the use of e-consults across 21 networks and for a diverse group of specialties. Specifically, the number of patients with primary care at CBOCs—which are relatively remote from sites where specialists are located&mdash;receiving an e-consult has increased. Use of e-consults has been associated with reductions in potentially unnecessary driving, as evidenced by potential miles saved. These findings have important implications for the VHA in terms of increasing access to specialty care in general and to veterans who live in rural communities in particular, and the findings may apply to other healthcare systems.

The authors thank the Evaluation Center Quantitative Analysis Group: Jackie Szarka, PhD; Jeffrey A. Todd-Stenberg, BA; Christian D. Helfrich, PhD, MPH; Anne C. Lambert-Kerzner, PhD, MSPH; Catherine T. Battaglia, PhD, MS; Thomas J. Glorioso, MS; Kent Davis; Katherine M. Fagan Williams, MPH. All authors and the Evaluation Center Quantitative Group had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

 Office of Specialty Care Services (SK, OC, GG), Patient Care Services (RJ), Veterans Health Administration, Washington, DC; Louis Stokes Cleveland VA Medical Center (SK, DCA), Cleveland, OH; Case Western Reserve University School of Medicine (SK, DCA), Cleveland, OH; Eastern Colorado VA Medical Center (EC, PMH), Denver, CO; Puget Sound Health Services Research & Development Center of Innovation, Department of Veterans Affairs (DHA), Seattle, WA; Department of Medicine, University of Washington (DHA), Seattle, WA; New Jersey Veterans Health Care System, East Orange Campus (C-LT, HF), East Orange, NJ

 This material is based upon work supported by the US Department of Veterans Affairs, the Office of Specialty Care Transformation and the Office of Research and Development Quality Enhancement Research Initiative.

 The authors are employed by the VA and this work was funded, in part, by the VA Office of Specialty Care Transformation, the office overseeing the e-consult initiative. The views expressed here do not necessarily reflect the position or policy of the Department of Veterans Affairs. The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

 Concept and design (OC, HF, DHA, DCA, GG, RJ, PMH, EC, SK); acquisition of data (DHA, GG, EC, SK); analysis and interpretation of data (OC, HF, DHA, C-LT, DCA, GG, RJ, EC, SK); drafting of the manuscript (DHA, C-LT, DCA, EC, SK); critical revision of the manuscript for important intellectual content (OC, DHA, DCA, GG, RJ, PMH, EC, SK); statistical analysis (HF, C-LT, EC, SK); provision of patients or study materials (GG, SK); obtaining funding (OC, DHA, DCA, SK); administrative, technical, or logistic support (DHA, GG, PMH, SK); and supervision (DHA, GG, RJ, SK).

 Susan Kirsh, MD, MPH, Case Western Reserve University School of Medicine, 10701 East Blvd, Cleveland, OH 44106. E-mail: susan.kirsh@va.gov.

1. Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. 

2. Kizer KW, Dudley RA. Extreme makeover: transformation of the Veterans Health Care System. 

3. Kizer KW. Veterans and the Affordable Care Act. 

4. Chen AH, Yee HF Jr. Improving the primary care-specialty care interface: getting from here to there. 

5. Mehrotra A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. 

E-consult implementation grew from 12 to 122 VHA sites with multiple specialties. The adjusted e-consult rate of 1.93/100 consults saved significant patient travel miles and costs.

Impact of a National Specialty E-Consultation Implementation Project on Access

To evaluate whether systolic blood pressure (SBP) control is maintained following participation in a multimodal hypertension intervention.

 This was a retrospective cohort of patients completing the Improving Blood Pressure in Colorado study, a randomized trial comparing a multimodal intervention with usual care for patients who had uncontrolled hypertension. Chart review assessed the first SBP measurement recorded as part of routine care after the study ended. Among patients who had controlled SBP at the final study visit, the proportions who had uncontrolled SBP during follow-up were compared for the intervention and usual care (UC) groups. Kaplan-Meier estimates assessed time to uncontrolled SBP by treatment arm.

Of 283 patients completing the Improving Blood Pressure in Colorado study, 51.5% in the intervention and 46.9% in the UC group had controlled SBP at the final study visit. Of patients with controlled SBP, 37.0% and 46.4% of patients in the intervention and UC groups, respectively, had uncontrolled SBP at their initial measurement during follow-up (

 = .32). There was no difference in median time to uncontrolled SBP (126 vs 114 days for the intervention and UC groups, respectively; 

SBP control was not maintained in a significant proportion of patients in both groups following hypertension study participation. These findings suggest the need for interventions to focus on longer-term BP control in contrast to the short duration of most hypertension intervention trials.

	A randomized trial compared a multimodal intervention with usual care for patients who had uncontrolled hypertension.

		Systolic blood pressure control was not maintained in a significant proportion of patients in both groups following participation in this study.

		These results suggest the need to focus on long-term hypertension control.

	Approximately 65 million US adults have hypertension and this number is expected to grow as the population ages.

Hypertension is a significant cardiovascular risk factor and the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7) recommends achieving systolic blood pressure (SBP) goal as the primary focus of hypertension therapy.

Although the degree to which SBP is lowered is directly related to the degree to which adverse outcomes are reduced, national surveys demonstrate that only about half of patients with hypertension have their blood pressure (BP) treated to levels recommended by guidelines.

 Further, BP control over time is directly associated with cardiovascular events, suggesting that both achieving and maintaining BP goals are important.

	Interventions using pharmacists, nurses, home BP monitoring, and/or technology such as interactive voice response (IVR) alone or in combination have resulted in improved BP control compared with usual care in the short term (6-12 months).

 However, given that hypertension is a chronic condition, long-term management is necessary. Unfortunately, little is known about maintenance of BP control following patients’ participation in hypertension intervention studies or about the durability of BP reduction after the conclusion of a study.

	The Improving Blood Pressure in Colorado study was a randomized study that evaluated a multimodal intervention for patients with uncontrolled hypertension, utilizing IVR technology and clinical pharmacist— physician management for 6 months.

 The objective of the current study was to evaluate maintenance of SBP control in the short term after participation ended in the clinical trial, when patients returned to their primary care providers for care.

	The Improving Blood Pressure in Colorado study (clinicaltrials. gov; Identifier: NCT00520988) was conducted from 2006 to 2009 at 3 healthcare systems across the Denver metropolitan area, including the Denver Veterans Affairs Medical Center, Kaiser Permanente Colorado, and the Denver Health Medical Center. The randomized study was approved by the institutional review board at each site. The current study was approved as an addendum to the initial study and included patient waiver of consent.

Methods for the Improving Blood Pressure in Colorado Study

	Criteria for enrollment into the main prospective, randomized study were the following: 18 to 85 years of age, diagnosed with hypertension, uncontrolled BP (>140/90 mm Hg, or >130/80 mm Hg for patients with diabetes or chronic kidney disease), and receiving <4 antihypertensive medications. All patients attended a baseline, in-person clinic visit to have their BP assessed. Three BP measurements were obtained and if the average of the latter 2 measurements was above patient-specific targets, those patients were eligible for enrollment. Patients randomized to the intervention were provided a hypertension educational booklet from the National Institutes of Health (

), instruction on the use of the IVR system, and an electronic BP monitoring machine. Patients were taught how to measure home BP, instructed to take their BP 3 to 4 times per week, and told to report their BP readings via IVR weekly. A clinical pharmacist reviewed the BP entries from the IVR system and managed BP via telephone visits, utilizing treatment protocols approved by primary care physicians. Patients randomized to usual care were given the same educational booklet on hypertension and instructed to follow up with their primary care providers for BP management. All patients returned at 6 months and had 3 BP readings taken, with the latter 2 averaged to determine the final BP reading for the study. The person obtaining the final readings was blinded to patient group assignment. Blood pressure control was defined as <130/80 mm Hg for patients with diabetes or chronic kidney disease and <140/90 mm Hg for all others.

	To evaluate maintenance of SBP control following patient participation in the current study, we performed chart reviews for all participants who completed the end-of-study visit (n = 283). We abstracted BP measurements taken as part of routine clinical care at all outpatient nonemergent care visits within 6 months following the end of participation in the clinical trial. Data were collected for both intervention and usual care group patients. Patients with no follow-up BP values in the 6-month period (n = 55) were not included in the analysis. The primary outcome for the follow-up study was the proportion of patients with uncontrolled SBP using JNC 7—defined goals (ie, SBP >140 mm Hg, or >130 mm Hg for those with diabetes or chronic kidney disease) based on their first BP measurement following the end of participation in the Improving Blood Pressure in Colorado study.

	Descriptive statistics (means, standard deviations, and percentages) were used for patient demographics at baseline for each group. X

 tests for continuous variables were used for group comparisons between intervention and usual care patients. Kaplan-Meier estimates were used to model the probability of uncontrolled SBP by follow-up time for each treatment arm. To identify predictors of uncontrolled SBP, a multivariable logistic regression model was created, using baseline characteristics at the time of participation in the clinical trial and included all 

variables. As a secondary analysis, we also assessed the proportions of patients with uncontrolled SBP in the intervention and usual care groups by averaging all available SBP measurements during the initial 6 months following the end of study participation. Statistical analyses were performed using SAS version 9.2 (SAS Institute, Inc, Cary, NC).

	Of the 283 patients who completed the Improving Blood Pressure in Colorado study, 51.5% (71/138) in the intervention group and 46.9% (68/145) in the control group had controlled SBP at the final study visit. The magnitude of SBP reduction was greater in the intervention group than in the usual care group despite similar rates of patients with controlled SBP (-13 mm Hg vs -7 mm Hg for the intervention group vs the usual care group, respectively; 

 = .004). More than 24% (n = 34) and 14% (n = 21) of the intervention and usual care group members, respectively (

 = .03), did not have follow-up BP measurements within 6 months of study completion. Patients with a BP measurement following study completion were older than patients without a BP measurement (65.4 vs 61.5 years; 

 = .02), but other baseline characteristics were comparable (Table 1).

	Baseline characteristics of patients with BP measurements within the 6 months following study end were comparable, including the number of patients with diabetes or chronic kidney disease (

). The mean and median numbers of BP measurements in the 6 months after end of study participation were 2.8 and 2.0, respectively. The median time to first BP assessment was 48 days with no difference between treatment groups (45.5 vs 51.0 days [

), with SBP increasing from 128 mm Hg to 139 mm Hg. Of the 56 usual care patients whose SBP was controlled at the end of the study, 46.4% (n = 26) had out-of-control SBP at their first follow-up, with SBP increasing from 129 mm Hg to 143 mm Hg (

 = .32 for comparison between treatment arms for the percentage of patients with uncontrolled SBP and 

 = .60 for comparison between groups for average SBP at first follow- up). There was no significant difference in the median time to uncontrolled SBP for intervention (126 days) and usual care (114 days) patients (

 = .47). In the assessment of factors associated with follow-up SBP control, only SBP control at end of study was associated with subsequent control (odds ratio, 0.31; 95% confidence interval, 0.16-0.57).

	In secondary analysis (when we used all available SBP measurements taken during the 6 months after the study ended), a similar proportion of intervention (29.6%) and usual care (29.1%) patients had uncontrolled SBP. For patients with uncontrolled SBP at the study’s end, 70% (n = 35) of intervention patients and 63.2% (n = 43) of usual care patients continued to have uncontrolled SBP at their first assessment following the study (

	The objective of this study was to describe SBP control following participation in a hypertension intervention study, when patients returned to their usual care settings. Among patients whose SBP was controlled at the end of the study, 42% had out-of-control SBP at the first BP measurement obtained within 6 months of study completion. Among patients whose SBP was not controlled at the end of the study, the majority continued to have uncontrolled SBP during follow-up (~66%). These findings highlight the difficulty of maintaining SBP control and suggest the importance of trying to obtain consistent BP control over time rather than control at discrete time points.

	Little is known about SBP control following participation in hypertension intervention studies. To date, only 1 prior published study assessed BP control following participation in a hypertension intervention study, which compared a physician— pharmacist collaborative intervention with usual care. Carter et al showed that at 9 months following the completion of the study, SBP had worsened for both intervention and control patients, although to a greater degree in the usual care group.

 In their study, approximately one-third of patients whose SBP was controlled at the end of the study lost that control at 9 months. In contrast, we found that a greater proportion of patients developed uncontrolled SBP and that the uncontrolled SBP developed much sooner, at around 4 months in our patient population. These findings demonstrate that gains in SBP control made during the study were not sustained. Future studies should identify potential reasons for loss of SBP control, including nonadherence to medications or lifestyle changes.

	The findings of our study highlight the importance of approaching hypertension control longitudinally rather than at discrete time points. This conclusion is supported by studies that have demonstrated that ambulatory BP measurements (which generally include more measurements over time) predict cardiovascular outcomes better than clinic BP measurements. 

 In addition, the INVEST Study showed that a higher proportion of visits where BP was controlled (which indicated consistency of control) were associated with a reduction in death, nonfatal myocardial infarction, or stroke.

 Despite the importance of longitudinal BP control, national epidemiologic studies such as the National Health and Nutrition Examination Survey and quality-of-care performance measures from the Healthcare Effectiveness Data and Information Set still assess BP control based on measurements at 1 point in time or a single measurement, which may overlook the chronic nature of hypertension.

Systolic blood pressure control was not maintained in a large proportion of patients after the end of participation in a hypertension intervention study.

P. Michael Ho, MD, PhD

Articles

Cost Savings Associated With Electronic Specialty Consultations

Impact of a National Specialty E-Consultation Implementation Project on Access

Systolic Blood Pressure Control After Participation in a Hypertension Intervention Study

A Multimodal Blood Pressure Control Intervention in 3 Healthcare Systems