A Cost-Effectiveness Analysis of Cardiology eConsults for Medicaid Patients

January 17, 2018

A randomized trial of eConsults for cardiology referrals from primary care resulted in significant reductions in total cost of care compared with traditional face-to-face consultations.

ABSTRACT

Objectives: To evaluate the cost-effectiveness of electronic consultations (eConsults) for cardiology compared with traditional face-to-face consults.

Study Design: Cost-effectiveness analysis for a subset of Medicaid-insured patients in a cluster-randomized trial of eConsults versus the traditional face-to-face consultation process in a statewide federally qualified health center.

Methods: A total of 369 Medicaid patients were referred for cardiology consultations by primary care providers who were randomly assigned to use either eConsults or their usual face-to-face referral process. Primary care providers in the eConsult arm transmitted consults to cardiologists using a secure peer-to-peer communication platform in an electronic health record. Intention-to-treat analysis was used to assess the total cost of care and cost across 7 categories: inpatient, outpatient, emergency department, pharmacy, labs, cardiac procedures, and “all other.” Costs are from the payer’s perspective.

Results: Six months after the cardiology consult, patients in the eConsult group had significantly lower mean unadjusted total costs by $655 per patient, or lower mean costs by $466 per patient when adjusted for non-normality, compared with those in the face-to-face arm. The eConsult group had a significantly lower cost by $81 per patient in the outpatient cardiac procedures category.

Conclusions: These findings suggest that eConsults are associated with total cost savings to payers due principally to reductions in the cost of cardiac outpatient procedures.

Am J Manag Care. 2018;24(1):e9-e16Takeaway Points

Electronic consultations (eConsults) improve access, timeliness, and coordination of care compared with traditional face-to-face consultations. Findings from this study suggest that the use of eConsults is associated with cost savings to payers due principally to reductions in the cost of cardiac outpatient procedures.

  • The implications of cost savings demonstrated in this study are important for state Medicaid agencies and other health systems seeking new ways to improve access and quality while reducing cost.
  • Policy changes that support the use of eConsults could result in significant savings to the Medicaid program in a relatively short time frame.

Many initiatives aimed at transforming primary care have concentrated on the development of patient-centered medical homes, with emphasis on elements including the adoption of electronic health records (EHRs), multidisciplinary team-based care, and care coordination. Fewer efforts have been directed at improving the interface between primary care providers (PCPs) and specialists in the outpatient setting.1-3 This gap is notable given the significant clinical importance and financial impact of the PCP—specialist relationship. Outpatient specialty visits represent a disproportionate source of year-over-year increases in healthcare expenditures,4,5 with research suggesting that a typical PCP interacts with more than 200 specialists in a year.6 Such financial considerations are increasingly important as payment reform gains momentum across the country and stimulates experimentation with novel reimbursement arrangements. Additionally, the proliferation and adoption of new technologies, including EHRs and secure health information exchanges, are creating fertile conditions for improving the interface between specialists and PCPs.

Electronic consultations (eConsults) are non—face-to-face (F2F) consultations between a PCP and a specialist that utilize secure messaging to exchange information. Unlike electronic referral systems that link primary care practices with specialty providers for F2F appointment triage, eConsults provide a virtual consultation by the specialist after clinical information sent by the PCP is reviewed and returned with recommendations, which potentially eliminates the need for the patient to be seen in person by the specialist. Health systems that implemented eConsults have improved specialty access, reduced wait times,7 and decreased F2F consultations between 9% and 51% depending on setting and specialty.8-14 However, few studies have evaluated the effects of PCP access to a secure eConsult platform on total healthcare expenditures. Findings using retrospective data from an eConsult program in Canada suggest the potential for cost savings,15,16 but these studies were not randomized and did not evaluate the impact on total cost of care. The reduction in F2F visits with specialists is a potential source of cost savings to payers, but these savings could be offset by an increase in primary care costs and the cost of administering an eConsult program. We recently published results of a cluster-randomized controlled trial of eConsults for cardiology in a statewide federally qualified health center (FQHC) in Connecticut14 that demonstrate significant improvements in access and timeliness of care with a reduction in cardiology utilization. In this article, we report the impact of the intervention on cost for the subset of Medicaid-insured patients in this trial.

METHODS

Setting

Community Health Center, Inc, (CHCI) is a statewide multisite FQHC providing comprehensive primary medical, behavioral, and dental care to medically underserved patients in Connecticut. CHCI delivers care in 13 primary care clinics as well as in numerous school-based and homeless shelter—based facilities. All sites use an integrated EHR. Patients receive primary medical care from internists, family physicians, pediatricians, nurse practitioners, and physician assistants. Most of CHCI’s practice sites refer to hospitals and specialists within their neighboring communities or to large regional academic medical centers. During the study, more than 60% of CHCI’s patients were racial/ethnic minorities, more than 90% had incomes at or below 200% of the federal poverty level, more than 60% had state Medicaid insurance, and almost 25% were uninsured.

Study Design

Complete details of the design and methods of the trial have been published.14 Briefly, the intervention period for the eConsult study was between August 1, 2012, and June 30, 2013, and involved 590 patients and 36 providers from CHCI and 3 cardiologists from the University of Connecticut Health Center (UCHC). All consenting PCPs were assigned to the intervention (eConsult) or control (F2F) arm using 1:1 blocked randomization at the level of the PCP. No other parameters were used. There were no significant differences in site of practice between the intervention and control sites. All providers at all practices accepted all patients regardless of insurance status.

Intervention providers used eConsults for all nonurgent cardiology referrals except for patients who had an established relationship with a cardiologist. Determination of urgency was at the discretion of the PCP. The eConsult option was a function embedded in the EHR that allowed direct electronic communication between the PCP and the cardiologist. The eConsult included a specific question and relevant documentation, such as a brief clinical history, electrocardiograms, medication lists, laboratory and procedure results, and progress notes. A referral coordinator managed the eConsult process. The participating cardiologist received an email notification each time an eConsult was submitted, retrieved the eConsult from a secure Web portal, and responded within 2 business days. Their responses generally provided answers to PCPs’ questions and included other relevant suggestions, such as additional laboratory readings/tests or therapeutic trials prior to a subsequent consult, or occasionally a recommendation for a F2F visit. When a F2F consultation was recommended, providers and patients were free to choose any cardiologist accepting FQHC referrals in the service area. Providers in the control group sent all cardiology consults via the traditional F2F referral process at CHCI (Figure). The institutional review board of CHCI approved the study.

Data Sources

The economic analysis used demographic information for participating PCPs and their patients from CHCI’s practice management system and Medicaid paid claims data between August 8, 2011, and February 21, 2014.

Statistical Analysis

Three types of analysis were conducted: 1) an intention-to-treat (ITT) analysis, 2) an analysis of actual treatment (AT) received, and 3) a sensitivity analysis.

In the ITT analysis (Group B vs C in the Figure), all claims from patients in the PCP intervention and control arms were counted in their respective groups regardless of provider’s or patient’s adherence to their assigned consultation arm.

In the AT analysis, patients were grouped based on actual consultation choice (eConsult vs F2F), regardless of the provider’s assigned group. This second analysis regrouped claims of patients of intervention PCPs who were reassigned to a F2F consult as per the study protocol (B+E vs F in the Figure). This analysis presents the postrandomization (“real-world”) provider referral behavior.

The sensitivity analysis used 3 hypothetical fee combinations. All combinations were tested for the ITT and AT scenarios. In addition to the $25 eConsult fee charged for this study, we used $185 per visit for F2F visits and $45 per visit for eConsults. The latter 2 reimbursement rates correspond to the average commercial reimbursement rate for a 30-minute new patient F2F office consultation in the same zip code as the UCHC and a cost-based estimate for eConsults, respectively.17,18

Cost items were segregated into the categories shown in Table 1. Baseline costs were established by evaluating all claims for 180 days preceding the cardiology consult request. Cost analysis for the intervention period was based on claims inclusive of the date of the referral and the following 180 days. All claims included a 3-month lag. Extreme costs were not truncated.

All cost analyses were performed from the payer’s (Connecticut Medicaid) perspective. Transportation costs paid for by Medicaid for F2F visits were not included. At the time of the study, Medicaid did not reimburse for eConsults and therefore payment was not reflected in the claims extracts. All analyses included a $25 fee for each eConsult visit paid to the cardiologist by CHCI. Cardiology F2F new visits (Current Procedural Technology code, 99243) were reimbursed by Medicaid at their customary rate of $66. Any costs borne by PCPs (eg, additional time spent creating and reviewing eConsults), specialists (eg, lost revenue from “no-shows”), or patients (eg, co-pays, unpaid time off work, or out-of-pocket transportation costs) were not included.

Healthcare costs are typically not normally distributed (ie, they are skewed),19 resulting in the distributions of repeated cost variables being “pulled up” toward a higher mean by a few extreme scores. Several statistical paths were followed to ensure that comparisons of changes in costs between F2F and eConsult patients yielded robust results.

Baseline and intervention costs were assessed across 7 categories (inpatient, emergency department, outpatient, pharmacy, labs, cardiac procedures, and all other) for departure from normality. Then, non-normal cost changes were modeled using Mplus version 7.4 software (Muthén & Muthén; Los Angeles, California).20,21 Its skew-t estimation method allows for direct comparisons of means without the need to truncate scores, by estimating 2 parameters beyond mean and variance, namely the skewness and t degrees of freedom for extreme scores (to model “thick-tailed” distributions).22

Patient demographic characteristics and raw baseline costs were first evaluated for baseline equivalency. This was followed by analyses of differences between the non-normality—adjusted means of the cost changes (ie, change scores adjusted for baseline values).23,24 All results are reported as the test of differences in changes between cost categories from the baseline to intervention periods for the total cost. Amounts paid in 2013, 2014, and 2015 were converted to 2016 dollars. All claims categories (ie, cardiac and noncardiac) were included in the analysis.

RESULTS

Thirty-six PCPs participated in the trial; 19 were randomly assigned to the control group and 17 to the intervention group. Characteristics of the PCPs in both groups were balanced, with no statistically significant differences in age, clinical experience, gender, race/ethnicity, or primary care specialty (Table 2).

During the study period, these participating PCPs initiated 590 adult cardiology consults. Of those, 369 patients had Medicaid insurance continuously for the duration of the study and were pooled for this comparative cost analysis.

The number of Medicaid patients in each group included 235 (64%) in the F2F group and 134 (34%) in the eConsult group. A portion of this difference was accounted for by the fact that 2 providers in the intervention group dropped out of the study at the outset and 2 additional intervention providers left the health center before completion of the study. Patient demographic and clinical characteristics are shown in Table 3. There were no significant demographic differences between the 2 groups. Clinically, rates of smoking and diabetes were similar in both groups, as were average blood pressure, body mass index, cholesterol level, and composite cardiovascular risk as measured by the Framingham Risk Score.25 The average total cost of care for the 6-month period prior to the referral date was $4102 in the control group and $4667 in the intervention group (P = .650 for difference).

The Figure shows the distribution of patients and the flow of patient referrals included in this analysis. Of the 134 consults in the intervention group, 59 (44%) were sent directly for a F2F visit due to the perceived urgency of the referral or the existence of an established relationship with a cardiologist. Seventy-five consults (56%) were referred to the reviewing cardiologist. Fifty-four (72%) of these eConsults contained advice for management in primary care and a recommendation that a F2F visit was unnecessary. Nineteen (25%) of the eConsults recommended a F2F visit by the patient, of whom 10 (53%) completed a visit and 9 (47%) did not (the PCP did not order a F2F visit for 4 patients, 2 were no-shows, and the status of the 3 remaining patients was unknown). Two patients (3%) referred for an eConsult did not receive it, 1 due to technical problems and 1 for an unknown reason.

Of the 235 patients in the control group, 196 (83%) had a F2F visit with a cardiologist, 35 (15%) were not seen, and the status of 4 (2%) patients was unknown. Of the 35 patients who were not seen, 24 were no-shows (10% of those patients who were originally referred).

Table 4 shows the ITT unadjusted and adjusted means20,26 for all cost categories in both arms of the study. For 6 months following the request for the cardiology consult, patients referred by providers in the eConsult arm had a mean unadjusted total cost of care that was $652 per patient lower than that of patients referred by providers in the F2F group. After adjusting for skewness, t shape, and baseline differences, overall cost in the eConsult group was $466 per patient lower than in the F2F group.

Further analysis demonstrated that the number of claims for cardiac testing, total claims, and the total cost diverged between treatment and control groups immediately following initiation of the cardiology consult, with higher rates in the control group, suggesting that the observed differences were in fact the result of differences in utilization.

Although a portion of the cost difference between the 2 groups can be attributed to the difference in cost between an eConsult and a F2F visit ($25 vs $66 for this study), this difference accounted for only a small part of the actual observed savings. Even after applying a $66 charge to all patients in the eConsult arm, including for those not seen F2F, the savings were still significant ($433; P = .032); the AT analysis (75 patients in eConsult vs 296 in F2F) showed savings of $550 per patient (P = .084).

A sensitivity analysis further demonstrates the potential cost savings with various reimbursement rates for eConsults and F2F visits. Case scenario 1 (ITT eConsult, $45; F2F, $66.40) showed a reduction in total adjusted savings for eConsults of $450 (P = .025). In case scenario 2 (ITT eConsult, $25; F2F, $185), the adjusted savings was $557 (P = .006). In case scenario 3 (ITT eConsult $45; F2F, $185), the adjusted savings was $541 per patient (P = .007).

DISCUSSION

Inadequate access to specialty services among Medicaid beneficiaries is a well-recognized barrier to optimal health outcomes and a contributing factor to healthcare disparities.27-29 Previous studies have demonstrated that eConsults improve access by reducing referral waiting times,8,30 but until now, the economic impact of giving practicing PCPs access to a secure, efficient eConsult platform to enhance their interactions with specialists was unknown. The results of our analysis show for the first time that when PCPs are given an option to use eConsults for Medicaid beneficiaries, the total costs and the cost of outpatient cardiac tests and procedures at 6 months are significantly lower, by $466 and $81, respectively, compared with the traditional F2F approach. Although we randomized providers, rather than patients, baseline data demonstrate that patients in both PCP groups were similar in demographics, cost of care, and clinical characteristics. In addition, there were no differences between providers in the 2 treatment arms or in their sites of practice. This relatively rapid decline in cost (6 months) is unusual in health services studies. Moreover, the results suggested that, given the conservatism inherent in the ITT or “as randomized” method, the analysis may underestimate savings with eConsults compared with the “as treated” case scenario. Our secondary analysis using the as treated scenario confirmed significant savings of $93 per patient for cardiac tests and procedures and a favorable trend of $533 for overall costs. This analysis should give confidence to payers looking for innovative delivery models that reduce costs and improve access, timeliness, and convenience for patients and specialists alike.

At the outset, a hypothetical explanation for potential savings with eConsults was based on more timely initiation of a treatment plan and reduced duplication of tests and procedures. Our study was not able to elucidate the impact of considerable improvements in timeliness on cost of care, but it did demonstrate a net reduction in overall outpatient procedures. This finding is a direct result of the redesigned process itself, rather than individual provider behaviors, suggesting that this transformation is potentially durable.

Our analysis was conservative, as it only evaluated claims-related costs from the payer perspective and did not evaluate other plausible sources of cost savings. For example, many Medicaid patients receive reimbursement for transportation to F2F appointments. The claims file did not include payments related to patient transportation, but those unmeasured cost savings in the eConsult group accrued to Medicaid.

There were several additional potential cost implications to the PCP. The use of eConsults reduced the administrative work of scheduling F2F consults and coordinating F2F visits with patients, which could have staffing implications. Some safety-net health centers invest significant resources not only in scheduling specialty visits for their patients, but also in providing extra support to help patients overcome financial, transportation, and other logistical barriers to reduce the likelihood of a no-show.31

The eConsult workflow used in this project required little additional work or training on behalf of the PCP. Consults were routed via the eConsult system by a referral coordinator who was responsible for managing the consult process. Any additional work for providers reviewing and implementing eConsult treatment recommendations was likely offset by a reduction in the work required to address and manage complaints while patients were waiting for their F2F visit.

The impact of this intervention on costs to patients was also not considered in this analysis. One study from Canada has demonstrated that cost savings to patients may be significant16 due to avoided transportation costs and lost productivity and wages from taking uncompensated time off from work. These potential benefits associated with the eConsult represent unmeasured but potentially important cost savings that accrued to patients in this study.

One final cost savings to specialists (but not to payers) that was not measured in our study was the potential reduction in no-show rates in the F2F group. Reducing the number of F2F visits and only sending those patients who truly require one may also reduce rates of costly no-shows. Of the 235 patients in the F2F group, 35 (15%) patients never saw the cardiologist and 24 (10%) were confirmed no-shows. No-shows are not only costly to the specialist, but missing appointments also means forfeiting needed input on the patient’s care. This can result in costly complications later on that may have been preventable.

Limitations

This study had several limitations. The short 6-month duration of follow-up may have resulted in an inability to detect any seasonal cost variations. It is also possible that shorter-term cost savings resulted in cost increases at a later date. In addition, the focus on a single specialty precludes generalizing these findings to other specialties. Many eConsult systems provide access to a wide range of specialties for which the cost implications are unknown. Also, this evaluation only included patients with Medicaid, which precludes drawing broader conclusions on the impact of eConsults for the uninsured or for patients with Medicare or private insurance, as Medicaid costs are significantly different from those of other payers.

CONCLUSIONS

We conducted the first randomized controlled trial of eConsults for cardiology and demonstrated that they resulted in reduced total healthcare costs for Medicaid members’ care. The implications of the cost savings demonstrated in this study are important for state Medicaid agencies and other health systems seeking new ways to improve access and quality while reducing cost. Policy changes that support the use of eConsults as a new service modality could result in significant savings to the Medicaid program in a relatively short time frame. However, sustaining eConsult programs will require changes in reimbursement policies, either by authorizing payments for eConsults on a fee-for-service basis or by increasing the opportunities for primary care and specialty providers to share in the savings that accrue from more efficient and effective care. Future studies should examine the cost—benefit balance of eConsults for multiple specialties and in more diverse settings to further inform these policy changes as well as which changes in costs trigger changes in other costs. Longer follow-up will also be useful to determine the durability of savings realized in the short term.Author Affiliations: Weitzman Institute (DA, IZ, JV), Middletown, CT; UConn Health Disparities Institute, University of Connecticut (VV, EC), Farmington, CT; RPM Health (AH), Farmington, CT; Center for Excellence in Primary Care, University of California, San Francisco (JNO), San Francisco, CA.

Source of Funding: Connecticut Health Foundation.

Author Disclosures: Dr Villagra is president of Health & Technology Vector, which signed a contract with CeCN, an eConsult company, after completion of this work. Dr Olayiwola began work for an eConsult company in March 2017, well after this work was completed. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (DA, IZ, JV, JNO); acquisition of data (DA, VV); analysis and interpretation of data (DA, VV, EC, AH, JV); drafting of the manuscript (DA, VV, EC, IZ, AH); critical revision of the manuscript for important intellectual content (DA, VV, EC, IZ, AH, JV, JNO); statistical analysis (DA, VV, EC, AH); provision of patients or study materials (DA); obtaining funding (DA, JNO); administrative, technical, or logistic support (DA, JNO); and supervision (DA).

Address Correspondence to: Daren Anderson, MD, Weitzman Institute, 631 Main St, Middletown, CT 06457. Email: daren@chc1.com.REFERENCES

1. Yee HF Jr. The patient-centered medical neighbor: a subspecialty physician’s view. Ann Intern Med. 2011;154(1):63-64. doi: 10.7326/0003-4819-154-1-201101040-00011.

2. Hollingsworth JM, Saint S, Hayward RA, Rogers MA, Zhang L, Miller DC. Specialty care and the patient-centered medical home. Med Care. 2011;49(1):4-9. doi: 10.1097/MLR.0b013e3181f537b0.

3. Pham HH. Good neighbors: how will the patient-centered medical home relate to the rest of the health-care delivery system? J Gen Intern Med. 2010;25(6):630-634. doi: 10.1007/s11606-009-1208-1.

4. Starfield B, Chang HY, Lemke KW, Weiner JP. Ambulatory specialist use by nonhospitalized patients in US health plans: correlates and consequences. J Ambul Care Manage. 2009;32(3):216-225. doi: 10.1097/JAC.0b013e3181ac9ca2.

5. Barnett ML, Song Z, Landon BE. Trends in physician referrals in the United States, 1999-2009. Arch Intern Med. 2012;172(2):163-170. doi: 10.1001/archinternmed.2011.722.

6. Pham HH, O’Malley AS, Bach PB, Saiontz-Martinez C, Schrag D. Primary care physicians’ links to other physicians through Medicare patients: the scope of care coordination. Ann Intern Med. 2009;150(4):236-242.

7. Chen AH, Kushel MB, Grumbach K, Yee HF Jr. A safety-net system gains efficiencies through ‘eReferrals’ to specialists. Health Aff (Millwood). 2010;29(5):969-971. doi: 10.1377/hlthaff.2010.0027.

8. Chen AH, Murphy EJ, Yee HF Jr. eReferral--a new model for integrated care. N Engl J Med. 2013;368(26):2450-2453. doi: 10.1056/NEJMp1215594.

9. Kim-Hwang JE, Chen AH, Bell DS, Guzman D, Yee HF Jr, Kushel MB. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med. 2010;25(10):1123-1128. doi: 10.1007/s11606-010-1402-1.

10. Barnett ML, Yee HF Jr, Mehrotra A, Giboney P. Los Angeles safety-net program eConsult system was rapidly adopted and decreased wait times to see specialists. Health Aff (Millwood). 2017:36(3):492-499. doi: 10.1377/hlthaff.2016.1283.

11. Kim Y, Chen AH, Keith E, Yee HF Jr, Kushel MB. Not perfect, but better: primary care providers’ experiences with electronic referrals in a safety net health system. J Gen Intern Med. 2009;24(5):614-619. doi: 10.1007/s11606-009-0955-3.

12. Scherpbier-de Haan ND, van Gelder VA, Van Weel C, Vervoort GM, Wetzels JF, de Grauw WJ. Initial implementation of a web-based consultation process for patients with chronic kidney disease. Ann Fam Med. 2013;11(2):151-156. doi: 10.1370/afm.1494.

13. Palen TE, Price D, Shetterly S, Wallace KB. Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study. BMC Med Inform Decis Mak. 2012;12:65. doi: 10.1186/1472-6947-12-65.

14. Olayiwola JN, Anderson D, Jepeal N, et al. Electronic consultations to improve the primary care-specialty care interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med. 2016;14(2):133-140. doi: 10.1370/afm.1869.

15. Liddy C, Deri Armstrong C, Dronsinis P, Mito-Yobo F, Afkham A, Keely E. What are the costs of improving access to specialists through eConsultation? the Champlain BASE experience. Stud Health Technol Inform. 2015;209:67-74. doi: 10.3233/978-1-61499-505-0-67.

16. Liddy C, Drosinis P, Deri Armstrong C, McKellips F, Afkham A, Keely E. What are the cost savings associated with providing access to specialist care through the Champlain BASE eConsult service? a costing evaluation. BMJ Open. 2016;6(6):e010920. doi: 10.1136/bmjopen-2015-010920.

17. Office visit, new patient (~30 min.). Healthcare Bluebook website. healthcarebluebook.com/page_ProcedureDetails.aspx?id=220&dataset=MD&g=Office+Visit%2c+New+Patient+(~30+min. Updated 2016.

18. FH medical cost lookup: get started. Fair Health Consumer website. fairhealthconsumer.org/estimate-costs/step-1. Updated 2016.

19. Loisel P, Lemaire J, Poitras S, et al. Cost-benefit and cost-effectiveness analysis of a disability prevention model for back pain management: a six year follow up study. Occup Environ Med. 2002;59(12):807-815.

20. Lee SX, McLachlan GJ. On mixtures of skew normal and skew t-distributions. Adv Data Anal Classif. 2013;7(3):241-266.

21. Lee S, McLachlan GJ. Finite mixtures of multivariate skew t-distributions: some recent and new results. Stat Comput. 2014;24(2):181-202.

22. Asparouhov T, Muthén B. Structural equation models and mixture models with continuous nonnormal skewed distributions. Struct Equ Modeling. 2016;23(1):1-19. doi: 10.1080/10705511.2014.947375.

23. Ferrer E, McArdle J. Alternative structural models for multivariate longitudinal data analysis. Struct Equ Modeling. 2003;10(4):493-524. doi: 10.1207/S15328007SEM1004_1.

24. McArdle JJ. Latent variable modeling of differences and changes with longitudinal data. Annu Rev Psychol. 2009;60:577-605. doi: 10.1146/annurev.psych.60.110707.163612.

25. Anderson KM, Odell PM, Wilson PW, Kannel WB. Cardiovascular disease risk profiles. Am Heart J. 1991;121(1 pt 2):293-298.

26. Azzalini A, Dalla Valle A. The multivariate skew-normal distribution. Biometrika. 1996;83(4):715-726.

27. Shields HM, Stoffel EM, Chung DC, et al. Disparities in evaluation of patients with rectal bleeding 40 years and older. Clin Gastroenterol Hepatol. 2014;12(4):669-675. doi: 10.1016/j.cgh.2013.07.008.

28. Rhodes KV, Bisgaier J, Lawson CC, Soglin D, Krug S, Van Haitsma M. “Patients who can’t get an appointment go to the ER”: access to specialty care for publicly insured children. Ann Emerg Med. 2013;61(4):394-403. doi: 10.1016/j.annemergmed.2012.10.030.

29. Cunningham PJ, O’Malley AS. Do reimbursement delays discourage Medicaid participation by physicians? Health Aff (Millwood). 2009;28(1):w17-w28. doi: 10.1377/hlthaff.28.1.w17.

30. Chen AH, Yee HF Jr. Improving primary care-specialty care communication: lessons from San Francisco’s safety net: comment on “Referral and consultation communication between primary care and specialist physicians”. Arch Intern Med. 2011;171(1):65-67. doi: 10.1001/archinternmed.2010.484.

31. Spatz ES, Phipps MS, Lagarde S, et al. Project Access—New Haven: improving access to specialty care for patients without insurance. Conn Med. 2011;75(6):349-354.