Though symptom checking apps and websites have been touted by some as the future of clinical diagnostics, a recent study has found that they are far less accurate than actual physicians at identifying the correct diagnosis when given a clinical vignette.
Though symptom checking apps and websites have been touted by some as the future of clinical diagnostics, a recent study has found that they are far less accurate than actual physicians at identifying the correct diagnosis when given a clinical vignette. The results of the study, published in a research letter in JAMA, compared the performance of physicians and online symptom checkers in diagnosing hypothetical clinical situations.
Lead author Ateev Mehrotra, MD, MPH, of Harvard Medical School, and colleagues had previously assessed the accuracy of 23 web- and app-based symptom checkers in a 2015 BMJ study, finding that the computer algorithms identified the correct diagnosis just 34% of the time. The 45 clinical vignettes used to test these symptom checkers provided the patient’s medical history and symptom presentation, but did not include physical examination or test findings.
In this study, the researchers distributed the same 45 clinical vignettes to physicians participating in Human Dx, an online platform where doctors generate differential diagnoses based on such vignettes. The doctors did not know which cases were part of this research study.
The 234 physicians who assessed the study vignettes determined the correct diagnosis first 72.1% of the time, which was more than double the accuracy rate of the symptom checkers. They were also significantly more likely (84.3% vs 51.2%) to have listed the correct condition in their top 3 diagnoses. Diagnosis accuracy was similar whether the physician was an attending, a fellow or resident, or an intern.
Accuracy also varied based on the acuity and prevalence of the conditions described in the vignettes. Physicians had an accuracy rate of 79.1% when diagnosing high-acuity situations for which urgent care would be required, compared to the 24.3% of symptom checkers that listed the correct diagnosis for these emergent conditions. In addition, 75.5% of physicians correctly identified an uncommon condition (defined as diagnoses that accounted for less than 0.3% of ambulatory visits in the US in 2009 to 2010) as opposed to 28.1% of symptom checking apps or websites.
The study authors acknowledged that although the symptom checkers had much lower accuracy rates, future research might examine whether these technologies could help support the diagnosis of a physician instead of replacing it.
Emily Goldberg Shares Insights as a Genetic Counselor for Breast Cancer Risk Screening
October 30th 2023On this episode of Managed Care Cast, Emily Goldberg, MS, CGC, a genetic counselor at JScreen, breaks down how genetic screening for breast cancer works and why it is so important to increase awareness and education around these screening tools available to patients who may be at risk for cancer.
Listen
Examining Telehealth Uptake to Increase Equitable Care Access
January 26th 2023To mark the publication of The American Journal of Managed Care®’s 12th annual health IT issue, on this episode of Managed Care Cast, we speak with Christopher M. Whaley, PhD, health care economist at the RAND Corporation, who focuses on health economics issues, including the influence of the COVID-19 pandemic on health care delivery.
Listen
Beyond Insulin: The Impact of Next-Generation Diabetes Technology
April 17th 2024Experts explain how new diabetes technologies like continuous glucose monitors are transforming care beyond intensive insulin therapy, offering personalized insights and improving outcomes for patients of all treatment levels.
Read More
Need for Pharmacogenomic Testing in Mental Health Care Explored at AMCP 2024
April 17th 2024Presenters from Mayo Clinic discussed the benefits of implementing pharmacogenomic testing in mental health care for the betterment of patient outcomes, reducing medical costs, and more at the Academy of Managed Care Pharmacy (AMCP) 2024 annual meeting.
Read More