Will Shapiro, vice president of data science at Flatiron Health, discusses how the practice is using large language models (LLMs) to increase efficiency, while ensuring there are frameworks set for safety and quality.
Machine learning (ML) is a very powerful artificial intelligence (AI) tool, and there needs to be guidelines to ensure patient safety and quality of care, says Will Shapiro, vice president of data science at Flatiron Health.
Transcript
How is Flatiron Health using AI in oncology care?
We're very focused on what the use case is. One of the things that that my team does a lot of is use and build machine learning algorithms to read through unstructured documents, because there's an enormous amount of really valuable information that isn't captured in a structured way routinely. Things like a patient's stage, biomarker status, even diagnosis date; these are really critical variables, especially for researchers. That traditionally has always had to be manually extracted.
Our mission as a company, part of it is to learn from the experience of every person with cancer. And that's something that really motivates me every day. Machine learning is a very powerful tool in that context.
For the task of using ML to read patient charts, we have a pretty lengthy evaluation framework that we've developed that looks at things like bias across many different strata. So, age, race, gender, ethnicity, location. But then, also says, “Okay, if we've generated something with ML, and we've generated it using humans, do they replicate in the same way when you're asking an analytic question or a research question?”
On the other hand, at the at the point of care, we've started to use machine learning more recently. So, one of the things I'm really excited about is we have a new feature that uses LLMs [large language models] to map regimens, authored by practices to NCCN [National Comprehensive Cancer Network] guidelines. And that has a totally different framework for thinking about evaluation.
One of the things this gets at is that with this really powerful new generation of LLMs and gen-AI [generative AI] tools that can do so many different things, you really need to think about what the use case is, and then think about what the appropriate evaluation framework for safety and quality within that specific context is, because it might be very different. It's very different to suggest a list of options than to say, “Here's what you should do.” Those have radically different implications. And the way that the output of an algorithm is contextualized within someone's workflow is a big part of this. And being told to do something is different than being presented with an array of options and the evidence that supported each of them.
Dr Kathy Zackowski Discusses the Importance of Rehabilitation Research and Trials in MS
April 26th 2024Kathy Zackowski, PhD, National MS Society, expresses the inherent value of quality rehabilitation trials for broadening clinical understandings of multiple sclerosis (MS) and bettering patient outcomes.
Read More
Examining Low-Value Cancer Care Trends Amidst the COVID-19 Pandemic
April 25th 2024On this episode of Managed Care Cast, we're talking with the authors of a study published in the April 2024 issue of The American Journal of Managed Care® about their findings on the rates of low-value cancer care services throughout the COVID-19 pandemic.
Listen
Empowering Community Health Through Wellness and Faith
April 23rd 2024To help celebrate and recognize National Minority Health Month, we are bringing you a special month-long podcast series with our Strategic Alliance Partner, UPMC Health Plan. In the third episode, Camille Clarke-Smith, EdD, MS, CHES, CPT, discusses approaching community health holistically through spiritual and community engagement.
Listen
HOPE-CAT Can Identify Maternal Cardiovascular Risk 2 Months Earlier Than Doctors, Study Says
April 25th 2024In a retrospective study, the machine learning tool was able to screen for potential risks of cardiovascular disease nearly 60 days before the patient's medical record showed any signs of a related condition or before they were officially diagnosed or treated for it.
Read More
Dr Kathy Zackowski Discusses the Importance of Rehabilitation Research and Trials in MS
April 26th 2024Kathy Zackowski, PhD, National MS Society, expresses the inherent value of quality rehabilitation trials for broadening clinical understandings of multiple sclerosis (MS) and bettering patient outcomes.
Read More
Examining Low-Value Cancer Care Trends Amidst the COVID-19 Pandemic
April 25th 2024On this episode of Managed Care Cast, we're talking with the authors of a study published in the April 2024 issue of The American Journal of Managed Care® about their findings on the rates of low-value cancer care services throughout the COVID-19 pandemic.
Listen
Empowering Community Health Through Wellness and Faith
April 23rd 2024To help celebrate and recognize National Minority Health Month, we are bringing you a special month-long podcast series with our Strategic Alliance Partner, UPMC Health Plan. In the third episode, Camille Clarke-Smith, EdD, MS, CHES, CPT, discusses approaching community health holistically through spiritual and community engagement.
Listen
HOPE-CAT Can Identify Maternal Cardiovascular Risk 2 Months Earlier Than Doctors, Study Says
April 25th 2024In a retrospective study, the machine learning tool was able to screen for potential risks of cardiovascular disease nearly 60 days before the patient's medical record showed any signs of a related condition or before they were officially diagnosed or treated for it.
Read More
2 Commerce Drive
Cranbury, NJ 08512