These models are already seeing patients - Dr Arun Thirunavukarasu, LLMs in Medicine
In this episode of Tech Anatomy, Anca Petre talks with Dr Arun Thirunavukarasu, academic ophthalmology trainee in London and honorary research fellow at the London School of Hygiene and Tropical Medicine. As a medical student at Cambridge, he was the first author of "Large language models in medicine", the Nature Medicine review published in July 2023 that has since been cited thousands of times and became the go-to primer for clinicians. Today he works on the rules we need to evaluate medical chatbots. Together, they look at what large language models can really do in medicine, why passing a medical exam proves so little, and how doctors can keep their judgment as AI enters their toolbox.
Topics covered:
What a large language model is, explained to a clinician, and why its results are "impressive but mixed".
Why a model that passes medical exams is not ready to practise, and why we need pragmatic, randomised research.
CHART, a reporting guideline to make chatbot studies more rigorous and transparent.
True or False: medical boards, hallucinations and model size, open source models, and the clinicians of tomorrow.
AI in ophthalmology today, from diabetic retinopathy screening to the limits of chatbots on eye images.
Never-skilling: when trainees delegate their thinking to AI and never acquire the skills.
Autonomous AI consultations, and what medicine can learn from self-driving cars.
Chapters:
0:00 Teaser
0:19 Introduction
1:05 What is a large language model?
2:24 Impressive but mixed
4:05 Putting GPT-4 through medical exams
7:17 How should we evaluate these models?
11:35 CHART, a guideline for chatbot studies
14:06 True or False
18:18 AI in ophthalmology today
20:20 Why chatbots struggle with eye images
22:02 The never-skilling problem
23:08 How Arun got into AI
24:10 Rethinking medical training
26:25 Autonomous AI consultations
29:13 Rapid questions
Credits:
Production: MedShake Studio
Host: Anca Petre
Subscribe to the Tech Anatomy newsletter:
https://www.linkedin.com/build-relation/newsletter-follow?entityUrn=7117418963067658241
Follow Anca on LinkedIn for more tips and free resources:
https://www.linkedin.com/in/ancapetre/
WhatsApp group, AI in health tips:
https://chat.whatsapp.com/CMdYp3pfcRrBX9Qxo2sKAz
Hosted on Ausha. See ausha.co/privacy-policy for more information.