Medical AI in plain language, with the error bars left in.
Each post takes one decision from inside a chest X-ray model and turns it into the question a hospital, a regulator, or a patient should be asking.
The instrument · what "99% accurate" means on your patients
Read the postEvery alarm a screening tool raises is either a real case or a false one. The split depends less on the model than on how rare the disease is among the people screened. Move the sliders.
Screen 100,000 people at this rate. About 3 have the disease. The tool flags 1,003: 3 real, 1,000 false alarms.
At this sensitivity it misses almost no one who is sick.
Real cases caught = rate × sensitivity. False alarms = everyone else × (1 − specificity). The share of alarms that are real is the positive predictive value. Rates from CDC (US, 2025) and WHO (global, 2024).
Uncertain Terms takes the decisions inside medical AI and translates them into questions a hospital administrator, a regulator, a legislator or a patient can ask. It argues that the rules should require a model to say how sure it is and to admit the limits of what it has seen.
Writing
Sources at the bottom of every post; what I'm unsure about in the margin.
Why the same patient can't be in both piles
If one patient appears in both the training and test sets, the score measures memory instead of skill, so a clearance summary should say how the split was made.
The model that learned the X-ray machine
Neural networks latch onto whatever signal is easiest to use, and in pooled hospital data that signal is often the machine and not the disease.
What it means for a model to say how sure it is
Modern neural networks tend to be overconfident. A hospital should ask for a calibration curve that shows whether the confidence reported by the model actually means anything.
Podcast
Ten to twelve minutes, one voice, one question per episode.
-
02
The weather forecaster
Why a weather application's seventy percent is reliable, why a deep learning model's ninety percent is often not, and what a calibration curve would tell a hospital before it makes a purchase.
Oct 24, 202610 min -
01
The number on the brochure
A one-page brochure at a conference booth puts ninety-nine percent accurate in the largest type on the page and Vinay works out slowly and out loud why that number is the least useful true thing about the tool.
Oct 3, 202610 min
Why this exists
I ran one of my models on chest X-rays from a hospital that wasn't in my training data, and the accuracy went from above 99% to somewhere in the sixties. There were two columns in a spreadsheet. The first was the kind of number a vendor might feature in a brochure. The second looked like a coin that had been tilted slightly.
Then came a long debate over which column to use in the paper. Both were true. The 99% was the model's performance on data similar to what it had been trained on. The sixties were its performance on strangers. I concluded that a hospital only ever meets strangers.
Contact
Questions from a hospital, a newsroom, a classroom, or a legislator's office are welcome. If a post gets something wrong, say so; corrections are noted in place with a date.
Guests and press
Guest requests for the podcast (clinicians, regulators, and people who buy these tools for a living) are especially welcome. Episodes are one question long.
Based in Coral Springs, Florida. Research at Florida Atlantic University, Boca Raton.