Why the same patient can't be in both piles

The best score one of my models ever posted came from a single fold of a cross-validation run, in a Colab notebook that had been going most of the afternoon. My first reaction wasn't excitement. I went looking to see whether the same patient showed up in both the training and test sets.

Think of a study guide. Before an exam the teacher hands one out. If the guide and the exam are different, a high score suggests the student learned the material. If the guide is the exam, a high score tells you almost nothing. The student may have learned the subject or may have memorized twenty answers, and the score can't say which.

Training a model works the same way. You split the data into a training set and a test set, train on the first, and measure on the second. The measurement only means something if the test set is data the model never saw. When information from the test set leaks into training, the score gets inflated, and nothing in the score itself tells you.

In medical imaging the duplicate is usually not a file. It's a person. A hospital patient often has more than one chest X-ray, say one at admission and another a week later to see how treatment is going. Those are two files with two names, but they're the same lungs, the same ribs, the same scanner, and often the same shadow in the same spot.

Put one of those X-rays in training and the other in testing, and you're asking the model to recognize a patient it has already met. It doesn't have to learn what tuberculosis looks like. It only has to remember those lungs. The score goes up, everyone feels good, and none of it carries over to the next patient who walks in.

The fix is to split by patient and not by X-ray, so every image from one patient lands on the same side. In my lab we go a step further and use patient-level five-fold cross-validation. Patients are divided into five groups. The model trains on four groups and is tested on the fifth, and that repeats five times, so every patient is tested once and never by a model that saw them during training. We average the five scores, and the spread between them tells us how stable the model is.

I tested this on one of my own projects. I trained the same models under two protocols, one splitting by X-ray and one splitting by patient, and compared them with two statistical tests. McNemar's test asks whether two models disagree on individual cases more often than chance would explain. DeLong's test asks the same thing about AUC, the ranking measure from the last post. On the dataset in that project the two protocols did not differ in a statistically meaningful way (p-values of 0.917 and 0.450). In plain terms, we couldn't detect a difference.

That surprised me, and I want to be honest about what it does and doesn't show. In this dataset, with this many patients and this many repeat scans per patient, leakage didn't produce a measurable boost. It does not mean leakage isn't a problem. A dataset with more repeat scans per patient could behave completely differently, and there's no way to know in advance.

So my rule hasn't changed. A suspiciously high score on one fold is leakage until shown otherwise, and the burden of proof is on the score.

The costs are lopsided, too. If I decide a score is too good and I'm wrong, I lose an evening redoing a split. If I decide a leaked score is clean, it can end up in a paper and maybe a product, and the first people to find out are patients at a hospital that never saw the paper.

Regulators have said something close to this. In October 2021 the FDA, Health Canada and the UK's MHRA published ten principles for Good Machine Learning Practice. Principle four is "Training Data Sets Are Independent of Test Sets." It's the right principle, but it's a principle and not a requirement. A developer can agree with it and still split by image, because it doesn't say what "independent" means for a patient with four X-rays.

There's evidence this isn't enforced. In the same 2021 Nature Medicine study, Wu and colleagues reviewed public reports for 130 AI devices cleared between 2015 and 2020. For 59 of them (45%), the size of the test set wasn't disclosed. Where it was, the median was 300. From a public report alone you can't tell whether 300 means 300 patients or 300 images from 80 patients. That is exactly the question.

So here's my request. A clearance summary should describe how the test set was built. Was the split made at the patient level, and how many unique patients were in the test set, not just how many images? Which sites did the test images come from, and did any of those sites also supply training images? When were the test images collected, so someone can tell whether they came after the model was frozen? And has anyone other than the developer had access to them?

That doesn't expose a trade secret. A team that did this properly already has the answers written down in a configuration file somewhere.

I'm not asking regulators to rerun anyone's experiments. I'm asking for one paragraph that lets someone else judge whether the experiment was real. The score is the easy part. The split is what makes it worth anything.