WEBVTT

NOTE week12-clip3-shortcuts
NOTE Narration read by an AI-generated voice.
NOTE The clip draws its own caption along the bottom of the frame. These cues
NOTE sit there too, so that no part of the animation is ever covered.

1
00:00:00.150 --> 00:00:03.613
A model can reach a high score
by reading the wrong thing.

2
00:00:03.613 --> 00:00:06.645
The score tells you the
label came out right.

3
00:00:06.645 --> 00:00:11.083
It does not tell you which
part of the image produced it.

4
00:00:11.283 --> 00:00:14.240
DeGrave and colleagues built two datasets.

5
00:00:14.240 --> 00:00:17.392
The first takes positives
from GitHub-COVID

6
00:00:17.392 --> 00:00:20.664
and negatives from ChestX-ray14,

7
00:00:20.664 --> 00:00:24.274
so source and label
line up almost exactly.

8
00:00:24.274 --> 00:00:29.504
The second, from BIMCV and PadChest,
is one region and one research team:

9
00:00:29.504 --> 00:00:31.750
the favorable case.

10
00:00:31.950 --> 00:00:33.910
Saliency maps and generative

11
00:00:33.910 --> 00:00:36.841
counterfactuals showed
what the models used:

12
00:00:36.841 --> 00:00:41.614
laterality markers, arrows and
text found only in one archive,

13
00:00:41.614 --> 00:00:45.973
the radiopacity of the image
borders, and patient positioning.

14
00:00:45.973 --> 00:00:48.746
Moving the clavicle
region to the top border

15
00:00:48.746 --> 00:00:52.817
raised the predicted odds of COVID 19.

16
00:00:53.017 --> 00:00:55.662
On radiographs from a new hospital system,

17
00:00:55.662 --> 00:01:00.608
half of the predictive performance
was lost. The authors concluded

18
00:01:00.608 --> 00:01:05.739
that only a minority of it was
ever genuine COVID 19 pathology.

19
00:01:05.739 --> 00:01:09.350
The rest was the source, not the disease.

20
00:01:09.550 --> 00:01:15.309
Here is the part that catches people
out. Some shortcuts do not transfer,

21
00:01:15.309 --> 00:01:19.827
so the score falls and you see it.
Others, like patient positioning,

22
00:01:19.827 --> 00:01:22.191
are consistent in both datasets:

23
00:01:22.191 --> 00:01:27.398
the score holds up and the model
is still not reading the lungs.

24
00:01:27.398 --> 00:01:31.150
External validation is
necessary, not sufficient.

25
00:01:31.350 --> 00:01:34.827
An average hides the group
a model works worst on.

26
00:01:34.827 --> 00:01:39.872
Sensitivity of 0.85 overall can be 0.90 in

27
00:01:39.872 --> 00:01:45.617
80% of patients and 0.65 in the other 20.

28
00:01:45.617 --> 00:01:50.718
Report subgroups with intervals, validate
externally, look at what the model uses,

29
00:01:50.718 --> 00:01:55.580
and test whether the site can
be predicted from its features.