WEBVTT

NOTE week06-clip3-linear-probing
NOTE Narration read by an AI-generated voice.
NOTE The clip draws its own caption along the bottom of the frame. These cues
NOTE sit there too, so that no part of the animation is ever covered.

1
00:00:00.150 --> 00:00:05.267
Now we test what pretraining learned.
Take the pretrained encoder and freeze it,

2
00:00:05.267 --> 00:00:08.150
so none of its weights can change.

3
00:00:08.350 --> 00:00:11.670
We need only a small labeled dataset.

4
00:00:11.670 --> 00:00:19.483
Here, each ECG is labeled either atrial
fibrillation, AF, or normal sinus rhythm.

5
00:00:19.683 --> 00:00:25.109
The frozen encoder maps each ECG
to an embedding of 256 numbers.

6
00:00:25.109 --> 00:00:29.883
We draw these embeddings as
points in two dimensions.

7
00:00:30.083 --> 00:00:33.547
On top, we add a single
linear prediction head.

8
00:00:33.547 --> 00:00:37.675
It computes a weighted sum
of the embedding plus a bias,

9
00:00:37.675 --> 00:00:42.217
and a sigmoid turns that
into the probability of AF.

10
00:00:42.417 --> 00:00:47.437
During training, gradients flow into
the head only. The encoder is frozen,

11
00:00:47.437 --> 00:00:50.017
so its weights never change.

12
00:00:50.217 --> 00:00:53.855
If one straight line can
separate AF from normal,

13
00:00:53.855 --> 00:00:57.086
the information was
already in the embedding.

14
00:00:57.086 --> 00:01:00.417
The head only has to read it out.

15
00:01:00.617 --> 00:01:05.042
With fine-tuning, every weight is
trainable, including the encoder.

16
00:01:05.042 --> 00:01:08.984
A linear probe trains only 257 weights,

17
00:01:08.984 --> 00:01:14.083
so it measures what the pretrained
encoder already learned.

18
00:01:14.283 --> 00:01:20.613
The same linear probe works for pathology.
A frozen encoder embeds each H&E tile,

19
00:01:20.613 --> 00:01:25.413
and a single linear head separates
tumor from normal mucosa.