WEBVTT

NOTE week03-clip2-causal-mask
NOTE Narration read by an AI-generated voice.
NOTE The clip draws its own caption along the bottom of the frame. These cues
NOTE sit there too, so that no part of the animation is ever covered.

1
00:00:00.150 --> 00:00:06.205
Here are the attention scores for five
tokens: the patient denies chest pain.

2
00:00:06.205 --> 00:00:11.483
Each row is one token's query,
and each column is a key.

3
00:00:11.683 --> 00:00:15.104
But a decoder is trained
to predict the next token.

4
00:00:15.104 --> 00:00:22.150
If patient could look at denies, chest,
and pain, it would simply copy the answer.

5
00:00:22.350 --> 00:00:27.738
The causal mask fixes this.
Every score above the diagonal,

6
00:00:27.738 --> 00:00:34.150
where a query would look at a later
token, is set to minus infinity.

7
00:00:34.350 --> 00:00:37.949
After the softmax, minus
infinity becomes zero.

8
00:00:37.949 --> 00:00:42.540
Each row now sums to one over
the tokens it is allowed to see:

9
00:00:42.540 --> 00:00:45.750
itself and the tokens before it.

10
00:00:45.950 --> 00:00:52.631
For example, the row for chest spreads
its attention over the, patient, denies,

11
00:00:52.631 --> 00:00:57.083
and chest itself, and
gives nothing to pain.

12
00:00:57.283 --> 00:01:02.853
Because of the mask, a single forward
pass trains every position at once:

13
00:01:02.853 --> 00:01:07.747
each token's output is used to
predict the token that follows it.