WEBVTT

NOTE week05-clip4-reasoning
NOTE Narration read by an AI-generated voice.
NOTE The clip draws its own caption along the bottom of the frame. These cues
NOTE sit there too, so that no part of the animation is ever covered.

1
00:00:00.150 --> 00:00:06.428
A reasoning model writes out its thinking
before it answers. For this ECG question,

2
00:00:06.428 --> 00:00:13.130
it reasons that ST elevation in leads II,
III, and aVF points to the inferior wall,

3
00:00:13.130 --> 00:00:18.550
usually supplied by the right
coronary artery, and answers B.

4
00:00:18.750 --> 00:00:24.753
DeepSeek-R1 learned this with rewards
that a simple program can check:

5
00:00:24.753 --> 00:00:27.728
one point if the final answer is correct,

6
00:00:27.728 --> 00:00:32.107
and one point if the reasoning
sits inside the think tags.

7
00:00:32.107 --> 00:00:35.533
R1-Zero used reinforcement learning alone.

8
00:00:35.533 --> 00:00:40.750
R1 first saw a small set
of long reasoning examples.

9
00:00:40.950 --> 00:00:44.975
The training method is called
GRPO. For each question,

10
00:00:44.975 --> 00:00:49.096
the model writes a group of
answers, and each one is scored.

11
00:00:49.096 --> 00:00:54.683
Here, only the first of four
answers picks the right artery.

12
00:00:54.883 --> 00:01:00.675
Each reward is compared with its
own group. The mean reward is 0.25,

13
00:01:00.675 --> 00:01:04.671
and the standard deviation is 0.433.

14
00:01:04.671 --> 00:01:08.725
So the correct answer scores plus 1.73,

15
00:01:08.725 --> 00:01:15.883
and each wrong answer scores minus 0.58.
No separate critic model is needed.

16
00:01:16.083 --> 00:01:18.386
The model is then updated

17
00:01:18.386 --> 00:01:22.517
so that answers above their
group's average become more likely,

18
00:01:22.517 --> 00:01:26.976
and answers below it become less
likely. Over many questions,

19
00:01:26.976 --> 00:01:31.550
reasoning that reaches
correct answers is reinforced.

20
00:01:31.750 --> 00:01:35.362
Thinking can also help at
test time. With thinking off,

21
00:01:35.362 --> 00:01:40.977
the small Qwen3 model gets the chest
pain question wrong in nine tokens.

22
00:01:40.977 --> 00:01:48.217
With thinking on, it writes 119 tokens and
gets it right, but it takes much longer.

23
00:01:48.417 --> 00:01:54.190
Med-PaLM 2 samples several reasoning
paths for a medical question,

24
00:01:54.190 --> 00:01:58.093
then reads them all to
write one refined answer.

25
00:01:58.093 --> 00:02:03.053
Together with a stronger base
model and medical fine-tuning,

26
00:02:03.053 --> 00:02:10.350
this raised MedQA accuracy
from 67.2 to 86.5%.

27
00:02:10.550 --> 00:02:14.452
But a fluent explanation is
not proof. On the ECG question,

28
00:02:14.452 --> 00:02:18.727
the same small model picks the
right artery with thinking off.

29
00:02:18.727 --> 00:02:22.588
With thinking on, it argues
confidently for the wrong artery,

30
00:02:22.588 --> 00:02:27.018
and after 1,500 tokens it is still unsure.

31
00:02:27.018 --> 00:02:30.913
We return to checking
reasoning in week 12.