WEBVTT

NOTE week09-clip3-compounding
NOTE Narration read by an AI-generated voice.
NOTE The clip draws its own caption along the bottom of the frame. These cues
NOTE sit there too, so that no part of the animation is ever covered.

1
00:00:00.150 --> 00:00:04.877
Every step of an agent is a chance
to fail. It can pick the wrong tool,

2
00:00:04.877 --> 00:00:08.695
pass the wrong patient
number, misread a result,

3
00:00:08.695 --> 00:00:11.122
or repeat the same call again and again.

4
00:00:11.122 --> 00:00:16.350
In ReAct, most failures were
reasoning errors like these.

5
00:00:16.550 --> 00:00:22.472
Suppose each step succeeds 95% of the
time, and steps fail independently.

6
00:00:22.472 --> 00:00:27.475
Then a task of n steps succeeds
with 0.95 to the power n:

7
00:00:27.475 --> 00:00:30.191
0.77 for 5 steps,

8
00:00:30.191 --> 00:00:35.683
0.6 for 10, and only 0.36 for 20.

9
00:00:35.883 --> 00:00:42.719
Raise each step to 99%, and 10
steps succeed 90% of the time,

10
00:00:42.719 --> 00:00:49.617
20 steps 82%. A small gain per step
makes a large difference for long tasks.

11
00:00:49.817 --> 00:00:54.732
Mitigations start with the tools.
The agent checks its own results,

12
00:00:54.732 --> 00:00:59.940
for example by re-reading the lab
table, and a mismatch stops the task.

13
00:00:59.940 --> 00:01:04.342
It gets only read-only tools,
such as get labs and get meds,

14
00:01:04.342 --> 00:01:08.483
never one that places
orders or sends messages.

15
00:01:08.683 --> 00:01:14.135
Any action that changes care, such as
an order or a message to a patient,

16
00:01:14.135 --> 00:01:20.712
stops at a checkpoint. The agent proposes
it, here a repeat potassium and an ECG,

17
00:01:20.712 --> 00:01:25.617
and a clinician approves or
rejects it before anything happens.

18
00:01:25.817 --> 00:01:29.801
To sum up: retrieval
grounds answers in sources,

19
00:01:29.801 --> 00:01:34.399
tools let the model act on real
data through text it writes,

20
00:01:34.399 --> 00:01:40.490
and every added step is another chance to
fail. So check results, limit the tools,

21
00:01:40.490 --> 00:01:43.113
and ask a clinician.