Week 2: Transformer review, Part 1: attention and the encoder

Slides 2-1

Code Notebook · Open in Colab

Papers for Thursday

• Devlin et al.: BERT
• Yang et al.: GatorTron

Animation clips

Each clip comes in two versions. The narrated version has an AI-generated voice and captions; turn them on with the CC button. The silent version is the same animation with its on-screen captions only.

Everything for this week

The Week 2 folder in the repository ↗, with the animation sources, the slides and the session guide.