Learning to (Learn at Test Time): RNNs with Expressive Hidden States

Paper here: arxiv.org/abs/2407.04620
Code!: github.com/test-time-training/ttt-lm-pytorch

Notes: drive.google.com/file/d/127a1UBm_IN_WMKG-DmEvfJ8Pj…

00:00 Intro
04:40 Problem with RNNs
06:38 Meta learning and method idea
09:13 Update rule and RNN inner loop
15:07 Learning the loss function outer loop
21:21 Parallelizing training
30:05 Results