Experience Grounds Language: Improving language models beyond the world of text

Now that language models have been trained on massive internet-scale text data, where are future improvements going to come from? Jay goes over the "Experience Grounds Language" paper which describes five "World Scopes" for learning language -- including multimodality (e.g. training on images + text) and beyond.


Contents:
Introduction (0:00)
Experience Grounds Language (1:20)
World Scopes (2:58)
World Scope 1 and 2 (3:33)
World Scope 3 - Multimodality (3:56)
World Scope 4 - Embodiment and Action (7:00)
World Scope 5 - The Social World (9:50)
Reading Excerpts from the paper (10:48)
World scopes encompass each other (16:50)
Interesting thought experiments (19:42)
Conclusion (21:09)




Paper:
Experience Grounds Language (2020)
arxiv.org/abs/2004.10151
Authors: Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, Joseph Turian

----


Twitter: twitter.com/JayAlammar
Blog: jalammar.github.io/
Mailing List: jayalammar.substack.com/




More videos by Jay:
Language Processing with BERT: The 3 Minute Intro (Deep learning for NLP)
   • Language Processing with BERT: The 3 Minut...  


Explainable AI Cheat Sheet - Five Key Categories
   • Explainable AI Cheat Sheet - Five Key Cate...  


The Narrated Transformer Language Model
   • The Narrated Transformer Language Model  


Jay's Visual Intro to AI
   • Jay's Visual Intro to AI  


How GPT-3 Works - Easily Explained with Animations
   • How GPT3 Works - Easily Explained with Ani...