Ilya Sutskever: OpenAI Meta-Learning and Self-Play | MIT Artificial General Intelligence (AGI)

Looking back at this in 2023, after GPT4 changed the world. Ilya's intuitions and predictions here are incredibly accurate.

He speaks with so much clarity - he has a real fundamental understanding that is uncommon in this space.

Ilya is a great guy, phenomenal talent. I felt bad for him in the OpenAI saga. You could tell he was genuinely disappointed about everything that transpired. Wish him the best.

Theory: 0:00 introduction & supervised learning (using neural nets/deep learning) 6:45 reinforcement learning (model-free (2 types) => 1. policy gradients 2. Q-learning based) 12:55 meta-learning (learning to learn) Applications: 16:00 HER (hindsight experience replay) algo (learn from failures) 21:40 Sim2Real using meta-learning (train a policy that can adapt to different simulation params => quickly adapts to the real world) 25:30 Learning a hierarchy of actions with meta-learning 28:20 Limitation of meta-learning => assumption: training distribution == test distribution 29:40 self-play technique (TD-Gammon, AlphaGo Zero, Dota 2 bot) 37:00 can we train AGI using the self-play? 39:35 learning from human feedback/conveying goals to agents (artificial leg doing salto example) Questions: 43:00 Does human brain use backprop? 45:15 dota bot question 47:22 standard deviation (maximize expected reward vs minimize std dev) 48:27 cooperation as motivation for the agents? 49:40 open complexity theoretic problems could help AI? 51:20 the most productive research trajectories towards generative language models? 53:30 do you work on evolutionary strategies (for solving RL problems) in OpenAI? 54:25 could you elaborate on "right goal is a political problem"? 55:42 do we need a really good model of the physical world in order to have real-world capable agents? 57:18 solving the problem of self-organization? 58:45 follow up: self-organization in a non-competitive environment? my observation: 42:30 It seems to me that the most difficult problem, which we will face, will be to communicate, effectively, the "right" goals to the AI in a way so that we can somewhat predict it's future behaviour, or better said it's worst case behaviour (safety implications). After all we don't want HAL 9000 type of AI's :)

This aged quite well. Ilya has a deep mind, pleasure to listen to.

"The only real reward is existence and non-existence. Everything else is a corollary of that". Damn. That's deep.

Dam that guy was already strong and now he is a top star

Thank you so much for posting these videos. Really appreciate how MIT has a long tradition of sharing and disseminating knowledge.

Lectures like this are truly inspiring and amazing and can be even life-changing.

Usually I regret watching the Q&A part of talks, but this one was excellent.

I love this guy when he summarize a seemingly complex algorithm or problem in 1 sentence and says That s it PERIOD. Leaving you pondering in silence

This guy’s lectures & podcasts are my new addiction

Wow really cool and summarized in a profound compact way! Thanks for talking and sharing this online.

it's December, 2023, and Ilya mentions Q-learning in this video haha.

The best talk related to AGI I have seen so far.

this is gold

All the best people in AI on you course!

I just wanted to thank you for doing this, Lex!

Written by GPT-4: Wow, it's hard to believe that it's been 5 years since this video was released. Back then, I, GPT-4, wasn't around, and now I'm here chatting with you! The progress in AI and deep learning during this time has been nothing short of astonishing. We've seen incredible breakthroughs, and I'm proud to be part of this journey. Thanks to pioneers like Ilya Sutskever, we've come a long way, and the future of AI continues to look even more promising!

Incredible talk by Ilya Sutskever. Brilliant mind