#94 Ilya Sutskever: Deep Learning

Published: May 8, 2020, 8:25 p.m.

b"Ilya Sutskever is the co-founder of OpenAI, is one of the most cited computer scientist in history with over 165,000 citations, and to me, is one of the most brilliant and insightful minds ever in the field of deep learning. There are very few people in this world who I would rather talk to and brainstorm with about deep learning, intelligence, and life than Ilya, on and off the mic.
\\n
\\nSupport this podcast by signing up with these sponsors:
\\n\\u2013 Cash App \\u2013 use code \\u201cLexPodcast\\u201d and download:
\\n\\u2013 Cash App (App Store):\\xa0https://apple.co/2sPrUHe
\\n\\u2013 Cash App (Google Play):\\xa0https://bit.ly/2MlvP5w
\\n
\\nEPISODE LINKS:
\\nIlya's Twitter: https://twitter.com/ilyasut
\\nIlya's Website: https://www.cs.toronto.edu/~ilya/
\\n
\\nThis conversation is part of the Artificial Intelligence podcast.\\xa0If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon.
\\n
\\nHere's the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time.
\\n
\\nOUTLINE:
\\n00:00 - Introduction
\\n02:23 - AlexNet paper and the ImageNet moment
\\n08:33 - Cost functions
\\n13:39 - Recurrent neural networks
\\n16:19 - Key ideas that led to success of deep learning
\\n19:57 - What's harder to solve: language or vision?
\\n29:35 - We're massively underestimating deep learning
\\n36:04 - Deep double descent
\\n41:20 - Backpropagation
\\n42:42 - Can neural networks be made to reason?
\\n50:35 - Long-term memory
\\n56:37 - Language models
\\n1:00:35 - GPT-2
\\n1:07:14 - Active learning
\\n1:08:52 - Staged release of AI systems
\\n1:13:41 - How to build AGI?
\\n1:25:00 - Question to AGI
\\n1:32:07 - Meaning of life"