MIT : Recurrent Neural Networks and Transformers

MIT Introduction to Deep Learning : Lecture 2 Recurrent Neural Networks Lecturer: Ava Soleimany January 2022 For all lectures, slides, and lab materials: Lecture Outline 0:00​ - Introduction 1:59​ - Sequence modeling 4:16​ - Neurons with recurrence 10:09 - Recurrent neural networks 11:42​ - RNN intuition 14:44​ - Unfolding RNNs 16:43 - RNNs from scratch 19:49 - Design criteria for sequential modeling 21:00 - Word prediction example 27:49​ - Backpropagation through time 30:02 - Gradient issues 33:53​ - Long short term memory (LSTM) 35:35​ - RNN applications 40:22 - Attention fundamentals 43:12 - Intuition of attention 44:53 - Attention and search relationship 47:16 - Learning attention with neural networks 54:52 - Scaling attention and applications 56:09 - Summary Subscribe to stay up to date with new deep learning lectures at MIT, or follow us @MITDeepLearning on Twitter and Instagram to stay fully-connected!!