The Lottery Ticket Hypothesis and pruning in PyTorch

In this video, we are going to explain how one can do pruning in PyTorch. We will then use this knowledge to implement a paper called “The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks“. The paper states that feedforward neural networks have subnetworks (winning tickets) inside of them that perform as good as (or even better than) the original network. It also proposes a recipe how to find them. Paper: Official code: Code from this video: 00:00 Intro 00:50 Paper overview: Hypothesis (diagram) [slides] 01:33 Paper overview: Hypothesis (formal) [slides] 02:15 Paper overview: Finding winning tickets [slides] 03:44 Paper overview: Our setup [slides] 05:08 Pruning 101 in PyTorch [code] 10:29 Data - MNIST [code] 12:18 Multilayer perceptron [code] 14:05 Pruning: Linear MLP [code]