Fast and Slow Learning of Recurrent Independent Mechanisms (Machine Learning Paper Explained)

#metarim #deeprl #catastrophicforgetting Reinforcement Learning is very tricky in environments where the objective shifts over time. This paper explores agents in multi-task environments that are usually subject to catastrophic forgetting. Building on the concept of Recurrent Independent Mechanisms (RIM), the authors propose to separate the learning procedures for the mechanism parameters (fast) and the attention parameters (slow) and achieve superior results and more stability, and even better zero-shot transfer performance. OUTLINE: 0:00 - Intro & Overview 3:30 - Recombining pieces of knowledge 11:30 - Controllers as recurrent neural networks 14:20 - Recurrent Independent Mechanisms 21:20 - Learning at different time scales 28:40 - Experimental Results & My Criticism 44:20 - Conclusion & Comments Paper: RIM Paper: Abstract: Decomposing knowledge into interchangeable pieces promises a generalization advantage when there are changes in dist