Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning

Nvidia presented parts of this work at GTC 2022, revealing our humanoid-quadruped transformer! Title: Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning Authors: Eric Vollenweider, Marko Bjelonic, Victor Klemm, Nikita Rudin, Joonho Lee and Marco Hutter Paper submitted to IEEE/RSJ International Conference on Intelligent Robots and Systems in Kyoto. Preprint: Abstract: In recent years, reinforcement learning (RL) has shown outstanding performance for locomotion control of highly articulated robotic systems. Such approaches typically involve tedious reward function tuning to achieve the desired motion style. Imitation learning approaches such as adversarial motion priors aim to reduce this problem by encouraging a pre-defined motion style. In this work, we present an approach to augment the concept of adversarial motion prior-based RL to allow for multiple, discretely switchable styles. We show that multiple styles and skills can be learned simultaneously without notable performance differences, even in combination with motion data-free skills. Our approach is validated in several real-world experiments with a wheeled-legged quadruped robot showing skills learned from existing RL controllers and trajectory optimization, such as ducking and walking, and novel skills such as switching between a quadrupedal and humanoid configuration. For the latter skill, the robot is required to stand up, navigate on two wheels, and sit down. Instead of tuning the sit-down motion, we verify that a reverse playback of the stand-up movement helps the robot discover feasible sit-down behaviors and avoids tedious reward function tuning. Note: The following parts of the video are sped up: - Door opening and closing when the robot stands inside the elevator between 00:20 and 00:21 ( 200%) - In-between the standing up and sitting down sequence between 00:49 and 01:10 ( 200% only the navigation on two legs) - Reaction with the crowd between 1:41 and 1:50 ( 150%) - Last drone footage after 2:01 ( 200%) Learn more at