Mixed Reality Teleoperation Assistance for Direct Control of Humanoids

Teleoperation plays a crucial role in enabling robot operations in challenging environments, yet existing limitations in effectiveness and accuracy necessitate the development of innovative strategies for improving teleoperated tasks. The work illustrated in this video introduces a novel approach that utilizes mixed reality and assistive autonomy to enhance the efficiency and precision of humanoid robot teleoperation. By leveraging Probabilistic Movement Primitives, object detection, and Affordance Templates, the assistance combines user motion with autonomous capabilities, achieving task efficiency while maintaining human-like robot motion. Experiments and feasibility studies on the Nadia robot confirm the effectiveness of the proposed framework. Associated publication available at L. Penco et al., “Mixed Reality Teleoperation Assistance for Direct Control of Humanoids,“ in IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1937-194