ViPlanner: Visual Semantic Imperative Learning for Local Navigation (ICRA 2024)

This video demonstrates how our fully-learned local planner can navigate complex environments by recognizing different terrains and their traversability. Our paper introduces a novel planner design that combines depth and semantic information. It employs the imperative learning paradigm for optimizing the planner weights end-to-end based on the planning task objective. The optimization uses a differentiable formulation of a semantic costmap, enabling the planner to differentiate between different terrains and accurately identify obstacles. Trained entirely in simulation, ViPlanner can be applied to real-world scenes in a zero-shot manner. Moreover, our experimental results demonstrate resistance to noise and a significant decrease in terms of traversability costs compared to purely geometric approaches. For more information: - Visit our Project Website at - Read our Paper - Checkout our Code