Sim-to-Real Transfer with Incremental Environment Complexity for Reinforcement Learning of Depth-based Robot Navigation

DOI10.5220/0009821603140323zbMATH Open1530.93312arXiv2004.14684OpenAlexW3042729974MaRDI QIDQ6091986FDOQ6091986

Authors: Thomas Chaffre, Julien Moras, Adrien Chan-Hon-Tong, Julien Marzat 

Publication date: 22 November 2023

Published in: Proceedings of the 17th International Conference on Informatics in Control, Automation and Robotics (Search for Journal in Brave)

Abstract: Transferring learning-based models to the real world remains one of the hardest problems in model-free control theory. Due to the cost of data collection on a real robot and the limited sample efficiency of Deep Reinforcement Learning algorithms, models are usually trained in a simulator which theoretically provides an infinite amount of data. Despite offering unbounded trial and error runs, the reality gap between simulation and the physical world brings little guarantee about the policy behavior in real operation. Depending on the problem, expensive real fine-tuning and/or a complex domain randomization strategy may be required to produce a relevant policy. In this paper, a Soft-Actor Critic (SAC) training strategy using incremental environment complexity is proposed to drastically reduce the need for additional training in the real world. The application addressed is depth-based mapless navigation, where a mobile robot should reach a given waypoint in a cluttered environment with no prior mapping information. Experimental results in simulated and real environments are presented to assess quantitatively the efficiency of the proposed approach, which demonstrated a success rate twice higher than a naive strategy.

Full work available at URL: https://arxiv.org/abs/2004.14684

Recommendations

• Robot control policy transfer based on progressive neural network
• Constructing an intelligent navigation system for autonomous mobile robot based on deep reinforcement learning
• Blind spot detection for safe sim-to-real transfer
• The Bottleneck Simulator: A Model-Based Deep Reinforcement Learning Approach
• Module-based reinforcement learning: Experiments with a real robot

zbMATH Keywords

reinforcement learningsim-to-real transferautonomous robot navigation

Mathematics Subject Classification ID

Agent technology and artificial intelligence (68T42) Automated systems (robots, etc.) in control theory (93C85)

Cited In (1)

• Echo-Enhanced Embodied Visual Navigation

This page was built for publication: Sim-to-Real Transfer with Incremental Environment Complexity for Reinforcement Learning of Depth-based Robot Navigation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6091986)