Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity

Allard, Maxime; Smith, Simón C.; Chatzilygeroudis, Konstantinos; Lim, Bryan; Cully, Antoine

Abstract

In real-world environments, robots need to be resilient to damages and robust to unforeseen scenarios. Quality-Diversity (QD) algorithms have been successfully used to make robots adapt to damages in seconds by leveraging a diverse set of learned skills. A high diversity of skills increases the chances of a robot to succeed at overcoming new situations since there are more potential alternatives to solve a new task. However, finding and storing a large behavioural diversity of multiple skills often leads to an increase in computational complexity. Furthermore, robot planning in a large skill space is an additional challenge that arises with an increased number of skills. Hierarchical structures can help to reduce this search and storage complexity by breaking down skills into primitive skills. In this article, we extend the analysis of the Hierarchical Trial and Error algorithm, which uses a hierarchical behavioural repertoire to learn diverse skills and leverages them to make the robot adapt quickly in the physical world. We show that the hierarchical decomposition of skills enables the robot to learn more complex behaviours while keeping the learning of the repertoire tractable. Experiments with a hexapod robot both in simulation and the physical world show that our method solves a maze navigation task with up to, respectively, 20% and 43% less actions than the best baselines while having 78% less complete failures.

• Journal Article Type: Article
• Acceptance Date: May 1, 2023
• Online Publication Date: Jun 28, 2023
• Publication Date: 2023-06
• Deposit Date: Jul 11, 2023
• Journal: ACM Transactions on Evolutionary Learning and Optimization
• Print ISSN: 2688-299X
• Electronic ISSN: 2688-3007
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Volume: 3
• Issue: 2
• Article Number: 6
• DOI: https://doi.org/10.1145/3596912
• Keywords: Quality-Diversity, Hierarchical learning, robot learning