Etor Arza
Generalized Early Stopping in Evolutionary Direct Policy Search
Arza, Etor; Le Goff, Léni K.; Hart, Emma
Authors
Abstract
Lengthy evaluation times are common in many optimization problems such as direct policy search tasks, especially when they involve conducting evaluations in the physical world, e.g. in robotics applications. Often when evaluating solution over a fixed time period it becomes clear that the objective value will not increase with additional computation time (for example when a two wheeled robot continuously spins on the spot). In such cases, it makes sense to stop the evaluation early to save computation time. However, most approaches to stop the evaluation are problem specific and need to be specifically designed for the task at hand. Therefore, we propose an early stopping method for direct policy search. The proposed method only looks at the objective value at each time step and requires no problem specific knowledge. We test the introduced stopping criterion in five direct policy search environments drawn from games, robotics and classic control domains, and show that it can save up to 75% of the computation time. We also compare it with problem specific stopping criteria and show that it performs comparably, while being more generally applicable.
Citation
Arza, E., Le Goff, L. K., & Hart, E. (2024). Generalized Early Stopping in Evolutionary Direct Policy Search. ACM Transactions on Evolutionary Learning and Optimization, 4(3), Article 14. https://doi.org/10.1145/3653024
Journal Article Type | Article |
---|---|
Acceptance Date | Mar 13, 2024 |
Online Publication Date | Mar 20, 2024 |
Publication Date | 2024-07 |
Deposit Date | Mar 15, 2024 |
Publicly Available Date | Mar 20, 2024 |
Print ISSN | 2688-299X |
Electronic ISSN | 2688-3007 |
Publisher | Association for Computing Machinery (ACM) |
Peer Reviewed | Peer Reviewed |
Volume | 4 |
Issue | 3 |
Article Number | 14 |
DOI | https://doi.org/10.1145/3653024 |
Keywords | Applied computing, Engineering, Mathematics of computing, Mathematical optimization, Computing methodologies, Simulation evaluation, Optimization, Early Stopping, Policy Learning |
Public URL | http://researchrepository.napier.ac.uk/Output/3564434 |
Files
Generalized Early Stopping In Evolutionary Direct Policy Search (Accepted version)
(1.5 Mb)
PDF
You might also like
Bootstrapping artificial evolution to design robots for autonomous fabrication
(2020)
Journal Article
Robotics and Autonomous Systems for Environmental Sustainability: Monitoring Terrestrial Biodiversity
(2023)
Preprint / Working Paper
Towards a Unified Framework for Software-Hardware Integration in Evolutionary Robotics
(2024)
Journal Article
On Pros and Cons of Evolving Topologies with Novelty Search
(2020)
Presentation / Conference Contribution
Understanding fitness landscapes in morpho-evolution via local optima networks
(2024)
Presentation / Conference Contribution