Improving Algorithm-Selection and Performance-Prediction via Learning Discriminating Training Samples

Renau, Quentin; Hart, Emma

Abstract

The choice of input-data used to train algorithm-selection models is recognised as being a critical part of the model success. Recently, feature-free methods for algorithm-selection that use short trajec-tories obtained from running a solver as input have shown promise. However, it is unclear to what extent these trajectories reliably discriminate between solvers. We propose a meta approach to generating discriminatory trajectories with respect to a portfolio of solvers. The algorithm-configuration tool irace is used to tune the parameters of a simple Simulated Annealing algorithm (SA) to produce trajectories that maximise the performance metrics of ML models trained on this data. We show that when the trajectories obtained from the tuned SA algorithm are used in ML models for algorithm-selection and performance prediction, we obtain significantly improved performance metrics compared to models trained both on raw trajectory data and on exploratory landscape features.

Citation

Renau, Q., & Hart, E. (2024, July). Improving Algorithm-Selection and Performance-Prediction via Learning Discriminating Training Samples. Presented at GECCO 2024, Melbourne, Australia

• Presentation Conference Type: Conference Paper (published)
• Conference Name: GECCO 2024
• Start Date: Jul 14, 2024
• End Date: Jul 18, 2024
• Acceptance Date: Mar 21, 2024
• Online Publication Date: Jul 14, 2024
• Publication Date: 2024
• Deposit Date: May 2, 2024
• Publicly Available Date: Jul 14, 2024
• Publisher: Association for Computing Machinery (ACM)
• Peer Reviewed: Peer Reviewed
• Pages: 1026 - 1035
• Book Title: GECCO '24: Proceedings of the Genetic and Evolutionary Computation Conference
• ISBN: 9798400704949
• DOI: https://doi.org/10.1145/3638529.3654025
• Keywords: Algorithm Selection; Performance Prediction; Black-Box Optimisa- tion; Algorithm Trajectory