Improving Novelty Search with a Surrogate Model and Accuracy Objectives to Build High-Performing Ensembles of Classifiers

Cardoso, Rui P.; Hart, Emma; Pitt, Jeremy V.

Abstract

Neuroevolution combined with Novelty Search to promote behavioural diversity is capable of constructing high-performing ensembles for classification. However, using gradient descent to train evolved architectures during the search can be computationally prohibitive. We have proposed a method to overcome this limitation by using a surrogate model which estimates the behavioural distance between two neural network architectures, required to calculate novelty scores. This has demonstrated a speedup of 10 times over previous work, significantly improving on previous reported results on three benchmark datasets from Computer Vision—CIFAR-10, CIFAR-100, and SVHN. This method makes an explicit search for diversity considerably more tractable for the same bounded resources. Here we investigate a range of search methods that span the full spectrum of favouring accuracy, diversity, or different combinations of both. Surprisingly, we show that multiple unique combinations between a diversity metric and accuracy give rise to similar results. This enables us to posit the existence of a diversity-accuracy duality in ensembles of classifiers, which suggests that there might not be a need to find a trade-off between the two.

Citation

Cardoso, R. P., Hart, E., & Pitt, J. V. (2025). Improving Novelty Search with a Surrogate Model and Accuracy Objectives to Build High-Performing Ensembles of Classifiers. SN Computer Science, 6(6), Article 631. https://doi.org/10.1007/s42979-025-04056-4

• Journal Article Type: Article
• Acceptance Date: May 13, 2025
• Online Publication Date: Jul 11, 2025
• Publication Date: 2025
• Deposit Date: Jul 14, 2025
• Publicly Available Date: Jul 14, 2025
• Journal: SN Computer Science
• Electronic ISSN: 2661-8907
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 6
• Issue: 6
• Article Number: 631
• DOI: https://doi.org/10.1007/s42979-025-04056-4
• Keywords: Surrogate, Local competition, Novelty search, Diversity, Ensemble