WILDA: Wide Learning of Diverse Architectures for Classification of Large Datasets

Pitt, Jeremy; Burth Kurka, David; Hart, Emma; Cardoso, Rui P.

Abstract

In order to address scalability issues, which can be a challenge for Deep Learning methods, we propose Wide Learning of Diverse Architectures-a model that scales horizontally rather than vertically, enabling distributed learning. We propose a distributed version of a quality-diversity evolutionary algorithm (MAP-Elites) to evolve an architecturally diverse ensemble of shallow networks, each of which extracts a feature vector from the data. These features then become the input to a single shallow network which is optimised using gradient descent to solve a classification task. The technique is shown to perform well on two benchmark classification problems (MNIST and CIFAR). Additional experiments provide insight into the role that diversity plays in contributing to the performance of the repertoire.

Citation

Pitt, J., Burth Kurka, D., Hart, E., & Cardoso, R. P. (2021). WILDA: Wide Learning of Diverse Architectures for Classification of Large Datasets. In Applications of Evolutionary Computation: EvoApplications 2021 Proceedings (649-664). https://doi.org/10.1007/978-3-030-72699-7_41

• Conference Name: 24th European Conference, EvoApplications 2021
• Conference Location: Online
• Start Date: Apr 7, 2021
• End Date: Apr 9, 2021
• Acceptance Date: Jan 20, 2021
• Online Publication Date: Apr 1, 2021
• Publication Date: 2021-04
• Deposit Date: Mar 30, 2021
• Publicly Available Date: Apr 2, 2022
• Publisher: Springer
• Pages: 649-664
• Series Title: Lecture Notes in Computer Science
• Series Number: 12694
• Series ISSN: 1611-3349
• Book Title: Applications of Evolutionary Computation: EvoApplications 2021 Proceedings
• ISBN: 978-3-030-72698-0
• DOI: https://doi.org/10.1007/978-3-030-72699-7_41
• Keywords: Diversity, MAP-elites, Machine Learning, Ensemble
• Public URL: http://researchrepository.napier.ac.uk/Output/2756732
• Publisher URL: https://www.springer.com/gp/book/9783030726980