On Optimizing the Structure of Neural Networks Through a Compact Codification of Their Architecture

Lupión, Marcos; Cruz, N. C.; Paechter, B.; Ortigosa, P. M.

Abstract

Neural networks stand out in Artificial Intelligence for their capacity of being applied to multiple challenging tasks such as image classification. However, designing a neural network to address a particular problem is also a demanding task that requires expertise and time-consuming trial-and-error stages. The design of methods to automate the designing of neural networks define a research field that generally relies on different optimization algorithms, such as population meta-heuristics. This work studies utilizing Teaching-Learning-based Optimization (TLBO), which had not been used before for this purpose up to the authors’ knowledge. It is widespread and does not have specific parameters. Besides, it would be compatible with deep neural network design, i.e., architectures with many layers, due to its conception as a large-scale optimizer. A new encoding scheme has been proposed to make this continuous optimizer compatible with neural network design. This method, which is of general application, i.e., not linked to TLBO, can represent different network architectures with a plain vector of real values. A compatible objective function that links the optimizer and the representation of solutions has also been developed. The performance of this framework has been studied by addressing the design of an image classification neural network based on the CIFAR-10 dataset. The achieved result outperforms the initial solutions designed by humans after letting them evolve.

Citation

Lupión, M., Cruz, N. C., Paechter, B., & Ortigosa, P. M. (2022, July). On Optimizing the Structure of Neural Networks Through a Compact Codification of Their Architecture. Presented at Metaheuristics: 14th International Conference, MIC 2022, Syracuse, Italy

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Metaheuristics: 14th International Conference, MIC 2022
• Start Date: Jul 11, 2022
• End Date: Jul 14, 2022
• Acceptance Date: Jun 1, 2022
• Online Publication Date: Feb 23, 2023
• Publication Date: 2023-02
• Deposit Date: Jun 12, 2023
• Publisher: Springer
• Pages: 133-142
• Series Title: Lecture Notes in Computer Science
• Series Number: 13838
• Series ISSN: 1611-3349
• Book Title: Metaheuristics: 14th International Conference, MIC 2022, Syracuse, Italy, July 11–14, 2022, Proceedings
• ISBN: 978-3-031-26503-7
• DOI: https://doi.org/10.1007/978-3-031-26504-4_10
• Keywords: Artificial intelligence, Neural network architecture optimization, Meta-heuristics, Teaching-Learning-based optimization
• Public URL: http://researchrepository.napier.ac.uk/Output/3033616