Skip to main content

Research Repository

Advanced Search

Accelerating neural network architecture search using multi-GPU high-performance computing

Lupión, Marcos; Cruz, N. C.; Sanjuan, Juan F.; Paechter, Ben; Ortigosa, Pilar M.


Marcos Lupión

N. C. Cruz

Juan F. Sanjuan

Pilar M. Ortigosa


Neural networks stand out from artificial intelligence because they can complete challenging tasks, such as image classification. However, designing a neural network for a particular problem requires experience and tedious trial and error. Automating this process defines a research field usually relying on population-based meta-heuristics. This kind of optimizer generally needs numerous function evaluations, which are computationally demanding in this context as they involve building, training, and evaluating different neural networks. Fortunately, these algorithms are also well suited for parallel computing. This work describes how the teaching–learning-based optimization algorithm has been adapted for designing neural networks exploiting a multi-GPU high-performance computing environment. The optimizer, not applied before for this purpose up to the authors’ knowledge, has been selected because it lacks specific parameters and is compatible with large-scale optimization. Thus, its configuration does not result in another problem and could design architectures with many layers. The parallelization scheme is decoupled from the optimizer. It can be seen as an external evaluation service managing multiple GPUs for promising neural network designs, even at different machines, and multiple CPU’s for low-performing solutions. This strategy has been tested in designing a neural network for image classification based on the CIFAR-10 dataset. The architectures found outperform human designs, and the sequential process is accelerated 4.2 times with 4 GPUs and 96 cores thanks to parallelization, being the ideal speed up 4.39 in this case.

Journal Article Type Article
Acceptance Date Nov 16, 2022
Online Publication Date Dec 1, 2022
Publication Date 2023-05
Deposit Date Nov 21, 2022
Publicly Available Date Dec 2, 2023
Journal Journal of Supercomputing
Print ISSN 0920-8542
Publisher Springer
Peer Reviewed Peer Reviewed
Volume 79
Pages 7609-7625
Keywords Artificial neural networks, Neural network design, HPC, TLBO, Multi-GPU
Public URL


Accelerating Neural Network Architecture Search Using Multi-GPU High-performance Computing (accepted version) (469 Kb)

You might also like

Downloadable Citations