Dr Sarah L. Thomson S.Thomson4@napier.ac.uk
Lecturer
Channel Configuration for Neural Architecture: Insights from the Search Space
Thomson, Sarah L.; Ochoa, Gabriela; Veerapen, Nadarajen; Michalak, Krzysztof
Authors
Gabriela Ochoa
Nadarajen Veerapen
Krzysztof Michalak
Abstract
We consider search spaces associated with neural network channel configuration. Architectures and their accuracy are visualised using low-dimensional Euclidean embedding (LDEE). Optimisation dynamics are captured using local optima networks (LONs). LONs are a compression of a fitness landscape: the nodes are local optima and the edges are search transitions between them. Several neural architecture search algorithms are tested on the search space and we discover that iterated local search (ILS) is a competitive algorithm for neural channel configuration. We additionally implement a landscape-aware ILS which performs well. Observations from the search and landscape space analyses bring visual clarity and insight to the science of neural network channel design: the results indicate that a high number of channels, kept constant throughout the network, is beneficial.
Citation
Thomson, S. L., Ochoa, G., Veerapen, N., & Michalak, K. (2023, July). Channel Configuration for Neural Architecture: Insights from the Search Space. Presented at GECCO '23, Lisbon, Portugal
Presentation Conference Type | Conference Paper (Published) |
---|---|
Conference Name | GECCO '23 |
Start Date | Jul 15, 2023 |
End Date | Jul 19, 2023 |
Acceptance Date | Apr 1, 2023 |
Online Publication Date | Jul 12, 2023 |
Publication Date | 2023-07 |
Deposit Date | Aug 16, 2023 |
Publicly Available Date | Aug 17, 2023 |
Publisher | Association for Computing Machinery (ACM) |
Pages | 1267-1275 |
Book Title | GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference |
ISBN | 9798400701207 |
DOI | https://doi.org/10.1145/3583131.3590386 |
Keywords | Fitness Landscapes, Neural Architecture Search, Local Optima Networks (LONs) |
Files
Channel Configuration For Neural Architecture: Insights From The Search Space
(5.9 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
Comparing communities of optima with funnels in combinatorial fitness landscapes
(2017)
Presentation / Conference Contribution
Multifractality and dimensional determinism in local optima networks
(2018)
Presentation / Conference Contribution
On the Fractal Nature of Local Optima Networks
(2018)
Presentation / Conference Contribution
The effect of landscape funnels in QAPLIB instances
(2017)
Presentation / Conference Contribution
Unexplained Fluctuations in Particle Swarm Optimisation Performance with Increasing Problem Dimensionality
(2023)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search