Simone Scardapane
Distributed Reservoir Computing with Sparse Readouts [Research Frontier]
Scardapane, Simone; Panella, Massimo; Comminiello, Danilo; Hussain, Amir; Uncini, Aurelio
Authors
Massimo Panella
Danilo Comminiello
Prof Amir Hussain A.Hussain@napier.ac.uk
Professor
Aurelio Uncini
Abstract
In a network of agents, a widespread problem is the need to estimate a common underlying function starting from locally distributed measurements. Real-world scenarios may not allow the presence of centralized fusion centers, requiring the development of distributed, message-passing implementations of the standard machine learning training algorithms. In this paper, we are concerned with the distributed training of a particular class of recurrent neural networks, namely echo state networks (ESNs). In the centralized case, ESNs have received considerable attention, due to the fact that they can be trained with standard linear regression routines. Based on this observation, in our previous work we have introduced a decentralized algorithm, framed in the distributed optimization field, in order to train an ESN. In this paper, we focus on an additional sparsity property of the output layer of ESNs, allowing for very efficient implementations of the resulting networks. In order to evaluate the proposed algorithm, we test it on two well-known prediction benchmarks, namely the Mackey-Glass chaotic time series and the 10th order nonlinear auto regressive moving average (NARMA) system.
Journal Article Type | Article |
---|---|
Online Publication Date | Oct 10, 2016 |
Publication Date | 2016-11 |
Deposit Date | Oct 4, 2019 |
Journal | IEEE Computational Intelligence Magazine |
Print ISSN | 1556-603X |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 11 |
Issue | 4 |
Pages | 59-70 |
DOI | https://doi.org/10.1109/MCI.2016.2601759 |
Keywords | Regression analysis, Training data, Algorithm design and analysis, Machine learning algorithms, Linear regression, Optimization, Recurrent neural networks |
Public URL | http://researchrepository.napier.ac.uk/Output/1792689 |
You might also like
Applications of Deep Learning and Reinforcement Learning to Biological Data
(2018)
Journal Article
Guided Policy Search for Sequential Multitask Learning
(2018)
Journal Article
Learning Latent Features With Infinite Nonnegative Binary Matrix Trifactorization
(2018)
Journal Article
Cross-modality interactive attention network for multispectral pedestrian detection
(2018)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search