Multilayered Echo State Machine: A Novel Architecture and Algorithm

Malik, Z.K.; Hussain, A.; Wu, Q.J.

Abstract

In this paper, we present a novel architecture and learning algorithm for a multilayered echo state machine (ML-ESM). Traditional echo state networks (ESNs) refer to a particular type of reservoir computing (RC) architecture. They constitute an effective approach to recurrent neural network (RNN) training, with the (RNN-based) reservoir generated randomly, and only the readout trained using a simple computationally efficient algorithm. ESNs have greatly facilitated the real-time application of RNN, and have been shown to outperform classical approaches in a number of benchmark tasks. In this paper, we introduce a novel criteria for integrating multiple layers of reservoirs within the ML-ESM. The addition of multiple layers of reservoirs are shown to provide a more robust alternative to conventional RC networks. We demonstrate the comparative merits of this approach in a number of applications, considering both benchmark datasets and real world applications.

• Journal Article Type: Article
• Acceptance Date: Feb 8, 2016
• Online Publication Date: Jun 20, 2016
• Publication Date: 2017-04
• Deposit Date: Sep 23, 2019
• Journal: IEEE Transactions on Cybernetics
• Print ISSN: 2168-2267
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 47
• Issue: 4
• Pages: 946-959
• DOI: https://doi.org/10.1109/TCYB.2016.2533545
• Keywords: Learning, multiple layer network and time series neural network, neural network
• Public URL: http://researchrepository.napier.ac.uk/Output/1792553