Novel Biologically Inspired Approaches to Extracting Online Information from Temporal Data

Malik, Zeeshan Khawar; Hussain, Amir; Wu, Jonathan

Abstract

In this paper, we aim to develop novel learning approaches for extracting invariant features from time series. Specifically, we implement an existing method of solving the generalized eigenproblem and use this to firstly implement the biologically inspired technique of slow feature analysis (SFA) originally developed by Wiskott and Sejnowski (Neural Comput 14:715–770, 2002) and a rival method derived earlier by Stone (Neural Comput 8(7):1463–1492, 1996). Secondly, we investigate preprocessing the data using echo state networks (ESNs) (Lukosevicius and Jaeger in Comput Sci Rev 3(3):127–149, 2009) and show that the combination of generalized eigensolver and ESN is very powerful as a more biologically plausible implementation of SFA. Thirdly, we also investigate the effect of higher-order derivatives as a smoothing constraint and show the overall smoothness in the output signal. We demonstrate the potential of our proposed techniques, benchmarked against state-of-the-art approaches, using datasets comprising artificial, MNIST digits and hand-written character trajectories.

Citation

Malik, Z. K., Hussain, A., & Wu, J. (2014). Novel Biologically Inspired Approaches to Extracting Online Information from Temporal Data. Cognitive Computation, 6(3), 595-607. https://doi.org/10.1007/s12559-014-9257-0

• Journal Article Type: Article
• Acceptance Date: Mar 17, 2014
• Online Publication Date: Apr 3, 2014
• Publication Date: 2014-09
• Deposit Date: Sep 26, 2019
• Journal: Cognitive Computation
• Print ISSN: 1866-9956
• Electronic ISSN: 1866-9964
• Publisher: BMC
• Peer Reviewed: Peer Reviewed
• Volume: 6
• Issue: 3
• Pages: 595-607
• DOI: https://doi.org/10.1007/s12559-014-9257-0
• Keywords: Slow feature analysis; Echo state network; Generalized eigenvalue problem; Recurrent Neural Network GenEigSfa; Stone’s criterion; Higher-order changes
• Public URL: http://researchrepository.napier.ac.uk/Output/1793053