Cloud based scalable object recognition from video streams using orientation fusion and convolutional neural networks

Usman Yaseen, Muhammad; Anjum, Ashiq; Fortino, Giancarlo; Liotta, Antonio; Hussain, Amir

Abstract

Object recognition from live video streams comes with numerous challenges such as the variation in illumination conditions and poses. Convolutional neural networks (CNNs) have been widely used to perform intelligent visual object recognition. Yet, CNNs still suffer from severe accuracy degradation, particularly on illumination-variant datasets. To address this problem, we propose a new CNN method based on orientation fusion for visual object recognition. The proposed cloud-based video analytics system pioneers the use of bi-dimensional empirical mode decomposition to split a video frame into intrinsic mode functions (IMFs). We further propose these IMFs to endure Reisz transform to produce monogenic object components, which are in turn used for the training of CNNs. Past works have demonstrated how the object orientation component may be used to pursue accuracy levels as high as 93%. Herein we demonstrate how a feature-fusion strategy of the orientation components leads to further improving visual recognition accuracy to 97%. We also assess the scalability of our method, looking at both the number and the size of the video streams under scrutiny. We carry out extensive experimentation on the publicly available Yale dataset, including also a self generated video datasets, finding significant improvements (both in accuracy and scale), in comparison to AlexNet, LeNet and SE-ResNeXt, which are three most commonly used deep learning models for visual object recognition and classification.

Citation

Usman Yaseen, M., Anjum, A., Fortino, G., Liotta, A., & Hussain, A. (2022). Cloud based scalable object recognition from video streams using orientation fusion and convolutional neural networks. Pattern Recognition, 121, Article 108207. https://doi.org/10.1016/j.patcog.2021.108207

• Journal Article Type: Article
• Acceptance Date: Mar 1, 2021
• Online Publication Date: Jul 27, 2021
• Publication Date: 2022-01
• Deposit Date: Oct 7, 2021
• Journal: Pattern Recognition
• Print ISSN: 0031-3203
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 121
• Article Number: 108207
• DOI: https://doi.org/10.1016/j.patcog.2021.108207
• Keywords: Scalable video anaytics, Feature fusion, Object orientation, Object recognition, Convolutional neural networks, Cloud-based video analytics
• Public URL: http://researchrepository.napier.ac.uk/Output/2810544