Haotian Xu
Exploiting Attention-Consistency Loss For Spatial-Temporal Stream Action Recognition
Xu, Haotian; Jin, Xiaobo; Wang, Qiufeng; Hussain, Amir; Huang, Kaizhu
Abstract
Currently, many action recognition methods mostly consider the information from spatial streams. We propose a new perspective inspired by the human visual system to combine both spatial and temporal streams to measure their attention consistency. Specifically, a branch-independent convolutional neural network (CNN) based algorithm is developed with a novel attention-consistency loss metric, enabling the temporal stream to concentrate on consistent discriminative regions with the spatial stream in the same period. The consistency loss is further combined with the cross-entropy loss to enhance the visual attention consistency. We evaluate the proposed method for action recognition on two benchmark datasets: Kinetics400 and UCF101. Despite its apparent simplicity, our proposed framework with the attention consistency achieves better performance than most of the two-stream networks, i.e. 75.7% top-1 accuracy on Kinetics400 and 95.7% on UCF101, while reducing 7.1% computational cost compared with our baseline. Particularly, our proposed method can attain remarkable improvements on complex action classes, showing that our proposed network can act as a potential benchmark to handle complicated scenarios in industry 4.0 applications.
Citation
Xu, H., Jin, X., Wang, Q., Hussain, A., & Huang, K. (2022). Exploiting Attention-Consistency Loss For Spatial-Temporal Stream Action Recognition. ACM transactions on multimedia computing communications and applications, 18(2S), Article 119. https://doi.org/10.1145/3538749
Journal Article Type | Article |
---|---|
Acceptance Date | Apr 6, 2022 |
Online Publication Date | May 28, 2022 |
Publication Date | Oct 6, 2022 |
Deposit Date | Jun 22, 2022 |
Publicly Available Date | Jul 8, 2022 |
Journal | ACM Transactions on Multimedia Computing, Communications, and Applications |
Print ISSN | 1551-6857 |
Electronic ISSN | 1551-6865 |
Publisher | Association for Computing Machinery (ACM) |
Peer Reviewed | Peer Reviewed |
Volume | 18 |
Issue | 2S |
Article Number | 119 |
DOI | https://doi.org/10.1145/3538749 |
Keywords | Action Recognition, Attention Consistency, Multi-level Attention, Two-stream Structure |
Public URL | http://researchrepository.napier.ac.uk/Output/2876251 |
Files
Exploiting Attention-Consistency Loss For Spatial-Temporal Stream Action Recognition (accepted version)
(23.5 Mb)
PDF
You might also like
Machine Un-learning: An Overview of Techniques, Applications, and Future Directions
(2023)
Journal Article
Multi-criteria decision making-based waste management: A bibliometric analysis
(2023)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search