Class Feature Pyramids for Video Explanation

Stergiou, Alexandros; Kapidis, Georgios; Kalliatakis, Grigorios; Chrysoulas, Christos; Poppe, Ronald; Veltkamp, Remco

Abstract

Deep convolutional networks are widely used in video
action recognition. 3D convolutions are one prominent approach to deal with the additional time dimension. While
3D convolutions typically lead to higher accuracies, the inner workings of the trained models are more difficult to interpret. We focus on creating human-understandable visual
explanations that represent the hierarchical parts of spatiotemporal networks. We introduce Class Feature Pyramids,
a method that traverses the entire network structure and
incrementally discovers kernels at different network depths
that are informative for a specific class. Our method does
not depend on the network’s architecture or the type of 3D
convolutions, supporting grouped and depth-wise convolutions, convolutions in fibers, and convolutions in branches.
We demonstrate the method on six state-of-the-art 3D convolution neural networks (CNNs) on three action recognition (Kinetics-400, UCF-101, and HMDB-51) and two
egocentric action recognition datasets (EPIC-Kitchens and
EGTEA Gaze+).

Citation

Stergiou, A., Kapidis, G., Kalliatakis, G., Chrysoulas, C., Poppe, R., & Veltkamp, R. (2020). Class Feature Pyramids for Video Explanation. . https://doi.org/10.1109/iccvw.2019.00524

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
• Start Date: Oct 27, 2019
• End Date: Oct 28, 2019
• Acceptance Date: Aug 20, 2019
• Online Publication Date: Mar 5, 2020
• Publication Date: Mar 5, 2020
• Deposit Date: Mar 26, 2020
• Publisher: Institute of Electrical and Electronics Engineers
• Series ISSN: 2473-9944
• DOI: https://doi.org/10.1109/iccvw.2019.00524
• Public URL: http://researchrepository.napier.ac.uk/Output/2648982