Mingqi Fang
STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation
Fang, Mingqi; Yu, Lingyun; Xie, Hongtao; Tan, Qingfeng; Tan, Zhiyuan; Hussain, Amir; Wang, Zezheng; Li, Jiahong; Tian, Zhihong
Authors
Lingyun Yu
Hongtao Xie
Qingfeng Tan
Dr Thomas Tan Z.Tan@napier.ac.uk
Associate Professor
Prof Amir Hussain A.Hussain@napier.ac.uk
Professor
Zezheng Wang
Jiahong Li
Zhihong Tian
Abstract
The impressive development of facial manipulation techniques has raised severe public concerns. Identity-aware methods, especially suitable for protecting celebrities, are seen as one of promising face forgery detection approaches with additional reference video. However, without in-depth observation of fake video’s characteristics, most existing identity-aware algorithms are just naive imitation of face verification model and fail to exploit discriminative information. In this article, we argue that it is necessary to take both spatial and temporal perspectives into consideration for adequate inconsistency clues and propose a novel forgery detector named SpatioTemporal IDentity network (STIDNet). To effectively capture heterogeneous spatiotemporal information in a unified formulation, our STIDNet is following a knowledge distillation architecture that the student identity extractor receives supervision from a spatial information encoder (SIE) and a temporal information encoder (TIE) through multiteacher training. Specifically, a regional sensitive identity modelling paradigm is proposed in SIE by introducing facial blending augmentation but with uniform identity label, thus encourage model to focus on spatial discriminative region like outer face. Meanwhile, considering the strong temporal correlation between audio and talking face video, our TIE is devised in a cross-modal pattern that the audio information is introduced to supervise model exploiting temporal personalized movements. Benefit from knowledge transfer from SIE and TIE, STIDNet is able to capture individual’s essential spatiotemporal identity attributes and sensitive to even subtle identity deviation caused by manipulation. Extensive experiments indicate the superiority of our STIDNet compared with previous works. Moreover, we also demonstrate STIDNet is more suitable for real-world implementation in terms of model complexity and reference set size.
Citation
Fang, M., Yu, L., Xie, H., Tan, Q., Tan, Z., Hussain, A., Wang, Z., Li, J., & Tian, Z. (2024). STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation. IEEE Transactions on Computational Social Systems, 11(4), 5354 - 5366. https://doi.org/10.1109/tcss.2024.3356549
Journal Article Type | Article |
---|---|
Acceptance Date | Jan 12, 2024 |
Online Publication Date | Feb 12, 2024 |
Publication Date | 2024-08 |
Deposit Date | Feb 16, 2024 |
Publicly Available Date | Feb 19, 2024 |
Journal | IEEE Transactions on Computational Social Systems |
Electronic ISSN | 2329-924X |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 11 |
Issue | 4 |
Pages | 5354 - 5366 |
DOI | https://doi.org/10.1109/tcss.2024.3356549 |
Keywords | Face Forgery Detection, Knowledge Distillation, Video Forensics, Deep Learning |
Public URL | http://researchrepository.napier.ac.uk/Output/3510279 |
Files
STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation (accepted version)
(7.5 Mb)
PDF
You might also like
Detection of Ransomware
(2024)
Patent
Machine Un-learning: An Overview of Techniques, Applications, and Future Directions
(2023)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search