Skip to main content

Research Repository

Advanced Search

STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation

Fang, Mingqi; Yu, Lingyun; Xie, Hongtao; Tan, Qingfeng; Tan, Zhiyuan; Hussain, Amir; Wang, Zezheng; Li, Jiahong; Tian, Zhihong


Mingqi Fang

Lingyun Yu

Hongtao Xie

Qingfeng Tan

Zezheng Wang

Jiahong Li

Zhihong Tian


The impressive development of facial manipulation techniques has raised severe public concerns. Identity-aware methods, especially suitable for protecting celebrities, are seen as one of promising face forgery detection approaches with additional reference video. However, without in-depth observation of fake video’s characteristics, most existing identity-aware algorithms are just naive imitation of face verification model and fail to exploit discriminative information. In this article, we argue that it is necessary to take both spatial and temporal perspectives into consideration for adequate inconsistency clues and propose a novel forgery detector named SpatioTemporal IDentity network (STIDNet). To effectively capture heterogeneous spatiotemporal information in a unified formulation, our STIDNet is following a knowledge distillation architecture that the student identity extractor receives supervision from a spatial information encoder (SIE) and a temporal information encoder (TIE) through multiteacher training. Specifically, a regional sensitive identity modelling paradigm is proposed in SIE by introducing facial blending augmentation but with uniform identity label, thus encourage model to focus on spatial discriminative region like outer face. Meanwhile, considering the strong temporal correlation between audio and talking face video, our TIE is devised in a cross-modal pattern that the audio information is introduced to supervise model exploiting temporal personalized movements. Benefit from knowledge transfer from SIE and TIE, STIDNet is able to capture individual’s essential spatiotemporal identity attributes and sensitive to even subtle identity deviation caused by manipulation. Extensive experiments indicate the superiority of our STIDNet compared with previous works. Moreover, we also demonstrate STIDNet is more suitable for real-world implementation in terms of model complexity and reference set size.

Journal Article Type Article
Acceptance Date Jan 12, 2024
Online Publication Date Feb 12, 2024
Deposit Date Feb 16, 2024
Publicly Available Date Feb 19, 2024
Journal IEEE Transactions on Computational Social Systems
Electronic ISSN 2329-924X
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Keywords Face Forgery Detection, Knowledge Distillation, Video Forensics, Deep Learning
Public URL


STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation (accepted version) (7.5 Mb)

You might also like

Downloadable Citations