Skip to main content

Research Repository

Advanced Search

Multimodal salient object detection via adversarial learning with collaborative generator

Tu, Zhengzheng; Yang, Wenfang; Wang, Kunpeng; Hussain, Amir; Luo, Bin; Li, Chenglong


Zhengzheng Tu

Wenfang Yang

Kunpeng Wang

Bin Luo

Chenglong Li


Multimodal salient object detection(MSOD), which utilizes multimodal information (e.g., RGB image and thermal infrared or depth image) to detect common salient objects, has received much attention recently. Different modalities reflect different appearance properties of salient objects, some of which could contribute to improving the precision and/or recall of MSOD. To greatly improve both Precision and Recall by fully exploring multimodal data, in this work, we propose an effective adversarial learning framework based on a novel collaborative generator for accurate multimodal salient object detection. In particular, the collaborative generator consists of three generators (generator1, generator2 and generator3), which aim at decreasing the false positive and false negative of the generated saliency maps and improving F-measure of the final saliency maps respectively. Generator1 and generator2 contain two encoder–decoder networks for multimodal inputs, and we propose a new co-attention model to perform adaptive interactions between different modalities. Furthermore, we apply generator3 to integrate feature maps from generator1 and generator2 in a complementary way. Through adversarially learning the collaborative generator and discriminator, both Precision and Recall of the predicted maps are boosted with the complementary benefits of multimodal data. Extensive experiments on three RGBT datasets and six RGBD datasets show that our method performs quite well against state-of-the-art MSOD methods.


Tu, Z., Yang, W., Wang, K., Hussain, A., Luo, B., & Li, C. (2023). Multimodal salient object detection via adversarial learning with collaborative generator. Engineering Applications of Artificial Intelligence, 119, Article 105707.

Journal Article Type Article
Acceptance Date Dec 1, 2022
Online Publication Date Dec 19, 2022
Publication Date 2023-03
Deposit Date Feb 15, 2023
Publicly Available Date Dec 20, 2023
Journal Engineering Applications of Artificial Intelligence
Print ISSN 0952-1976
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 119
Article Number 105707
Keywords Multimodal salient object detection, Collaborative generator, Adversarial learning
Public URL


Multimodal Salient Object Detection Via Adversarial Learning With Collaborative Generator (accepted version) (1.8 Mb)

You might also like

Downloadable Citations