Attention-Based Multi-modal Emotion Recognition from Art

Tashu, Tsegaye Misikir; Horváth, Tomáš

Abstract

Emotions are very important in dealing with human decisions, interactions, and cognitive processes. Art is an imaginative human creation that should be appreciated, thought-provoking, and elicits an emotional response. The automatic recognition of emotions triggered by art is of considerable importance. It can be used to categorize artworks according to the emotions they evoke, recommend paintings that accentuate or balance a particular mood, and search for paintings of a particular style or genre that represent custom content in a custom state of impact. In this paper, we propose an attention-based multi-modal approach to emotion recognition that aims to use information from both the painting and title channels to achieve more accurate emotion recognition. Experimental results on the WikiArt emotion dataset showed the efficiency of the model we proposed and the usefulness of image and text modalities in emotion recognition.

Citation

Tashu, T. M., & Horváth, T. (2021, January). Attention-Based Multi-modal Emotion Recognition from Art. Presented at ICPR 2021, Online

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: ICPR 2021
• Start Date: Jan 10, 2021
• End Date: Jan 11, 2021
• Online Publication Date: Feb 21, 2021
• Publication Date: 2021
• Deposit Date: Apr 8, 2024
• Publisher: Springer
• Pages: 604-612
• Series Title: Lecture Notes in Computer Science
• Series Number: 12663
• Series ISSN: 0302-9743
• Book Title: Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10–15, 2021, Proceedings, Part III
• ISBN: 9783030687953
• DOI: https://doi.org/10.1007/978-3-030-68796-0_43
• Keywords: Emotion recognition, Emotion analysis, Multi-modal
• Public URL: http://researchrepository.napier.ac.uk/Output/3587392