A Novel Deep Learning-Based Multilevel Parallel Attention Neural (MPAN) Model for Multidomain Arabic Sentiment Analysis

El-Affendi, Mohammed A.; Alrajhi, Khawla; Hussain, Amir

Abstract

Over the past few years, much work has been done to develop machine learning models that perform Arabic sentiment analysis (ASA) tasks at various levels and in different domains. However, most of this work has been based on shallow machine learning, with little attention given to deep learning approaches. Furthermore, the deep learning models used for ASA have been based on noncontextualized embedding schemes that negatively impact model performances. This article proposes a novel deep learning-based multilevel parallel attention neural (MPAN) model that uses a simple positioning binary embedding scheme (PBES) to simultaneously compute contextualized embeddings at the character, word, and sentence levels. The MPAN model then computes multilevel attention vectors and concatenates them at the output level to produce competitive accuracies. Specifically, the MPAN model produces state-of-the-art results that outperform all established ASA baselines using 34 publicly available ASA datasets. The proposed model is further shown to produce new state-of-the-art accuracies for two multidomain collections: 95.61% for a binary classification collection and 94.25% for a tertiary classification collection. Finally, the performance of the MPAN model is further validated using the public IMDB movie review dataset, on which it produces an accuracy of 96.13%, placing it in second position on the global IMDB leaderboard.

• Journal Article Type: Article
• Acceptance Date: Dec 27, 2020
• Online Publication Date: Jan 6, 2021
• Publication Date: 2021
• Deposit Date: Feb 4, 2021
• Publicly Available Date: Feb 4, 2021
• Journal: IEEE Access
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 9
• Pages: 7508-7518
• DOI: https://doi.org/10.1109/access.2021.3049626
• Keywords: Arabic sentiment analysis, deep learning, multilevel parallel attention, natural language processing, positioning binary embedding, power-of-two, polynomial space positioning attention
• Public URL: http://researchrepository.napier.ac.uk/Output/2725493