Seham Basabain S.Basabain@napier.ac.uk
Research Student
Enhancing Arabic-text Feature Extraction Utilizing Label-semantic Augmentation in Few/Zero-shot Learning
Basabain, Seham; Cambria, Erik; Alomar, Khalid; Hussain, Amir
Authors
Erik Cambria
Khalid Alomar
Prof Amir Hussain A.Hussain@napier.ac.uk
Professor
Abstract
A growing amount of research use pre-trained language models to address few/zero-shot text classification problems. Most of these studies neglect the semantic information hidden implicitly beneath the natural language names of class labels and develop a meta learner from the input texts solely. In this work, we demonstrate how label information can be utilized to extract enhanced feature representation of the input text from a Transformer-based pre-trained language model such as AraBERT. In addition, how this approach can improve performance when the data resources are scarce like in the Arabic language and the input text is short with little semantic information as is the case using tweets. The work also applies zero-shot text classification to predict new classes with no training examples across different domains including sarcasm detection and sentiment analysis using the information in the last layer of a trained classifier in a transfer learning setting. Experiments show that our approach has a better performance for the few-shot sentiment classification compared to baseline models and models trained without augmenting label information. Moreover, the zero-shot implementation achieved an accuracy up to 0.874 in Arabic sarcasm detection from a model trained on a sentiment analysis task.
Journal Article Type | Article |
---|---|
Acceptance Date | Apr 19, 2023 |
Online Publication Date | May 3, 2023 |
Publication Date | 2023-09 |
Deposit Date | May 3, 2023 |
Publicly Available Date | May 3, 2023 |
Print ISSN | 0266-4720 |
Electronic ISSN | 1468-0394 |
Publisher | Wiley |
Peer Reviewed | Peer Reviewed |
Volume | 40 |
Issue | 8 |
Article Number | e13329 |
DOI | https://doi.org/10.1111/exsy.13329 |
Keywords | Arabic text classification, contextual embeddings, feature extraction, few/zero-shot learning, label semantics |
Files
Enhancing Arabic-text Feature Extraction Utilizing Label-semantic Augmentation in Few/Zero-shot Learning
(1.9 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by-nc/4.0/
You might also like
Applications of Deep Learning and Reinforcement Learning to Biological Data
(2018)
Journal Article
Guided Policy Search for Sequential Multitask Learning
(2018)
Journal Article
Learning Latent Features With Infinite Nonnegative Binary Matrix Trifactorization
(2018)
Journal Article
Cross-modality interactive attention network for multispectral pedestrian detection
(2018)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search