Tsegaye Misikir Tashu
Synonym-Based Essay Generation and Augmentation for Robust Automatic Essay Scoring
Tashu, Tsegaye Misikir; Horváth, Tomáš
Authors
Tomáš Horváth
Contributors
Hujun Yin
Editor
David Camacho
Editor
Peter Tino
Editor
Abstract
Automatic essay scoring (AES) models based on neural networks (NN) have had a lot of success. However, research has shown that NN-based AES models have robustness issues, such that the output of a model changes easily with small changes in the input. We proposed to use keyword-based lexical substitution using BERT that generates new essays (adversarial samples) which are lexically similar to the original essay to evaluate the robustness of AES models trained on the original set. In order to evaluate the proposed approach, we implemented three NN-based scoring approaches and trained the scoring models using two stages. First, we trained each model using the original data and evaluate the performance using the original test and newly generated test set to see the impact of the adversarial sample of the model. Secondly, we trained the models by augmenting the generated adversarial essay with the original data to train a robust model against synonym-based adversarial attacks. The results of our experiments showed that extracting the most important words from the essay and replacing them with lexically similar words, as well as generating adversarial samples for augmentation, can significantly improve the generalization of NN-based AES models. Our experiments also demonstrated that the proposed defense is capable of not only defending against adversarial attacks, but also of improving the performance of NN-based AES models.
Citation
Tashu, T. M., & Horváth, T. (2022, November). Synonym-Based Essay Generation and Augmentation for Robust Automatic Essay Scoring. Presented at 23rd International Conference on Intelligent Data Engineering and Automated Learning (IDEAL), Manchester
Presentation Conference Type | Conference Paper (Published) |
---|---|
Conference Name | 23rd International Conference on Intelligent Data Engineering and Automated Learning (IDEAL) |
Start Date | Nov 24, 2022 |
End Date | Nov 26, 2022 |
Online Publication Date | Nov 21, 2022 |
Publication Date | 2022 |
Deposit Date | Apr 8, 2024 |
Publisher | Springer |
Pages | 12-21 |
Series Title | Lecture Notes in Computer Science |
Series Number | 13756 |
Series ISSN | 0302-9743 |
Book Title | Intelligent Data Engineering and Automated Learning – IDEAL 2022: 23rd International Conference, IDEAL 2022, Manchester, UK, November 24–26, 2022, Proceedings |
ISBN | 9783031217524 |
DOI | https://doi.org/10.1007/978-3-031-21753-1_2 |
Keywords | Adversarial attack, Data augmentation, Automatic essay scoring |
Public URL | http://researchrepository.napier.ac.uk/Output/3587396 |
You might also like
A Comparative Study of Assessment Metrics for Imbalanced Learning
(2023)
Presentation / Conference Contribution
Squared Symmetric Formal Contexts and Their Connections with Correlation Matrices
(2023)
Presentation / Conference Contribution
NCC: Neural concept compression for multilingual document recommendation
(2023)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search