Emiel van Miltenburg
Underreporting of errors in NLG output, and what to do about it
van Miltenburg, Emiel; Clinciu, Miruna-Adriana; Du�ek, Ond?ej; Gkatzia, Dimitra; Inglis, Stephanie; Lepp�nen, Leo; Mahamood, Saad; Manning, Emma; Schoch, Stephanie; Thomson, Craig; Wen, Luou
Authors
Miruna-Adriana Clinciu
Ond?ej Du�ek
Dr Dimitra Gkatzia D.Gkatzia@napier.ac.uk
Associate Professor
Stephanie Inglis
Leo Lepp�nen
Saad Mahamood
Emma Manning
Stephanie Schoch
Craig Thomson
Luou Wen
Abstract
We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overall performance metrics, the research community is left in the dark about the specific weaknesses that are exhibited by `state-of-the-art' research. Next to quantifying the extent of error under-reporting, this position paper provides recommendations for error identification, analysis and reporting.
Citation
van Miltenburg, E., Clinciu, M.-A., Dušek, O., Gkatzia, D., Inglis, S., Leppänen, L., Mahamood, S., Manning, E., Schoch, S., Thomson, C., & Wen, L. (2021, September). Underreporting of errors in NLG output, and what to do about it. Presented at 14th International Conference on Natural Language Generation, Aberdeen, UK
Presentation Conference Type | Conference Paper (published) |
---|---|
Conference Name | 14th International Conference on Natural Language Generation |
Start Date | Sep 20, 2021 |
End Date | Sep 24, 2021 |
Acceptance Date | Jul 26, 2021 |
Publication Date | 2021 |
Deposit Date | Aug 10, 2021 |
Publicly Available Date | Aug 10, 2021 |
Pages | 140-153 |
Book Title | Proceedings of the 14th International Conference on Natural Language Generation |
Public URL | http://researchrepository.napier.ac.uk/Output/2791216 |
Publisher URL | https://aclanthology.org/2021.inlg-1.14/ |
Related Public URLs | https://inlg2021.github.io/pages/calls.html |
Files
Underreporting Of Errors In NLG Output, And What To Do About It (accepted version)
(336 Kb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
Data-to-Text Generation Improves Decision-Making Under Uncertainty
(2017)
Journal Article
Multi-adaptive Natural Language Generation using Principal Component Regression
(2014)
Presentation / Conference Contribution
The REAL corpus
(2016)
Data
Monitoring Users’ Behavior: Anti-Immigration Speech Detection on Twitter
(2020)
Journal Article
Opportunities and risks in the use of AI in career development practice
(2022)
Journal Article
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search