Emiel Van Miltenburg
Barriers and enabling factors for error analysis in NLG research
Van Miltenburg, Emiel; Clinciu, Miruna; Dušek, Ondřej; Gkatzia, Dimitra; Inglis, Stephanie; Leppänen, Leo; Mahamood, Saad; Schoch, Stephanie; Thomson, Craig; Wen, Luou
Authors
Miruna Clinciu
Ondřej Dušek
Dr Dimitra Gkatzia D.Gkatzia@napier.ac.uk
Associate Professor
Stephanie Inglis
Leo Leppänen
Saad Mahamood
Stephanie Schoch
Dr Craig Thomson C.Thomson3@napier.ac.uk
Lecturer
Luou Wen
Abstract
Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and discusses several ways to do so. This paper is based on our shared experience as authors as well as a survey we distributed as a means of public consultation. We provide an overview of existing barriers to carrying out error analyses, and propose changes to improve error reporting in the NLG literature.
Journal Article Type | Article |
---|---|
Online Publication Date | Feb 21, 2023 |
Publication Date | 2023 |
Deposit Date | Feb 23, 2023 |
Publicly Available Date | Feb 23, 2023 |
Journal | Northern European Journal of Language Technology |
Print ISSN | 2000-1533 |
Publisher | Linköpings University Electronic Press |
Peer Reviewed | Peer Reviewed |
Volume | 9 |
Issue | 1 |
DOI | https://doi.org/10.3384/nejlt.2000-1533.2023.4529 |
Files
Barriers And Enabling Factors For Error Analysis In NLG Research
(356 Kb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
How to Talk to Strangers: generating medical reports for first time users
(2016)
Presentation / Conference Contribution
The REAL Corpus: a crowd-sourced corpus of human generated and evaluated spatial references to real-world urban scenes
(2016)
Presentation / Conference Contribution
Exploratory Navigation for Runners Through Geographic Area Classification with Crowd-Sourced Data
(2015)
Presentation / Conference Contribution
From the Virtual to the RealWorld: Referring to Objects in Real-World Spatial Scenes
(2015)
Presentation / Conference Contribution
A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation
(2015)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search