A Snapshot of NLG Evaluation Practices 2005 - 2014

Gkatzia, Dimitra; Mahamood, Saad

Abstract

In this paper we present a snapshot of endto-end NLG system evaluations as presented in conference and journal papers1 over the last ten years in order to better understand the nature and type of evaluations that have been undertaken. We find that researchers tend to favour specific evaluation methods, and that their evaluation approaches are also correlated with the publication venue. We further discuss what factors may influence the types of evaluation used for a given NLG system.

Citation

Gkatzia, D., & Mahamood, S. (2015). A Snapshot of NLG Evaluation Practices 2005 - 2014. . https://doi.org/10.18653/v1/w15-4708

• Conference Name: Proceedings of the 15th European Workshop on Natural Language Generation (ENLG)
• Conference Location: University of Brighton Brighton, UK
• Start Date: Sep 10, 2015
• End Date: Sep 11, 2015
• Acceptance Date: Sep 10, 2015
• Publication Date: Sep 10, 2015
• Deposit Date: Aug 1, 2016
• Publisher: Association for Computational Linguistics (ACL)
• Pages: 57-60
• ISBN: 9781941643785
• DOI: https://doi.org/10.18653/v1/w15-4708
• Keywords: Natural Language Generation, evaluation methods.
• Public URL: http://researchrepository.napier.ac.uk/Output/321807