Skip to main content

Research Repository

Advanced Search

A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation

Gkatzia, Dimitra; Cercas Curry, Amanda; Rieser, Verena; Lemon, Oliver

Authors

Amanda Cercas Curry

Verena Rieser

Oliver Lemon



Abstract

Decision-making is often dependent on uncertain data, e.g. data associated with confidence scores, such as probabilities. A concrete example of such data is weather data. We will demo a game-based setup for exploring the effectiveness of different approaches (graphics vs NLG) to communicating uncertainty in rainfall and temperature predictions (www.macs.hw.ac.uk/ InteractionLab/weathergame/ ). The game incorporates a natural language extension of the MetOffice Weather game1. The extended version of the game can beused in three ways: (1) to compare the effectiveness of different information presentations of uncertain data; (2) to collect data for the development of effective data-driven approaches; and (3) to serve as a task-based evaluation setup for Natural Language Generation (NLG).

Citation

Gkatzia, D., Cercas Curry, A., Rieser, V., & Lemon, O. (2015). A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation. In Proceedings of the 15th European Workshop on Natural Language Generation (112-113). https://doi.org/10.18653/v1/w15-4720

Conference Name 15th European Workshop on Natural Language Generation (ENLG 2015)
Conference Location University of Brighton, Brighton, UK
Start Date Sep 10, 2015
End Date Sep 11, 2015
Acceptance Date Sep 11, 2015
Publication Date Sep 10, 2015
Deposit Date Aug 1, 2016
Publisher Association for Computational Linguistics (ACL)
Pages 112-113
Book Title Proceedings of the 15th European Workshop on Natural Language Generation
ISBN 9781941643785
DOI https://doi.org/10.18653/v1/w15-4720
Keywords Natural Language Generation, NLG, data-driven, task-based evaluation.
Public URL http://researchrepository.napier.ac.uk/Output/321802