A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation

Gkatzia, Dimitra; Cercas Curry, Amanda; Rieser, Verena; Lemon, Oliver

Abstract

Decision-making is often dependent on uncertain data, e.g. data associated with confidence scores, such as probabilities. A concrete example of such data is weather data. We will demo a game-based setup for exploring the effectiveness of different approaches (graphics vs NLG) to communicating uncertainty in rainfall and temperature predictions (www.macs.hw.ac.uk/ InteractionLab/weathergame/ ). The game incorporates a natural language extension of the MetOffice Weather game1. The extended version of the game can beused in three ways: (1) to compare the effectiveness of different information presentations of uncertain data; (2) to collect data for the development of effective data-driven approaches; and (3) to serve as a task-based evaluation setup for Natural Language Generation (NLG).

Citation

Gkatzia, D., Cercas Curry, A., Rieser, V., & Lemon, O. (2015, September). A Game-Based Setup for Data Collection and Task-Based Evaluation of Uncertain Information Presentation. Presented at 15th European Workshop on Natural Language Generation (ENLG 2015), University of Brighton, Brighton, UK

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 15th European Workshop on Natural Language Generation (ENLG 2015)
• Start Date: Sep 10, 2015
• End Date: Sep 11, 2015
• Acceptance Date: Sep 11, 2015
• Publication Date: Sep 10, 2015
• Deposit Date: Aug 1, 2016
• Publisher: Association for Computational Linguistics (ACL)
• Pages: 112-113
• Book Title: Proceedings of the 15th European Workshop on Natural Language Generation
• ISBN: 9781941643785
• DOI: https://doi.org/10.18653/v1/w15-4720
• Keywords: Natural Language Generation, NLG, data-driven, task-based evaluation.
• Public URL: http://researchrepository.napier.ac.uk/Output/321802