Skip to main content

Research Repository

Advanced Search

From the Virtual to the RealWorld: Referring to Objects in Real-World Spatial Scenes

Gkatzia, Dimitra; Rieser, Verena; Bartie, Phil; Mackaness, William

Authors

Verena Rieser

Phil Bartie

William Mackaness



Abstract

Predicting the success of referring expressions (RE) is vital for real world applications such as navigation systems. Traditionally, research has focused on studying Referring Expression Generation (REG) in virtual, controlled environments. In this paper, we describe a novel study of spatial references from real scenes rather than virtual. First, we investigate how humans describe objects in open, uncontrolled scenarios and compare our findings to those reported in virtual environments. We show that REs in real-world scenarios differ significantly to those in virtual worlds. Second, we propose a novel approach to quantifying image complexity when complete annotations are not present (e.g. due to poor object recognition capabitlities), and third, we present a model for success prediction of REs for objects in real scenes. Finally, we discuss implications for Natural Language Generation (NLG) systems and future directions.

Citation

Gkatzia, D., Rieser, V., Bartie, P., & Mackaness, W. (2015, September). From the Virtual to the RealWorld: Referring to Objects in Real-World Spatial Scenes. Presented at 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon

Presentation Conference Type Conference Paper (Published)
Conference Name 2015 Conference on Empirical Methods in Natural Language Processing
Start Date Sep 17, 2015
End Date Sep 21, 2015
Acceptance Date Sep 17, 2015
Publication Date Sep 17, 2015
Deposit Date Aug 1, 2016
Publisher Association for Computational Linguistics (ACL)
Pages 1936-1942
Book Title Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing
ISBN 9781941643327
DOI https://doi.org/10.18653/v1/d15-1224
Keywords Referring expressions, RE, navigation systems, virtual environments, natural language generation, NLG.
Public URL http://researchrepository.napier.ac.uk/Output/321797






You might also like



Downloadable Citations