Skip to main content

Research Repository

Advanced Search

Towards visual dialogue for human-robot interaction

Part, Jose L.; Hernández García, Daniel; Yu, Yanchao; Gunson, Nancie; Dondrup, Christian; Lemon, Oliver


Jose L. Part

Daniel Hernández García

Nancie Gunson

Christian Dondrup

Oliver Lemon


The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations about the surrounding world is critical. In this paper, we describe current progress towards developing the necessary integrated visual and conversational capabilities for a robot to operate in such environments. Concretely, we introduce an architecture for conversing about objects and other entities present in an environment. The work described in this paper has applications that extend well beyond healthcare and can be used on any robot that requires to interact with its visual and spatial environment in order to be able to perform its duties.

Presentation Conference Type Conference Paper (Published)
Conference Name HRI '21: ACM/IEEE International Conference on Human-Robot Interaction
Start Date Mar 8, 2021
End Date Mar 11, 2021
Online Publication Date Mar 8, 2021
Publication Date 2021-03
Deposit Date Jun 27, 2023
Publisher Association for Computing Machinery (ACM)
Pages 670-672
Book Title HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction
ISBN 978-1-4503-8290-8
Keywords visual dialogue, robotics, conversational systems