Skip to main content

Research Repository

Advanced Search

Outputs (31)

How Much do Robots Understand Rudeness? Challenges in Human-Robot Interaction (2024)
Conference Proceeding
Orme, M., Yu, Y., & Tan, Z. (in press). How Much do Robots Understand Rudeness? Challenges in Human-Robot Interaction.

This paper concerns the pressing need to understand and manage inappropriate language within the evolving human-robot interaction (HRI) landscape. As intelligent systems and robots transition from controlled laboratory settings to everyday households... Read More about How Much do Robots Understand Rudeness? Challenges in Human-Robot Interaction.

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction (2023)
Conference Proceeding
Strathearn, C., Yu, Y., & Gkatzia, D. (2023). TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction. In Proceedings of The Joint CUI and HRI Workshop at HRI 2023

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans.... Read More about TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction.

MoDEsT: a Modular Dialogue Experiments and Demonstration Toolkit (2023)
Conference Proceeding
Yu, Y., & Oduronbi, D. (2023). MoDEsT: a Modular Dialogue Experiments and Demonstration Toolkit. In CUI '23: Proceedings of the 5th International Conference on Conversational User Interfaces. https://doi.org/10.1145/3571884.3604405

We present a modular dialogue experiments and demonstration toolkit (MoDEsT) that assists researchers in planning tailored conversational AI-related studies. The platform can: 1) assist users in picking multiple templates based on specific task needs... Read More about MoDEsT: a Modular Dialogue Experiments and Demonstration Toolkit.

The CRECIL Corpus: a New Dataset for Extraction of Relations between Characters in Chinese Multi-party Dialogues (2022)
Conference Proceeding
Jiang, Y., Xu, Y., Zhan, Y., He, W., Wang, Y., Xi, Z., …Yu, Y. (2022). The CRECIL Corpus: a New Dataset for Extraction of Relations between Characters in Chinese Multi-party Dialogues. In Proceedings of the Thirteenth Language Resources and Evaluation Conference (2337-2344)

We describe a new freely available Chinese multi-party dialogue dataset for automatic extraction of dialogue-based character relationships. The data has been extracted from the original TV scripts of a Chinese sitcom called “I Love My Home” with comp... Read More about The CRECIL Corpus: a New Dataset for Extraction of Relations between Characters in Chinese Multi-party Dialogues.

A Visually-Aware Conversational Robot Receptionist (2022)
Conference Proceeding
Gunson, N., Garcia, D. H., Sieińska, W., Addlesee, A., Dondrup, C., Lemon, O., …Yu, Y. (2022). A Visually-Aware Conversational Robot Receptionist. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue (645-648)

Socially Assistive Robots (SARs) have the potential to play an increasingly important role in a variety of contexts including healthcare, but most existing systems have very limited interactive capabilities. We will demonstrate a robot receptionist t... Read More about A Visually-Aware Conversational Robot Receptionist.

Combining Visual and Social Dialogue for Human-Robot Interaction (2021)
Conference Proceeding
Gunson, N., Hernandez Garcia, D., Part, J. L., Yu, Y., Sieińska, W., Dondrup, C., & Lemon, O. (2021). Combining Visual and Social Dialogue for Human-Robot Interaction. In ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction (841-842). https://doi.org/10.1145/3462244.3481303

We will demonstrate a prototype multimodal conversational AI system that will act as a receptionist in a hospital waiting room, combining visually-grounded dialogue with social conversation. The system supports visual object conversation in the waiti... Read More about Combining Visual and Social Dialogue for Human-Robot Interaction.

Coronabot: A conversational ai system for tackling misinformation (2021)
Conference Proceeding
Gunson, N., Sieińska, W., Yu, Y., Hernandez Garcia, D., Part, J. L., Dondrup, C., & Lemon, O. (2021). Coronabot: A conversational ai system for tackling misinformation. In GoodIT '21: Proceedings of the Conference on Information Technology for Social Good (265-270). https://doi.org/10.1145/3462203.3475874

Covid-19 has brought with it an onslaught of information for the public, some true and some false, across virtually every platform. For an individual, the task of sifting through the deluge for reliable, accurate facts is significant and potentially... Read More about Coronabot: A conversational ai system for tackling misinformation.

Towards visual dialogue for human-robot interaction (2021)
Conference Proceeding
Part, J. L., Hernández García, D., Yu, Y., Gunson, N., Dondrup, C., & Lemon, O. (2021). Towards visual dialogue for human-robot interaction. In HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (670-672). https://doi.org/10.1145/3434074.3447278

The goal of the EU H2020-ICT funded SPRING project is to develop a socially pertinent robot to carry out tasks in a gerontological healthcare unit. In this context, being able to perceive its environment and have coherent and relevant conversations a... Read More about Towards visual dialogue for human-robot interaction.

A comprehensive evaluation of incremental speech recognition and diarization for conversational AI (2020)
Conference Proceeding
Addlesee, A., Yu, Y., & Eshghi, A. (2020). A comprehensive evaluation of incremental speech recognition and diarization for conversational AI. In Proceedings of the 28th International Conference on Computational Linguistics (3492-3503)

Automatic Speech Recognition (ASR) systems are increasingly powerful and more accurate, but also more numerous with several options existing currently as a service (e.g. Google, IBM, and Microsoft). Currently the most stringent standards for such sys... Read More about A comprehensive evaluation of incremental speech recognition and diarization for conversational AI.

Optimising strategies for learning visually grounded word meanings through interaction (2018)
Thesis
Yu, Y. (2018). Optimising strategies for learning visually grounded word meanings through interaction. (Thesis)

Language Grounding is a fundamental problem in AI, regarding how symbols in Natural Language (e.g. words and phrases) refer to aspects of the physical environment (e.g. ob jects and attributes). In this thesis, our ultimate goal is to address an inte... Read More about Optimising strategies for learning visually grounded word meanings through interaction.

Incrementally learning semantic attributes through dialogue interaction (2018)
Conference Proceeding
Vanzo, A., Part, J. L., Yu, Y., Nardi, D., & Lemon, O. (2018). Incrementally learning semantic attributes through dialogue interaction. In AAMAS '18: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (865-873)

Enabling a robot to properly interact with users plays a key role in the effective deployment of robotic platforms in domestic environments. Robots must be able to rely on interaction to improve their behaviour and adaptively understand their operati... Read More about Incrementally learning semantic attributes through dialogue interaction.

The BURCHAK corpus: A challenge data set for interactive learning of visually grounded word meanings (2017)
Conference Proceeding
Yu, Y., Eshghi, A., Mills, G., & Lemon, O. J. (2017). The BURCHAK corpus: A challenge data set for interactive learning of visually grounded word meanings. In Proceedings of the Sixth Workshop on Vision and Language

We motivate and describe a new freely available human-human dialogue data set for interactive learning of visually grounded word meanings through ostensive definition by a tutor to a learner. The data has been collected using a novel, character-by-ch... Read More about The BURCHAK corpus: A challenge data set for interactive learning of visually grounded word meanings.

VOILA: An optimised dialogue system for interactively learning visually-grounded word meanings (demonstration system) (2017)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2017). VOILA: An optimised dialogue system for interactively learning visually-grounded word meanings (demonstration system). In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (197-200)

We present VOILA: an optimised, multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human user. VOILA is: (1) able to learn new visual categories interactively from users from scratch; (2) trained on real hum... Read More about VOILA: An optimised dialogue system for interactively learning visually-grounded word meanings (demonstration system).

Alana: Social dialogue using an ensemble model and a ranker trained on user feedback (2017)
Conference Proceeding
Papaioannou, I., Curry, A. C., Part, J. L., Shalyminov, I., Xu, X., Yu, Y., …Lemon, O. (2017). Alana: Social dialogue using an ensemble model and a ranker trained on user feedback. In 1st Proceedings of Alexa Prize

We describe our Alexa prize system (called ‘Alana’) which consists of an ensemble of bots, combining rule-based and machine learning systems, and using a contextual ranking mechanism to choose system responses. This paper reports on the version of th... Read More about Alana: Social dialogue using an ensemble model and a ranker trained on user feedback.

An ensemble model with ranking for social dialogue (2017)
Presentation / Conference
Papaioannou, I., Curry, A. C., Part, J. L., Shalyminov, I., Xu, X., Yu, Y., …Lemon, O. (2017, December). An ensemble model with ranking for social dialogue. Paper presented at NIPS 2017 Conversational AI Workshop, Long Beach, US

Open-domain social dialogue is one of the long-standing goals of Artificial Intelligence. This year, the Amazon Alexa Prize challenge was announced for the first time, where real customers get to rate systems developed by leading universities worldwi... Read More about An ensemble model with ranking for social dialogue.

Learning how to learn: An adaptive dialogue agent for incrementally learning visually grounded word meanings (2017)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2017). Learning how to learn: An adaptive dialogue agent for incrementally learning visually grounded word meanings. In Proceedings of the First Workshop on Language Grounding for Robotics

We present an optimised multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human tutor, trained on real human-human tutoring data. Within a life-long interactive learning period, the agent, trained using Rei... Read More about Learning how to learn: An adaptive dialogue agent for incrementally learning visually grounded word meanings.

Comparing dialogue strategies for learning grounded language from human tutors (2016)
Conference Proceeding
Yu, Y., Lemon, O., & Eshghi, A. (2016). Comparing dialogue strategies for learning grounded language from human tutors. In Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue (44-54)

We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanin... Read More about Comparing dialogue strategies for learning grounded language from human tutors.

An Incremental Dialogue System for Learning Visually Grounded Language (demonstration system) (2016)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2016). An Incremental Dialogue System for Learning Visually Grounded Language (demonstration system). In Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue (120-121)

We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor. The system integrates an incremental, semantic, and bi-directional grammar framework – Dynamic Syntax and Type Theory with Re... Read More about An Incremental Dialogue System for Learning Visually Grounded Language (demonstration system).

Training an adaptive dialogue policy for interactive learning of visually grounded word meanings (2016)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2016). Training an adaptive dialogue policy for interactive learning of visually grounded word meanings. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue (339-349). https://doi.org/10.18653/v1/w16-3643

We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor. The system integrates an incremental, semantic parsing/generation framework - Dynamic Syntax and Type Theory with Records (DS... Read More about Training an adaptive dialogue policy for interactive learning of visually grounded word meanings.

Interactively learning visually grounded word meanings from a human tutor (2016)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2016). Interactively learning visually grounded word meanings from a human tutor. In Proceedings of the 5th Workshop on Vision and Language (48-53)

We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor. The system integrates an incremental, semantic parsing/generation framework - Dynamic Syntax and Type Theory with Records (DS... Read More about Interactively learning visually grounded word meanings from a human tutor.

Information density and overlap in spoken dialogue (2015)
Journal Article
Dethlefs, N., Hastie, H., Cuayáhuitl, H., Yu, Y., Rieser, V., & Lemon, O. (2016). Information density and overlap in spoken dialogue. Computer Speech and Language, 37, 82-97. https://doi.org/10.1016/j.csl.2015.11.001

Incremental dialogue systems are often perceived as more responsive and natural because they are able to address phenomena of turn-taking and overlapping speech, such as backchannels or barge-ins. Previous work in this area has often identified disti... Read More about Information density and overlap in spoken dialogue.

Comparing attribute classifiers for interactive language grounding (2015)
Conference Proceeding
Yu, Y., Eshghi, A., & Lemon, O. (2015). Comparing attribute classifiers for interactive language grounding. In Proceedings of the Fourth Workshop on Vision and Language (60-69)

We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. We design a semantic and visual processing system to support this and illustrate how they can be integrated. We then focus on compar... Read More about Comparing attribute classifiers for interactive language grounding.

Two Alternative Frameworks for Deploying Spoken Dialogue Systems to Mobile Platforms for Evaluation “In the Wild” (2014)
Conference Proceeding
Hastie, H., Aufaure, M., Alexopoulos, P., Bouchard, H., Cuayáhuitl, H., Dethlefs, N., …Yu, Y. (2014). Two Alternative Frameworks for Deploying Spoken Dialogue Systems to Mobile Platforms for Evaluation “In the Wild”. In Proceedings of the 18th Workshop on the Semantics and Pragmatics of Dialogue (191)

We demonstrate two alternative frameworks for testing and evaluating spoken dialogue systems on mobile devices for use “in the wild”. We firstly present a spoken dialogue system that uses third party ASR (Automatic Speech Recognition) and TTS (Text-T... Read More about Two Alternative Frameworks for Deploying Spoken Dialogue Systems to Mobile Platforms for Evaluation “In the Wild”.

SpeechCity: A Conversational City Guide based on Open Data (2014)
Conference Proceeding
Rieser, V., Janarthanam, S., Taylor, A., Yu, Y., & Lemon, O. (2014). SpeechCity: A Conversational City Guide based on Open Data. In Proceedings of the 18th Workshop on the Semantics and Pragmatics of Dialogue - Poster Abstracts (234)

The PARLANCE mobile application for interactive search in English and Mandarin (2014)
Conference Proceeding
Hastie, H., Aufaure, M., Alexopoulos, P., Bouchard, H., Breslin, C., Cuayáhuitl, H., …Yu, Y. (2014). The PARLANCE mobile application for interactive search in English and Mandarin. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial) (260-262)

We demonstrate a mobile application in English and Mandarin to test and evaluate components of the Parlance dialogue system for interactive search under real-world conditions.

Explainable Representations of the Social State: A Model for Social Human-Robot Interactions
Working Paper
Hernandez Garcia, D., Yu, Y., Sieinska, W., Part, J. L., Gunson, N., Lemon, O., & Dondrup, C. (2020). Explainable Representations of the Social State: A Model for Social Human-Robot Interactions

In this paper, we propose a minimum set of concepts and signals needed to track the social state during Human-Robot Interaction. We look into the problem of complex continuous interactions in a social context with multiple humans and robots, and disc... Read More about Explainable Representations of the Social State: A Model for Social Human-Robot Interactions.