CiViL: Common-sense- and Visual-enhanced natural Language generation

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction (2023)
Presentation / Conference Contribution
Strathearn, C., Yu, Y., & Gkatzia, D. (2023, March). TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction. Presented at 'HRCI23, Stockholm, Sweden

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans.... Read More about TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction.

Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge (2023)
Presentation / Conference Contribution
Watson, L., & Gkatzia, D. (2023, September). Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge. Presented at 3rd Workshop on Human Evaluation of NLP Systems (HumEval), Varna, Bulgaria

Human evaluation is crucial for NLG systems as it provides a reliable assessment of the quality, effectiveness, and utility of generated language outputs. However, concerns about the reproducibility of such evaluations have emerged, casting doubt on... Read More about Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge.

Most NLG is Low-Resource: here's what we can do about it (2022)
Presentation / Conference Contribution
Howcroft, D. M., & Gkatzia, D. (2022, December). Most NLG is Low-Resource: here's what we can do about it. Presented at Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), Abu Dhabi, UAE

Many domains and tasks in natural language generation (NLG) are inherently 'low-resource', where training data, tools and linguistic analyses are scarce. This poses a particular challenge to researchers and system developers in the era of machine-lea... Read More about Most NLG is Low-Resource: here's what we can do about it.

Underreporting of errors in NLG output, and what to do about it (2021)
Presentation / Conference Contribution
van Miltenburg, E., Clinciu, M.-A., Dušek, O., Gkatzia, D., Inglis, S., Leppänen, L., Mahamood, S., Manning, E., Schoch, S., Thomson, C., & Wen, L. (2021, September). Underreporting of errors in NLG output, and what to do about it. Presented at 14th International Conference on Natural Language Generation, Aberdeen, UK

We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overa... Read More about Underreporting of errors in NLG output, and what to do about it.

Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems (2021)
Presentation / Conference Contribution
Strathearn, C., & Gkatzia, D. (2021, August). Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems. Presented at 14th International Conference on Natural Language Generation, Aberdeen

Conversational systems aim to generate responses that are accurate, relevant and engaging, either through utilising neural end-to-end models or through slot filling. Human-to-human conversations are enhanced by not only the latest utterance of the in... Read More about Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems.

It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems (2021)
Presentation / Conference Contribution
Mahamood, S., Clinciu, M., & Gkatzia, D. (2021, April). It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems. Presented at Workshop on Human Evaluation of NLP Systems (HumEval at EACL 2021), Kyiv, Ukraine (online)

Common sense is an integral part of human cognition which allows us to make sound decisions , communicate effectively with others and interpret situations and utterances. Endowing AI systems with commonsense knowledge capabilities will help us get cl... Read More about It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems.

"What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI (2021)
Presentation / Conference Contribution
Belvedere, F., & Gkatzia, D. (2021, March). "What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI. Presented at HRI'21: ACM/IEEE International Conference on Human-Robot Interaction, Online

Social robotics aim to equip robots with the ability to exhibit socially intelligent behaviour while interacting in a face-to-face context with human partners. An important aspect of face-to-face social interaction includes the efficient recognition... Read More about "What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI.

Second Workshop on Natural Language Generation for Human-Robot Interaction (2020)
Presentation / Conference Contribution
Buschmeier, H., Ellen Foster, M., & Gkatzia, D. (2020, March). Second Workshop on Natural Language Generation for Human-Robot Interaction. Presented at HRI '20: ACM/IEEE International Conference on Human-Robot Interaction, Cambridge

This workshop is the second in a series bringing together the Natural Language Generation and Human-Robot Interaction communities to discuss topics of mutual interest with the goal of developing an HRI-inspired NLG shared task. The workshop website i... Read More about Second Workshop on Natural Language Generation for Human-Robot Interaction.

Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition (2020)
Presentation / Conference Contribution
Howcroft, D., Belz, A., Clinciu, M., Gkatzia, D., Hasan, S. A., Mahamood, S., Mille, S., van Miltenburg, E., Santhanam, S., & Rieser, V. (2020, December). Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition. Presented at International Conference on Natural Language Generation (INLG 2020), Dublin, Ireland

Human assessment remains the most trusted form of evaluation in NLG, but highly diverse approaches and a proliferation of different quality criteria used by researchers make it difficult to compare results and draw conclusions across papers, with adv... Read More about Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition.

Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training (2020)
Presentation / Conference Contribution
Panagiaris, N., Hart, E., & Gkatzia, D. (2020, December). Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training. Presented at International Conference on Natural Language Generation (INLG 2020), Dublin, Ireland

In this paper we consider the problem of optimizing neural Referring Expression Generation (REG) models with sequence level objectives. Recently reinforcement learning (RL) techniques have been adopted to train deep end-to-end systems to directly opt... Read More about Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training.