All Outputs (36)

Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation (2024)
Conference Proceeding
Watson, L. N., & Gkatzia, D. (in press). Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.

Reproducibility is a cornerstone of scientific research, ensuring the reliability and generalisability of findings. The ReproNLP Shared Task on Reproducibility of Evaluations in NLP aims to assess the reproducibility of human evaluation studies. This... Read More about Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.

Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge (2023)
Conference Proceeding
Watson, L., & Gkatzia, D. (2023). Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge. In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems (69-74)

Human evaluation is crucial for NLG systems as it provides a reliable assessment of the quality, effectiveness, and utility of generated language outputs. However, concerns about the reproducibility of such evaluations have emerged, casting doubt on... Read More about Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge.

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction (2023)
Conference Proceeding
Strathearn, C., Yu, Y., & Gkatzia, D. (2023). TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction. In Proceedings of The Joint CUI and HRI Workshop at HRI 2023

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans.... Read More about TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction.

enunlg: a Python library for reproducible neural data-to-text experimentation (2023)
Conference Proceeding
Howcroft, D. M., & Gkatzia, D. (2023). enunlg: a Python library for reproducible neural data-to-text experimentation. In Proceedings of the 16th International Natural Language Generation Conference: System Demonstrations (4-5)

Over the past decade, a variety of neural ar-chitectures for data-to-text generation (NLG) have been proposed. However, each system typically has its own approach to pre-and post-processing and other implementation details. Diversity in implementatio... Read More about enunlg: a Python library for reproducible neural data-to-text experimentation.

Edge NLP for Efficient Machine Translation in Low Connectivity Areas (2023)
Conference Proceeding
Watt, T., Chrysoulas, C., & Gkatzia, D. (in press). Edge NLP for Efficient Machine Translation in Low Connectivity Areas.

Machine translation (MT) usually requires connectivity and access to the cloud which is often limited in many parts of the world, including hard to reach rural areas. Edge natural language processing (NLP) aims to solve this problem by processing lan... Read More about Edge NLP for Efficient Machine Translation in Low Connectivity Areas.

LOWRECORP: the Low-Resource NLG Corpus Building Challenge (2023)
Conference Proceeding
Chandu, K. R., Howcroft, D., Gkatzia, D., Chung, Y., Hou, Y., Emezue, C., …Adewumi, T. (2023). LOWRECORP: the Low-Resource NLG Corpus Building Challenge. In The 16th International Natural Language Generation Conference: Generation Challenges (1-9)

Most languages in the world do not have sufficient data available to develop neural-network-based natural language generation (NLG) systems. To alleviate this resource scarcity, we propose a novel challenge for the NLG community: low-resource languag... Read More about LOWRECORP: the Low-Resource NLG Corpus Building Challenge.

Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic) (2023)
Conference Proceeding
Howcroft, D. M., Lamb, W., Groundwater, A., & Gkatzia, D. (2023). Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic). In Proceedings of the 16th International Natural Language Generation Conference (443-448)

Gàidhlig (Scottish Gaelic; gd) is spoken by about 57k people in Scotland, but remains an under-resourced language with respect to natural language processing in general and natural language generation (NLG) in particular. To address this gap, we deve... Read More about Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic).

Responsible Design & Evaluation of a Conversational Agent for a National Careers Service (2023)
Conference Proceeding
Wilson, M., Cruickshank, P., Gkatzia, D., & Robertson, P. (in press). Responsible Design & Evaluation of a Conversational Agent for a National Careers Service.

This PhD project applies a research-through-design approach to the development of a conversational agent for a national career service for young people. This includes addressing practical, interactional and ethical aspects of the system. For each asp... Read More about Responsible Design & Evaluation of a Conversational Agent for a National Careers Service.

Most NLG is Low-Resource: here's what we can do about it (2022)
Conference Proceeding
Howcroft, D. M., & Gkatzia, D. (2022). Most NLG is Low-Resource: here's what we can do about it. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) (336-350)

Many domains and tasks in natural language generation (NLG) are inherently 'low-resource', where training data, tools and linguistic analyses are scarce. This poses a particular challenge to researchers and system developers in the era of machine-lea... Read More about Most NLG is Low-Resource: here's what we can do about it.

Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems (2021)
Conference Proceeding
Strathearn, C., & Gkatzia, D. (2021). Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems. In Proceedings of the 14th International Conference on Natural Language Generation (46-47)

Conversational systems aim to generate responses that are accurate, relevant and engaging, either through utilising neural end-to-end models or through slot filling. Human-to-human conversations are enhanced by not only the latest utterance of the in... Read More about Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems.

The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents (2021)
Conference Proceeding
Strathearn, C., & Gkatzia, D. (2021). The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents. In Proceedings of The Fourth International Conference on Natural Language and Speech Processing (ICNLSP 2021) (242-251)

This paper describes the Task2Dial dataset, a novel dataset of document-grounded task-based dialogues in the food preparation domain , where an Information Giver (IG) provides instructions to an Information Follower (IF) so that the latter can succes... Read More about The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents.

Underreporting of errors in NLG output, and what to do about it (2021)
Conference Proceeding
van Miltenburg, E., Clinciu, M., Dušek, O., Gkatzia, D., Inglis, S., Leppänen, L., …Wen, L. (2021). Underreporting of errors in NLG output, and what to do about it. In Proceedings of the 14th International Conference on Natural Language Generation (140-153)

We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overa... Read More about Underreporting of errors in NLG output, and what to do about it.

CAPE: Context-Aware Private Embeddings for Private Language Learning (2021)
Conference Proceeding
Plant, R., Gkatzia, D., & Giuffrida, V. (2021). CAPE: Context-Aware Private Embeddings for Private Language Learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (7970-7978)

Neural language models have contributed to state-of-the-art results in a number of downstream applications including sentiment analysis, intent classification and others. However, obtaining text representations or embeddings using these models risks... Read More about CAPE: Context-Aware Private Embeddings for Private Language Learning.

It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems (2021)
Conference Proceeding
Mahamood, S., Clinciu, M., & Gkatzia, D. (2021). It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems. In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval)

Common sense is an integral part of human cognition which allows us to make sound decisions , communicate effectively with others and interpret situations and utterances. Endowing AI systems with commonsense knowledge capabilities will help us get cl... Read More about It's Common Sense, isn't it? Demystifying Human Evaluations in Commonsense-enhanced NLG systems.

"What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI (2021)
Conference Proceeding
Belvedere, F., & Gkatzia, D. (2021). "What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI. In HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (205-209). https://doi.org/10.1145/3434074.3447160

Social robotics aim to equip robots with the ability to exhibit socially intelligent behaviour while interacting in a face-to-face context with human partners. An important aspect of face-to-face social interaction includes the efficient recognition... Read More about "What's this?" Comparing Active learning Strategies for Concept Acquisition in HRI.

Commonsense-enhanced Natural Language Generation for Human-Robot Interaction (2020)
Conference Proceeding
Gkatzia, D. (2020). Commonsense-enhanced Natural Language Generation for Human-Robot Interaction. In 2nd Workshop on Natural Language Generation for Human-Robot Interaction (HRI 2020)

Commonsense is vital for human communication, as it allows us to make inferences without explicitly mentioning the context. Equipping robots with commonsense knowledge would lead to better communication between humans and robots and will allow robots... Read More about Commonsense-enhanced Natural Language Generation for Human-Robot Interaction.

Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition (2020)
Conference Proceeding
Howcroft, D., Belz, A., Clinciu, M., Gkatzia, D., Hasan, S. A., Mahamood, S., …Rieser, V. (2020). Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition. In Proceedings of the 13th International Conference on Natural Language Generation (169-182)

Human assessment remains the most trusted form of evaluation in NLG, but highly diverse approaches and a proliferation of different quality criteria used by researchers make it difficult to compare results and draw conclusions across papers, with adv... Read More about Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definition.

Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training (2020)
Conference Proceeding
Panagiaris, N., Hart, E., & Gkatzia, D. (2020). Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training. In Proceedings of the 13th International Conference on Natural Language Generation (41-51)

In this paper we consider the problem of optimizing neural Referring Expression Generation (REG) models with sequence level objectives. Recently reinforcement learning (RL) techniques have been adopted to train deep end-to-end systems to directly opt... Read More about Improving the Naturalness and Diversity of Referring Expression Generation models using Minimum Risk Training.

Second Workshop on Natural Language Generation for Human-Robot Interaction (2020)
Conference Proceeding
Buschmeier, H., Ellen Foster, M., & Gkatzia, D. (2020). Second Workshop on Natural Language Generation for Human-Robot Interaction. In HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (646-647). https://doi.org/10.1145/3371382.3374853

This workshop is the second in a series bringing together the Natural Language Generation and Human-Robot Interaction communities to discuss topics of mutual interest with the goal of developing an HRI-inspired NLG shared task. The workshop website i... Read More about Second Workshop on Natural Language Generation for Human-Robot Interaction.

Proceedings of the Workshop on NLG for Human–Robot Interaction (2018)
Conference Proceeding
(2018). Proceedings of the Workshop on NLG for Human–Robot Interaction. In M. Ellen Foster, H. Buschmeier, & D. Gkatzia (Eds.),

Ellen Foster, M., H. Buschmeier, & D. Gkatzia (Eds.) (2018). Proceedings of the Workshop on NLG for Human–Robot Interaction.