Skip to main content

Research Repository

Advanced Search

All Outputs (43)

An Open Intent Discovery Evaluation Framework (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). An Open Intent Discovery Evaluation Framework. Presented at SIGDIAL 2024, Kyoto, Japan

Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot (2024)
Presentation / Conference Contribution
Wilson, M., Brazier, D., Gkatzia, D., & Robertson, P. (2024, July). Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot. Presented at ACM Conversational User Interfaces 2024 (CUI ’24), Luxembourg, Luxembourg

We present a study of collaboration with expert participants for the purpose of the responsible design of a conversational agent. The Delphi study was used to identify and develop design and evaluation criteria for an automated career support interve... Read More about Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot.

Automated Human-Readable Label Generation in Open Intent Discovery (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). Automated Human-Readable Label Generation in Open Intent Discovery. Presented at Interspeech 2024, Kos, Greece

The correct determination of user intent is key in dialog systems. However, an intent classifier often requires a large, labelled training dataset to identify a set of known intents. The creation of such a dataset is a complex and time-consuming task... Read More about Automated Human-Readable Label Generation in Open Intent Discovery.

Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation (2024)
Presentation / Conference Contribution
Watson, L. N., & Gkatzia, D. (2024, May). Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation. Presented at HumEval2024 at LREC-COLING 2024, Turin, Italy

Reproducibility is a cornerstone of scientific research, ensuring the reliability and generalisability of findings. The ReproNLP Shared Task on Reproducibility of Evaluations in NLP aims to assess the reproducibility of human evaluation studies. This... Read More about Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.

Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge (2023)
Presentation / Conference Contribution
Watson, L., & Gkatzia, D. (2023). Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge. In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems (69-74)

Human evaluation is crucial for NLG systems as it provides a reliable assessment of the quality, effectiveness, and utility of generated language outputs. However, concerns about the reproducibility of such evaluations have emerged, casting doubt on... Read More about Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge.

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction (2023)
Presentation / Conference Contribution
Strathearn, C., Yu, Y., & Gkatzia, D. (2023). TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction. In Proceedings of The Joint CUI and HRI Workshop at HRI 2023

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans.... Read More about TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction.

Edge NLP for Efficient Machine Translation in Low Connectivity Areas (2023)
Presentation / Conference Contribution
Watt, T., Chrysoulas, C., & Gkatzia, D. (2023, October). Edge NLP for Efficient Machine Translation in Low Connectivity Areas. Presented at IEEE 9th World Forum on Internet of Things: 2nd Workshop on Convergence of Edge Intelligence in IoT (EdgeAI-IoT 2023), Aveiro, Portugal

Machine translation (MT) usually requires connectivity and access to the cloud which is often limited in many parts of the world, including hard to reach rural areas. Edge natural language processing (NLP) aims to solve this problem by processing lan... Read More about Edge NLP for Efficient Machine Translation in Low Connectivity Areas.

Responsible Design & Evaluation of a Conversational Agent for a National Careers Service (2023)
Presentation / Conference Contribution
Wilson, M., Cruickshank, P., Gkatzia, D., & Robertson, P. (2023, September). Responsible Design & Evaluation of a Conversational Agent for a National Careers Service. Presented at Symposium on Future Directions in Information Access (FDIA) 2023, Vienna, Austria

This PhD project applies a research-through-design approach to the development of a conversational agent for a national career service for young people. This includes addressing practical, interactional and ethical aspects of the system. For each asp... Read More about Responsible Design & Evaluation of a Conversational Agent for a National Careers Service.

Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic) (2023)
Presentation / Conference Contribution
Howcroft, D. M., Lamb, W., Groundwater, A., & Gkatzia, D. (2023, September). Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic). Presented at The 16th International Natural Language Generation Conference

Gàidhlig (Scottish Gaelic; gd) is spoken by about 57k people in Scotland, but remains an under-resourced language with respect to natural language processing in general and natural language generation (NLG) in particular. To address this gap, we deve... Read More about Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic).

LOWRECORP: the Low-Resource NLG Corpus Building Challenge (2023)
Presentation / Conference Contribution
Chandu, K. R., Howcroft, D., Gkatzia, D., Chung, Y., Hou, Y., Emezue, C., Rajpoot, P., & Adewumi, T. (2023, September). LOWRECORP: the Low-Resource NLG Corpus Building Challenge. Presented at 16th International Natural Language Generation Conference, Prague, Czechia

Most languages in the world do not have sufficient data available to develop neural-network-based natural language generation (NLG) systems. To alleviate this resource scarcity, we propose a novel challenge for the NLG community: low-resource languag... Read More about LOWRECORP: the Low-Resource NLG Corpus Building Challenge.

enunlg: a Python library for reproducible neural data-to-text experimentation (2023)
Presentation / Conference Contribution
Howcroft, D. M., & Gkatzia, D. (2023). enunlg: a Python library for reproducible neural data-to-text experimentation. In Proceedings of the 16th International Natural Language Generation Conference: System Demonstrations (4-5)

Over the past decade, a variety of neural ar-chitectures for data-to-text generation (NLG) have been proposed. However, each system typically has its own approach to pre-and post-processing and other implementation details. Diversity in implementatio... Read More about enunlg: a Python library for reproducible neural data-to-text experimentation.

Most NLG is Low-Resource: here's what we can do about it (2022)
Presentation / Conference Contribution
Howcroft, D. M., & Gkatzia, D. (2022). Most NLG is Low-Resource: here's what we can do about it. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) (336-350)

Many domains and tasks in natural language generation (NLG) are inherently 'low-resource', where training data, tools and linguistic analyses are scarce. This poses a particular challenge to researchers and system developers in the era of machine-lea... Read More about Most NLG is Low-Resource: here's what we can do about it.

Multi3Generation: Multi-task, Multilingual, Multi-Modal Language Generation (2022)
Presentation / Conference Contribution
Barreiro, A., de Souza, J. G., Gatt, A., Bhatt, M., Lloret, E., Erdem, A., Gkatzia, D., Moniz, H., Russo, I., Kepler, F., Calixto, I., Paprzycki, M., Portet, F., Augenstein, I., & Alhasani, M. (2022, June). Multi3Generation: Multi-task, Multilingual, Multi-Modal Language Generation. Poster presented at 23rd Annual Conference of the European Association for Machine Translation (EAMT 2022), Ghent, Belgium

This paper presents the Multitask, Multilingual, Multimodal Language Generation COST Action – Multi3Generation (CA18231), an interdisciplinary network of research groups working on different aspects of language generation. This "metapaper" will serve... Read More about Multi3Generation: Multi-task, Multilingual, Multi-Modal Language Generation.

Underreporting of errors in NLG output, and what to do about it (2021)
Presentation / Conference Contribution
van Miltenburg, E., Clinciu, M., Dušek, O., Gkatzia, D., Inglis, S., Leppänen, L., Mahamood, S., Manning, E., Schoch, S., Thomson, C., & Wen, L. (2021, September). Underreporting of errors in NLG output, and what to do about it. Presented at 14th International Conference on Natural Language Generation, Aberdeen, UK

We observe a severe under-reporting of the different kinds of errors that Natural Language Generation systems make. This is a problem, because mistakes are an important indicator of where systems should still be improved. If authors only report overa... Read More about Underreporting of errors in NLG output, and what to do about it.

The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents (2021)
Presentation / Conference Contribution
Strathearn, C., & Gkatzia, D. (2021, November). The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents. Presented at 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021), Trento, Italy [Online]

This paper describes the Task2Dial dataset, a novel dataset of document-grounded task-based dialogues in the food preparation domain , where an Information Giver (IG) provides instructions to an Information Follower (IF) so that the latter can succes... Read More about The Task2Dial Dataset: A Novel Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents.

Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems (2021)
Presentation / Conference Contribution
Strathearn, C., & Gkatzia, D. (2021). Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems. In Proceedings of the 14th International Conference on Natural Language Generation (46-47)

Conversational systems aim to generate responses that are accurate, relevant and engaging, either through utilising neural end-to-end models or through slot filling. Human-to-human conversations are enhanced by not only the latest utterance of the in... Read More about Chefbot: A Novel Framework for the Generation of Commonsense-enhanced Responses for Task-based Dialogue Systems.

CAPE: Context-Aware Private Embeddings for Private Language Learning (2021)
Presentation / Conference Contribution
Plant, R., Gkatzia, D., & Giuffrida, V. (2021). CAPE: Context-Aware Private Embeddings for Private Language Learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (7970-7978)

Neural language models have contributed to state-of-the-art results in a number of downstream applications including sentiment analysis, intent classification and others. However, obtaining text representations or embeddings using these models risks... Read More about CAPE: Context-Aware Private Embeddings for Private Language Learning.