Skip to main content

Research Repository

Advanced Search

Outputs (49)

Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation (2024)
Presentation / Conference Contribution
Watson, L. N., & Gkatzia, D. (2024, May). Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation. Presented at HumEval2024 at LREC-COLING 2024, Turin, Italy

Reproducibility is a cornerstone of scientific research, ensuring the reliability and generalisability of findings. The ReproNLP Shared Task on Reproducibility of Evaluations in NLP aims to assess the reproducibility of human evaluation studies. This... Read More about Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.

TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction (2023)
Presentation / Conference Contribution
Strathearn, C., Yu, Y., & Gkatzia, D. (2023). TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction. In Proceedings of The Joint CUI and HRI Workshop at HRI 2023

The most effective way of communication between humans and robots is through natural language communication. However, there are many challenges to overcome before robots can effectively converse in order to collaborate and work together with humans.... Read More about TaskMaster: A Novel Cross-platform Task-based Spoken Dialogue System for Human-Robot Interaction.

Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge (2023)
Presentation / Conference Contribution
Watson, L., & Gkatzia, D. (2023). Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge. In Proceedings of the 3rd Workshop on Human Evaluation of NLP Systems (69-74)

Human evaluation is crucial for NLG systems as it provides a reliable assessment of the quality, effectiveness, and utility of generated language outputs. However, concerns about the reproducibility of such evaluations have emerged, casting doubt on... Read More about Unveiling NLG Human-Evaluation Reproducibility: Lessons Learned and Key Insights from Participating in the ReproNLP Challenge.

Working with troubles and failures in conversation between humans and robots: workshop report (2023)
Journal Article
Förster, F., Romeo, M., Holthaus, P., Wood, L. J., Dondrup, C., Fischer, J. E., …Kapetanios, E. (2023). Working with troubles and failures in conversation between humans and robots: workshop report. Frontiers in Robotics and AI, 10, Article 1202306. ht

This paper summarizes the structure and findings from the first Workshop on Troubles and Failures in Conversations between Humans and Robots. The workshop was organized to bring together a small, interdisciplinary group of researchers working on misc... Read More about Working with troubles and failures in conversation between humans and robots: workshop report.

enunlg: a Python library for reproducible neural data-to-text experimentation (2023)
Presentation / Conference Contribution
Howcroft, D. M., & Gkatzia, D. (2023). enunlg: a Python library for reproducible neural data-to-text experimentation. In Proceedings of the 16th International Natural Language Generation Conference: System Demonstrations (4-5)

Over the past decade, a variety of neural ar-chitectures for data-to-text generation (NLG) have been proposed. However, each system typically has its own approach to pre-and post-processing and other implementation details. Diversity in implementatio... Read More about enunlg: a Python library for reproducible neural data-to-text experimentation.

LOWRECORP: the Low-Resource NLG Corpus Building Challenge (2023)
Presentation / Conference Contribution
Chandu, K. R., Howcroft, D., Gkatzia, D., Chung, Y., Hou, Y., Emezue, C., …Adewumi, T. (2023). LOWRECORP: the Low-Resource NLG Corpus Building Challenge. In The 16th International Natural Language Generation Conference: Generation Challenges (1-9)

Most languages in the world do not have sufficient data available to develop neural-network-based natural language generation (NLG) systems. To alleviate this resource scarcity, we propose a novel challenge for the NLG community: low-resource languag... Read More about LOWRECORP: the Low-Resource NLG Corpus Building Challenge.

Edge NLP for Efficient Machine Translation in Low Connectivity Areas (2023)
Presentation / Conference Contribution
Watt, T., Chrysoulas, C., & Gkatzia, D. (2023, October). Edge NLP for Efficient Machine Translation in Low Connectivity Areas. Presented at IEEE 9th World Forum on Internet of Things: 2nd Workshop on Convergence of Edge Intelligence in IoT (EdgeAI-IoT 202

Machine translation (MT) usually requires connectivity and access to the cloud which is often limited in many parts of the world, including hard to reach rural areas. Edge natural language processing (NLP) aims to solve this problem by processing lan... Read More about Edge NLP for Efficient Machine Translation in Low Connectivity Areas.

Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic) (2023)
Presentation / Conference Contribution
Howcroft, D. M., Lamb, W., Groundwater, A., & Gkatzia, D. (2023, September). Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic). Presented at The 16th International Natural Language Generation

Gàidhlig (Scottish Gaelic; gd) is spoken by about 57k people in Scotland, but remains an under-resourced language with respect to natural language processing in general and natural language generation (NLG) in particular. To address this gap, we deve... Read More about Building a dual dataset of text-and image-grounded conversations and summarisation in Gàidhlig (Scottish Gaelic).

Responsible Design & Evaluation of a Conversational Agent for a National Careers Service (2023)
Presentation / Conference Contribution
Wilson, M., Cruickshank, P., Gkatzia, D., & Robertson, P. (in press). Responsible Design & Evaluation of a Conversational Agent for a National Careers Service.

This PhD project applies a research-through-design approach to the development of a conversational agent for a national career service for young people. This includes addressing practical, interactional and ethical aspects of the system. For each asp... Read More about Responsible Design & Evaluation of a Conversational Agent for a National Careers Service.