Outputs (58)

Participatory Design for Positive Impact: Behind the Scenes of Three NLP Projects (2025)
Presentation / Conference Contribution
Wilson, M., Howcroft, D. M., Konstas, I., Gkatzia, D., & Abercrombie, G. (2025, July). Participatory Design for Positive Impact: Behind the Scenes of Three NLP Projects. Presented at ACL 2025 Workshop on NLP for Positive Impact (NLP4PI 2025), Vienna

Researchers in Natural Language Processing (NLP) are increasingly adopting participatory design (PD) principles to better achieve positive outcomes for stakeholders. This paper evaluates two PD perspectives proposed by Delgado et al. (2023) and Casel... Read More about Participatory Design for Positive Impact: Behind the Scenes of Three NLP Projects.

From documents to dialogue: Context matters in common sense-enhanced task-based dialogue grounded in documents (2025)
Journal Article
Strathearn, C., Gkatzia, D., & Yu, Y. (2025). From documents to dialogue: Context matters in common sense-enhanced task-based dialogue grounded in documents. Expert Systems with Applications, 279, Article 127304. https://doi.org/10.1016/j.eswa.2025.127304

Humans can engage in a conversation to collaborate on multi-step tasks and divert briefly to complete essential sub-tasks, such as asking for confirmation or clarification, before resuming the overall task. This communication is necessary as some kno... Read More about From documents to dialogue: Context matters in common sense-enhanced task-based dialogue grounded in documents.

Evaluating Language Model Vulnerability to Poisoning Attacks in Low-Resource Settings (2024)
Journal Article
Plant, R., Giuffrida, M. V., Pitropakis, N., & Gkatzia, D. (2024). Evaluating Language Model Vulnerability to Poisoning Attacks in Low-Resource Settings. IEEE/ACM Transactions on Audio, Speech and Language Processing, 33, 54-67. https://doi.org/10.1109/taslp.2024.3507565

Pre-trained language models are a highly effective source of knowledge transfer for natural language processing tasks, as their development represents an investment of resources beyond the reach of most researchers and end users. The widespread avail... Read More about Evaluating Language Model Vulnerability to Poisoning Attacks in Low-Resource Settings.

You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models (2024)
Journal Article
Plant, R., Giuffrida, V., & Gkatzia, D. (2025). You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models. Computer Speech and Language, 90, Article 101746. https://doi.org/10.1016/j.csl.2024.101746

The widespread use of pre-trained language models has revolutionised knowledge transfer in natural language processing tasks. However, there is a concern regarding potential breaches of user trust due to the risk of re-identification attacks, where m... Read More about You Are What You Write: Author re-identification privacy attacks in the era of pre-trained language models.

Exploring the impact of data representation on neural data-to-text generation (2024)
Presentation / Conference Contribution
Howcroft, D. M., Watson, L. N., Nedopas, O., & Gkatzia, D. (2024, September). Exploring the impact of data representation on neural data-to-text generation. Presented at INLG 2024, Tokyo, Japan

A relatively under-explored area in research on neural natural language generation is the impact of the data representation on text quality. Here we report experiments on two leading input representations for data-to-text generation: attribute-value... Read More about Exploring the impact of data representation on neural data-to-text generation.

Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices (2024)
Presentation / Conference Contribution
Schmidtova, P., Mahamood, S., Balloccu, S., Dusek, O., Gatt, A., Gkatzia, D., Howcroft, D. M., Platek, O., & Sivaprasad, A. (2024, September). Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices. Presented at INLG 2024, Tokyo, Japan

Automatic metrics are extensively used to evaluate Natural Language Processing systems. However, there has been increasing focus on how the are used and reported by practitioners within the field. In this paper, we have conducted a survey on the use... Read More about Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices.

An Open Intent Discovery Evaluation Framework (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). An Open Intent Discovery Evaluation Framework. Presented at SIGDIAL 2024, Kyoto, Japan

In the development of dialog systems the discovery of the set of target intents to identify is a crucial first step that is often overlooked. Most intent detection works assume that a labelled dataset already exists, however creating these datasets i... Read More about An Open Intent Discovery Evaluation Framework.

Automated Human-Readable Label Generation in Open Intent Discovery (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). Automated Human-Readable Label Generation in Open Intent Discovery. Presented at Interspeech 2024, Kos, Greece

The correct determination of user intent is key in dialog systems. However, an intent classifier often requires a large, labelled training dataset to identify a set of known intents. The creation of such a dataset is a complex and time-consuming task... Read More about Automated Human-Readable Label Generation in Open Intent Discovery.

Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot (2024)
Presentation / Conference Contribution
Wilson, M., Brazier, D., Gkatzia, D., & Robertson, P. (2024, July). Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot. Presented at ACM Conversational User Interfaces 2024 (CUI ’24), Luxembourg, Luxembourg

We present a study of collaboration with expert participants for the purpose of the responsible design of a conversational agent. The Delphi study was used to identify and develop design and evaluation criteria for an automated career support interve... Read More about Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot.

Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation (2024)
Presentation / Conference Contribution
Watson, L. N., & Gkatzia, D. (2024, May). Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation. Presented at HumEval2024 at LREC-COLING 2024, Turin, Italy

Reproducibility is a cornerstone of scientific research, ensuring the reliability and generalisability of findings. The ReproNLP Shared Task on Reproducibility of Evaluations in NLP aims to assess the reproducibility of human evaluation studies. This... Read More about Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.