Dr Dimitra Gkatzia's Outputs (6)

Exploring the impact of data representation on neural data-to-text generation (2024)
Presentation / Conference Contribution
Howcroft, D. M., Watson, L. N., Nedopas, O., & Gkatzia, D. (2024, September). Exploring the impact of data representation on neural data-to-text generation. Presented at INLG 2024, Tokyo, Japan

A relatively under-explored area in research on neural natural language generation is the impact of the data representation on text quality. Here we report experiments on two leading input representations for data-to-text generation: attribute-value... Read More about Exploring the impact of data representation on neural data-to-text generation.

Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices (2024)
Presentation / Conference Contribution
Schmidtova, P., Mahamood, S., Balloccu, S., Dusek, O., Gatt, A., Gkatzia, D., Howcroft, D. M., Platek, O., & Sivaprasad, A. (2024, September). Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices. Presented at INLG 2024, Tokyo, Japan

Automatic metrics are extensively used to evaluate Natural Language Processing systems. However, there has been increasing focus on how the are used and reported by practitioners within the field. In this paper, we have conducted a survey on the use... Read More about Automatic Metrics in Natural Language Generation: A survey of Current Evaluation Practices.

An Open Intent Discovery Evaluation Framework (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). An Open Intent Discovery Evaluation Framework. Presented at SIGDIAL 2024, Kyoto, Japan

In the development of dialog systems the discovery of the set of target intents to identify is a crucial first step that is often overlooked. Most intent detection works assume that a labelled dataset already exists, however creating these datasets i... Read More about An Open Intent Discovery Evaluation Framework.

Automated Human-Readable Label Generation in Open Intent Discovery (2024)
Presentation / Conference Contribution
Anderson, G., Hart, E., Gkatzia, D., & Beaver, I. (2024, September). Automated Human-Readable Label Generation in Open Intent Discovery. Presented at Interspeech 2024, Kos, Greece

The correct determination of user intent is key in dialog systems. However, an intent classifier often requires a large, labelled training dataset to identify a set of known intents. The creation of such a dataset is a complex and time-consuming task... Read More about Automated Human-Readable Label Generation in Open Intent Discovery.

Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot (2024)
Presentation / Conference Contribution
Wilson, M., Brazier, D., Gkatzia, D., & Robertson, P. (2024, July). Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot. Presented at ACM Conversational User Interfaces 2024 (CUI ’24), Luxembourg, Luxembourg

We present a study of collaboration with expert participants for the purpose of the responsible design of a conversational agent. The Delphi study was used to identify and develop design and evaluation criteria for an automated career support interve... Read More about Participatory Design with Domain Experts: A Delphi Study for a Career Support Chatbot.

Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation (2024)
Presentation / Conference Contribution
Watson, L. N., & Gkatzia, D. (2024, May). Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation. Presented at HumEval2024 at LREC-COLING 2024, Turin, Italy

Reproducibility is a cornerstone of scientific research, ensuring the reliability and generalisability of findings. The ReproNLP Shared Task on Reproducibility of Evaluations in NLP aims to assess the reproducibility of human evaluation studies. This... Read More about Reproducing Human Evaluation of Meaning Preservation in Paraphrase Generation.