Skip to main content

Research Repository

Advanced Search

How Was Your Day? evaluating a conversational companion

Benyon, David; Gamb�ck, Bj�rn; Hansen, Preben; Mival, Oli; Webb, Nick

Authors

David Benyon

Bj�rn Gamb�ck

Preben Hansen

Nick Webb



Abstract

The “How Was Your Day” (HWYD) Companion is an embodied conversational agent that can discuss work-related issues, entering free-form dialogues that lack any clearly defined tasks and goals. The development of this type of Companion technology requires new models of evaluation. Here, we describe a paradigm and methodology for evaluating the main aspects of such functionality in conjunction with overall system behaviour, with respect to three parameters: functional ability (i.e., does it do the ‘right’ thing), content (i.e., does it respond appropriately to the semantic context), and emotional behaviour (i.e., given the emotional input from the user, does it respond in an emotionally appropriate way). We demonstrate the functionality of our evaluation paradigm as a method for both grading current system performance, and targeting areas for particular performance review. We show correlation between, for example, ASR performance and overall system performance (as is expected in systems of this type) but beyond this, we show where individual utterances or responses, which are indicated as positive or negative, show an immediate response from the user, and demonstrate how our combination evaluation approach highlights issues (both positive and negative) in the Companion system’s interaction behaviour

Citation

Benyon, D., Gambäck, B., Hansen, P., Mival, O., & Webb, N. (2012). How Was Your Day? evaluating a conversational companion. IEEE Transactions on Affective Computing, 4, 299-311. https://doi.org/10.1109/T-AFFC.2013.15

Journal Article Type Article
Publication Date 2012
Deposit Date May 31, 2012
Publicly Available Date May 16, 2017
Electronic ISSN 1949-3045
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 4
Pages 299-311
DOI https://doi.org/10.1109/T-AFFC.2013.15
Keywords Affectice computing; companions; evaluation; spoken interfaces;
Public URL http://researchrepository.napier.ac.uk/id/eprint/5309
Publisher URL http://doi.ieeecomputersociety.org/10.1109/T-AFFC.2013.15

Files







You might also like



Downloadable Citations