The dimensions and complexities of audio-visual association.

Tsiros, Augoustinos

Abstract

This paper draws on literature from psychology, neuroscience, linguistics, and philosophy to portray a cross-modal conception of auditory and visual phenomena focusing on the similarities in form, structure, and function and exploring the links to perception, conception, and language. Through an extensive literature review, we identify issues related to audio-visual association and explore divergences and convergences between the two modalities. The aim of this paper is to examine how recent research findings from brain science could inform the theoretical and methodological approaches used for studying similarity of auditory and visual percepts in the context of designing multimodal interaction.

Citation

Tsiros, A. (2013). The dimensions and complexities of audio-visual association. In Electronic Workshops in Computing (eWiC)

• Conference Name: Electronic Visualisation and the Arts (EVA 2013)
• Start Date: Jul 29, 2013
• End Date: Jul 31, 2013
• Publication Date: Sep 1, 2013
• Deposit Date: Sep 24, 2013
• Publicly Available Date: Sep 24, 2013
• Publisher: BCS, The Chartered Institute for IT
• Peer Reviewed: Peer Reviewed
• Book Title: Electronic Workshops in Computing (eWiC)
• ISBN: 9781780172156
• Keywords: Perception; Cognition; Multimodality; Structure; Embodied Metaphor; Similarity; Multimodal Interaction
• Public URL: http://researchrepository.napier.ac.uk/id/eprint/6379