Evaluating the Perceived Similarity Between Audio-Visual Features Using Corpus-Based Concatenative Synthesis.

Tsiros, Augoustinos

Abstract

This paper presents the findings of two exploratory studies. In these studies participants performed a series of image-sound association and detection tasks. The aim of the studies was to investigate the perceived similarity and the effectiveness of two multidimensional mappings, each consisting of three audio-visual associations. The purpose of the mappings is to enable visual control of corpus-based concatenative synthesis. The stimuli in the first study was designed to test the perceived similarity of six audio-visual associations, between the two mappings using three corpora resulting in 18 audio-visual stimuli. The corpora differ in terms of two sound characteristics: harmonic content and continuity. Data analysis revealed no significant differences in the participants' responses between the three corpora, or between the two mappings. However significant differences were revealed between the individual audio-visual association pairs. The second study investigates the effects of the mapping and the corpus in the ability of the participants to detect which image out of three similar images was used to synthesise the audio stimuli. Data analysis revealed significant differences in the ability of the participants to detect the correct image depending on which corpus was used. Less significant was the effect of the mapping in the success rate of the participants' responses.

• Conference Name: International Conference on New Interfaces for Musical Expression
• Start Date: Jun 30, 2014
• End Date: Jul 3, 2014
• Publication Date: 2014
• Deposit Date: Oct 27, 2014
• Publicly Available Date: May 16, 2017
• Peer Reviewed: Peer Reviewed
• Book Title: Proceedings of the Internation Conference on New Interfaces for Musical Expression
• Keywords: Concatenative Synthesis; Audio-visual Correspondence; Multimodal interaction; Embodied interaction
• Public URL: http://researchrepository.napier.ac.uk/id/eprint/7269
• Publisher URL: http://nime2014.org/proceedings/papers/484_paper.pdf
• Contract Date: May 16, 2017