The Effectiveness of Two Audiovisual Mappings to Control a Concatenative Synthesiser

Abstract

This paper presents the results of an investigation into audio visual (AV) correspondences conducted as part of the development of Morpheme, a painting interface to control a corpus-based concatenative sound synthesiser. Our goal was to measure the effects of the corpora on the effectiveness of the AV mappings used by Morpheme to synthesise sounds from images. Two mapping strategies and four corpora were empirically evaluated by 110 participants. To test the effectiveness of the mappings, sounds were generated using Morpheme from images designed to exhibit specific properties. The sounds were then played to the participants of the study and for each sound, they were presented with three images: the image used to synthesise the sound and two similar distractor images. Participants were asked to identify the correct source image and rate their level of confidence. The results show that the audio corpus used to synthesise the audio stimuli had a significant effect on the subjects' ability to identify the correct image while the mapping didn't. No effect of musical/sound training was observed.

Citation

Tsiros, A., & Leplâtre, G. (2017). The Effectiveness of Two Audiovisual Mappings to Control a Concatenative Synthesiser. In T. Lokki, J. Pätynen, & V. Välimäki (Eds.), Proceedings of the 14th Sound and Music Computing Conference 2017 (320-327)

• Conference Name: Sound and Music Computing 2017
• Start Date: Jul 5, 2017
• End Date: Jul 8, 2017
• Acceptance Date: Apr 28, 2017
• Publication Date: Jul 5, 2017
• Deposit Date: Mar 21, 2018
• Publicly Available Date: Mar 22, 2018
• Pages: 320-327
• Series ISSN: 2518-3672
• Book Title: Proceedings of the 14th Sound and Music Computing Conference 2017
• ISBN: 9789526037295
• Keywords: Audio visual; Morpheme; music; sound synthesiser.
• Public URL: http://researchrepository.napier.ac.uk/Output/1129825