A multidimensional sketching interface for corpus-based concatenative synthesis.

Tsiros, Augoustinos

Abstract

This paper presents Morpheme, a multidimensional interface that allows real-time control of concatenative synthesis through the act of sketching on a digital canvas. Morpheme extracts textural, spatial and volumetric features from a sketch developed by a practitioner and associates these to audio features for retrieval of audio units and to synthesis parameter for signal processing. Two mappings between audio and visual features were developed based on findings from previous studies that examined audio and visual feature correlation. One of the mappings is achromatic as the features extracted from the sketch are mainly volumetric and spatial, while the second mapping is chromatic as the features extracted from the sketch are based on color attributes. A number of simple algorithms are discussed that were developed to address three problems: (i) to estimate high level visual feature, (ii) set constrains in the audio corpus to improve the selection algorithm and the exploration of the corpus, and (iii) automatically adjust the weights to improve the efficacy of the selection algorithm in assessing similarity, by optimizing the algorithm in a corpus depended manner.

Citation

Tsiros, A. (2013). A multidimensional sketching interface for corpus-based concatenative synthesis. In Proceedings of the International Conference on Auditory Display 2013

• Conference Name: international Conference on auditory Display
• Start Date: Jul 6, 2013
• End Date: Jul 10, 2013
• Publication Date: 2013
• Deposit Date: Sep 24, 2013
• Publicly Available Date: May 16, 2017
• Peer Reviewed: Peer Reviewed
• Book Title: Proceedings of the International Conference on Auditory Display 2013
• Keywords: Morpheme; multidimensional interface; concatenative synthesis; signal processing
• Public URL: http://researchrepository.napier.ac.uk/id/eprint/6376