Sound Design pipeline for cross-platform 360 virtual productions

Project Description

With the increasing formats and devices for spatial audio in 360 VPs, the question of the most efficient way to deliver spatial audio becomes more complex. Being aware of audience perception and preferences is fundamental to delivering immersive 360 experiences and facilitating the work of producers. This exploration and prototyping phase will be a first step towards defining an industry pipeline for spatial audio production coupled with 360 videos for cross-platform delivery. Cropping the image is also a common technique to attenuate the feeling of dizziness. As a secondary aim, our listening evaluation will also look at any difference in the sense of dizziness between the two versions. We will look at the relationship between the use of different sound design techniques, the camera movement and different video and sound formats impact on the feeling of dizziness. These issues are worth exploring with a novel approach to sound design to solve a current industry problem sound designers face when producing content for cross-platform 360 experiences.
The role of selective auditory attention within 360 experiences will be explored to provide an effective balance between the visible and invisible. Whilst offscreen sounds are commonplace in traditional film, and to a lesser degree TV, in 360, they can quickly become a distraction, especially when combined with typical HRTF inaccuracies. However, offscreen sounds are also necessary to guide attention, provide acoustic context, as well as sustain interest in a scene. These are all key parameters in any content that allows an audience control over a virtual camera.
With VR becoming accessible and the use of immersive spatial audio techniques in consumer applications (cinema, music, social media, retail, real estate etc.), taking advantage of a well-designed and evaluated pipeline for Sound Design for 360 virtual productions can be beneficial to all sound designers, especially beginners, who are looking to see what the best option for their content is.
The audio software development industry can learn what is a sound designer pipeline, the limitations of the tools and what still needs to be done to improve this type of production. With this work, we aim to have a better understanding of the current VP possibilities and the audience's perception of these works. With these results, we will be able to scope what is needed from the industry for the future of 360 productions and, most importantly, what is expected by the audience.