Investigating sound function, aesthetics and interaction in media production to create effective sonic interaction design for everyday life.
As more and more everyday devices become smart and intelligent, we can “talk” to them, and they are starting to “talk” back. In this context sound is rising to be perhaps the most natural and free way to interact, access and communicate with computers. But while speech and music have rich literatures, codes and long documented histories upon which we can base our audio designs, other sounds (everyday sounds, abstract and synthesized sounds) do not.
In this project, sound design practices developed in media productions, such as film and theatre, are considered a rich, yet under-valued and under-researched, resource for sonic interaction design, auditory display, sonification and sound computing in general.
As point of departure, we adopt IRCAM Sound Design Research Team´s definition of sound design: Sound design: making an intention audible.
This definition implies that a designed sound is new and constructed, and represents something other than the sound itself. This can be an object, a concept or a system.
There are primarily two ‘intentions’ that need to be audible: form and function. A designed sound needs to have a form that is ‘appropriate’ for the object/concept/system it represents, and it needs to fulfill a function, to communicate information about the object/concept/ system to the individual. This information needs to be clearly heard and correctly interpreted for the design to be considered successful. (Susini, Patrick (2011) Le design sonore: un cadre experimental et applicative pour explorer la perception sonore, Dossier d’habilitation á diriger des recherches, Aix-Marseille II)
We extend this definition by emphasising the importance of context as a key variable that affects the effectiveness of a sound design, and we propose that our media production-based approach is particularly well-suited to the study of the relationship between effective communication through sound and its context.
S. Pauletto, " Embodied Knowledge in Foley Artistry ," in The Routledge Companion to Screen Music and Sound, : Routledge, 2017, p. 338.
F. Keenan and S. Pauletto, " Listening Back: Exploring the Sonic Interactions at the Heart of Historical Sound Effects Performance ," The New Soundtrack, vol. 7, no. 1, pp. 15-30, 2017.
B. P. Walus, S. Pauletto and A. Mason-Jones, " Sonification and music as support to the communication of alcohol-related health risks to young people. Study design and results ," Journal on Multimodal User Interfaces, vol. 10, no. 3, pp. 235-246, 2016.
S. Pauletto, " Film and theatre-based approaches for sonic interaction design ," Digital Creativity, vol. 25, no. 1, pp. 15-26, 2014.
S. Pauletto, " The sound design of cinematic voices ," The New Soundtrack, vol. 2, no. 2, pp. 127-142, 2012.
S. Pauletto et al., " Integrating theatrical strategies into sonic interaction design ," in Proceedings of Audio Mostly 2009 - A Conference on Interaction with Sound, 2009, pp. 77-82.
Guest Edited Journal Special Issues
Pauletto, S., Cambridge, H. and Susini, P., Guest Editors, Special Issue: Data Sonification and Sound Design in Interactive Systems, 85(1) International Journal of Human Computer Studies, Elsevier, January 2016.
Pauletto, S., Guest Editor, Special Issue: Perspectives on Sound Design, The New Soundtrack Journal, Vol: 4, Issue: 2. September 2014.
Organised Symposia and Conferences
2014, SoniHED - Conference on Sonification of Health and Environmental Data, University of York
2013, Symposium: Perspectives on Sound Design, University of York
2009, Sonic Interaction Design and its relation to Film and Theatre sound design, University of York