A great part of human communication and interaction with the surrounding environment is based on sound. Sound pervades our life; speech, sounds produced by our interaction with the environment, and everyday sounds are the most common sounds that we hear continuously during the day. In spite of this, interaction in HCI is mainly vision-based. In a world where graphical displays are becoming smaller and sometimes missing, there is a need for a new way of human-machine interaction based on sound.
This course offers a multi-disciplinary overview of the principles underlying the use of sound in interaction between users and machines.
The course combines the basics as well as the latest results in sound perception, sound in human-machine interaction, and sonification (e.g. data representation using the audio channel). The course includes:
an overview of sound perception including the perception and recognition of sound sources, sound illusions, and experimental methodology (with particular focus on practical issues when experimenting with sound);
an overview of latest technology for the use of sound in interaction, including sound models, data sonification methods, sensor platforms;
some hands-on laboratory sessions where course participants can try different methods, technology, experimental set-ups involving sound in interaction using sensors;
the realization of a project where each student will work in-depth within a selected area of interest.