ACM Siggraph and Eurographics. Campfire: Acoustic Rendering for Virtual Environments. Snowbird, Utah, 26-29 May 2001.
Position statement.
 

Investigation of multisensory spatial hearing:

from the sense of audition to multisensory interactions


Klaus A J Riederer
 

Laboratory of Computational Engineering
Laboratory of Acoustics and Audio Signal Processing

Helsinki University of Technology (HUT)
P.O. Box 9400, FIN-02015 HUT, Finland
Tel: +358 9 451 5328; Fax +358 9 451 4830
Email: Klaus.Riederer@hut.fi, URL: http://www.lce.hut.fi/~kar




The study of human multisensory, especially audio-visual perception, has recently obtained increased attention with the development of virtual reality systems, teleconferencing, computer games and home theatre systems. An immersive sound scape can be created by three-dimensional sound applying head-related transfer functions that “model” human spatial hearing. Already from practice one realizes that the (spatial) hearing sense is truly multi-modal, applying also other senses, such as motion (moving the head), vision, tactile sensing etc. However, due to the technical difficulties and other complexity, serious efforts to investigate the sensory interactions (especially concerning spatial hearing) are still lacking. Therefore, a deep understanding of our second most important sense - hearing -  is most incomplete. The author’s research focuses on the true spatial hearing including interactions between other sensory modalities, such as audio-visual, audio-motional, audio-visual-motional. This basic research has strong interdisciplinary connections in various fields of science and numerous application areas. Novel research paradigms will be addressed under various objectives.

Keywords: 3-D sound, audio-visual perception, basic research, heterosensory, homosensory, HRTF, multidisciplinary, multi-modal perception, spatial hearing, virtual reality