Goal: immersive conferencing. Yellow: perceived visual appearance, Blue Outline: perceived auditory appearance of people
Structure of Dynamic Binaural Audio Processing
How can we try to combat "Zoom Fatique" by making Online Meeting tools more immersive and realistic in the audio domain? I looked into that in more depth in my Bachelor's Thesis. A prototype of browser based dynamic binaural audio was implemented by me in JavaScript.
Face tracking & head pose estimation, 3D space conversions, binaural audio simulation using hrtfs, and room reflectance simulation techniques have been combined.
The prototype has also been tested on 12 subjects and the results have been evaluated systematically.
The prototype is available online - enable your headphones & webcam, make sure your face is detected (yellow borderbox) and click on "Audio Setup" once. Now you can play back the integrated audio files and enjoy them in Dynamic Binaural Audio.
Further background information as well as results of the evaluation may be retrieved by listening to my thesis presentation (in German). The thesis itself has been written in english and yielded the grade of 1.0 (A).