Imagine you’re in a life concert of one of your favourite bands and suddenly what you see doesn’t correspond to what your hear. They just started doing playback! Can you belive that? When the sound isn’t in time with what you’re seeing it might take a while before you notice. But once you do, it’s impossible to ignore and you’ll want to find a solution straight away. In this case that would be asking to get your money ticket back!
The match of audio-visual information from the outside world creates a phenomenon called intersensory facilitation. Consider that you’re required to process visual stimuli and press a button whenever you see a flash in front of you. You’ll probably respond faster (your reaction time will decrease) if when a the light pops up you hear an additionaly auditroy signal (e.g, could be a simple “bip”) coincident with or even slightly after the visual stimulus. To put it in other words, if audio-visual information matches (even though audio is not relevant for the task) your reaction times will get reduced, but you’ll be more likely to make an error. Isn’t that facinating? How intersensory might facilitate the response to an event just by the presence of multimodality sensory events.
While this approach has been sufficient to demonstrate that audiovisual interactions do occur and can influence spatial and temporal aspects of perception, it is more relevant to real-world contexts to examine how audiovisual interactions affect our ability to locate and identify objects in cluttered environments. In such cases, there are typically many objects distributed across space that compete for and distract our attention.
Feature image from Pexels – C0 license.