Cool Visual Illusions: Audivisual Fission and Fusion, I

You may have seen this illusion in a post from earlier in the week over at Cognitive Daily, but I thought I'd say a little bit more about it, and talk about a related illusion. First, click play (from Sham's demo site)

If the illusion is working -- Dave at Cognitive Daily had a bit of trouble getting it to work for his readers -- you should see the dot flash on the screen twice. However, the dot only flashes once. If you don't believe me, go to Sham's page and watch the single beep movie. It's the same movie, but with one beep instead of two. The illusion is pretty strong for me, but as Dave has suggested, this may be due to practice. I've watched it many times. If it doesn't work, try it a few times. If it still doesn't work, well then, you're just defective!

Interestingly, the effect only works to increase the number of flashes people see; you can't get people to see only one flash when they hear two one beep accompanying two flashes. This assymetry makes explaining the illusion, which Sham et al.1 call "sound-induced illusory flashing," difficult within traditional theoretical frameworks. Sham et al. therefore developed the discontinuity hypothesis, which states that the discontinuous modality (in this case, sound, with two separate beeps) dominates in cross-modal interactions like that between sound and vision in their stimuli. , They further argue that the robustness of the effect (in their experiments, at least, though apparently not on blogs) implies that these sorts of cross-modal interactions are the rule, rather than the exception, and that the asymmetry of the illusion implies that it is perceptual rather than cognitive.

The coolness doesn't end with sound-induced illusory flashing, though. In a followup study designed to sort out potential explanations for the illusion, Andersen et al.2 demonstrated the reverse effect. They had participants listen to one beep, played so softly that people could barely hear it, with the purpose of counting the number of beeps (in the Sham et al. study, participants were told to count the flashes) Simultaneous with the beeps, the experimenters showed them a dot that flashed twice. This caused the participants to hear two beeps. When the dot was flashed only once, they heard only one beep. They argue that their data implies several factors that influence cross-modal interactions, and thus the sound-induced illusory flash illusion and its reverse. First they argue that the discontinuity hypothesis plays a role, but that it is insufficient to explain the two illusions. Instead, modality appropriateness (different sensory modalities are better suited to process different kinds of information), the reliability of the sensory information for each modality, and the amount of attention we allocate to each modality, play a role in determining which dominates. The dominant modality can then influence what we perceive through the other modality, so that one flash can look like two when the flash lasts a very short period of time (as in the above video) and is accompanied by two clear beeps, and one beep can sound like two when we can barely hear it and it's accompanied by two flashes.

¹Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147-152.
²Andersen, T.S., Tippana, K., & Sams, M. (2004). Factors influencing audiovisual fission and fusion. Cognitive Brain Research, 21(3), 301-308.