Cool Visual Illusions: Audivisual Fission and Fusion, I

You may have seen this illusion in a post from earlier in the week over at Cognitive Daily, but I thought I'd say a little bit more about it, and talk about a related illusion. First, click play (from Sham's demo site)

If the illusion is working -- Dave at Cognitive Daily had a bit of trouble getting it to work for his readers -- you should see the dot flash on the screen twice. However, the dot only flashes once. If you don't believe me, go to Sham's page and watch the single beep movie. It's the same movie, but with one beep instead of two. The illusion is pretty strong for me, but as Dave has suggested, this may be due to practice. I've watched it many times. If it doesn't work, try it a few times. If it still doesn't work, well then, you're just defective!

Interestingly, the effect only works to increase the number of flashes people see; you can't get people to see only one flash when they hear two one beep accompanying two flashes. This assymetry makes explaining the illusion, which Sham et al.1 call "sound-induced illusory flashing," difficult within traditional theoretical frameworks. Sham et al. therefore developed the discontinuity hypothesis, which states that the discontinuous modality (in this case, sound, with two separate beeps) dominates in cross-modal interactions like that between sound and vision in their stimuli. , They further argue that the robustness of the effect (in their experiments, at least, though apparently not on blogs) implies that these sorts of cross-modal interactions are the rule, rather than the exception, and that the asymmetry of the illusion implies that it is perceptual rather than cognitive.

The coolness doesn't end with sound-induced illusory flashing, though. In a followup study designed to sort out potential explanations for the illusion, Andersen et al.2 demonstrated the reverse effect. They had participants listen to one beep, played so softly that people could barely hear it, with the purpose of counting the number of beeps (in the Sham et al. study, participants were told to count the flashes) Simultaneous with the beeps, the experimenters showed them a dot that flashed twice. This caused the participants to hear two beeps. When the dot was flashed only once, they heard only one beep. They argue that their data implies several factors that influence cross-modal interactions, and thus the sound-induced illusory flash illusion and its reverse. First they argue that the discontinuity hypothesis plays a role, but that it is insufficient to explain the two illusions. Instead, modality appropriateness (different sensory modalities are better suited to process different kinds of information), the reliability of the sensory information for each modality, and the amount of attention we allocate to each modality, play a role in determining which dominates. The dominant modality can then influence what we perceive through the other modality, so that one flash can look like two when the flash lasts a very short period of time (as in the above video) and is accompanied by two clear beeps, and one beep can sound like two when we can barely hear it and it's accompanied by two flashes.

1Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147-152.
2Andersen, T.S., Tippana, K., & Sams, M. (2004). Factors influencing audiovisual fission and fusion. Cognitive Brain Research, 21(3), 301-308.

More like this

Last week, I talked about the sound-induced flash illusion, in which presenting a single flash with two or more auditory beeps caused people to see two or more flashes. This week, a study showing that the same effect can be obtained by replacing the beeps with touches. Here's the setup (…
There was plenty of interest in yesterday's audio-visual illusion. In case you missed it, I'll post it again here: Play the movie with the sound turned up. If the illusion works, then you'll see a dot flash twice, accompanied by two beeps. But actually the dot only flashes once. Unfortunately, we'…
There's nothing cooler for a perception researcher (or writer) than a new visual illusion. When I learned about this one, I spent half the day Thursday trying to recreate it, but I couldn't get it to work. Finally, in five minutes on Friday morning, I think I figured it out. (Update: Actually, as…
Just about two weeks ago, I posted this visual illusion (if you haven't seen it yet, make sure to watch it with the sound turned ON): How many flashes do you see? In fact the dot only flashes once, but according to the study I report on in the post, the two beeps are supposed to throw you off. If…

One dot flash, two tones. Initially, i thought that the flash was of mutiple images, but that turned out to be an artifact of the playback. Never got a second flash, so count me in the "not tricked" category.

As for the reason: your mind will activate a channel between any two events that happen at the same time - it's how minds are built. Timing a second event with only a partial match to the first at an axon release node will induce a secondary illusionary event to match the missing input. Again, this is how brains are built. The illusion can be made stronger by having a longer set-up period, as axons are very rythmic. A slightly slower sequence is preferable, too; sequences that are too fast will be lost on some participants.

By Jonathan Schattke (not verified) on 31 Oct 2006 #permalink