Synthetic Space: Binding Errors In Synesthesia

Some people experience an intermingling of the senses, known as synaesthesia, in which certain shapes become combined or "bound with" certain colors, or that certain colors are strongly associated with certain sounds. Of course, in healthy normal adults, color and shape become bound together only when visible shapes really are visibly colored - a process which is itself still a mystery of neuroscience. Yet similar mechanisms may be involved in both these kinds of "binding," but are merely more active in synaesthetes than in normal adults.

Based on Patient RM, we know that binding between color and shape relies on the parietal lobes - without them, the sense of space dissolves and the visual world disintegrates into individual and unconnected sensations. If the parietal lobes are important for binding, one might expect them to also be critical for synaesthetes tightly integrated sensations. To test this idea, Esterman et al temporarily disabled the parietal lobes of two synaesthetes who reliably associated particular letters with particular colors.

The subjects first identified the RGB values which were mentally most strongly associated with each of the alphabet's 26 letters. Then, for each of many trials, the synaesthetes had to indicate whether a colored letter was actually red, green or yellow. Critically, some of these letters were colored in a way that was consistent with each individual's synaesthesia, others in a way that was inconsistent with each individual's synaesthesia, and yet other letters or symbols which were colored but not mentally associated with any color for the subjects (such as "#").

Replicating previous work with synaesthetes, these subjects showed interference from their mental sensation of color when it conflicted with the actual color of the letters. That is, subjects were slightly slower (less than 60 msec) to name the color of a letter when it mismatched their synaesthetic experience than when the letter was not associated with a color. These kinds of subtle, consistent differences can be difficult to fake, suggesting that the two subjects were indeed synesthetic.

But even more interesting is what happened when the experimenters disabled the subjects' parietal lobes.

The experimenters targeted each subject's posterior parietal cortex with strong, repetitive magnetic stimulation, which is thought to temporarily "scramble" the activity in that region of the brain. The technique, known as "rTMS," is thought by many to deactivate the targeted region of neural tissue, which in this case was the intraparietal sulcus and the transverse occipital sulcus.

When rTMS was applied to their right parietal lobe, these subjects no longer showed the pattern described above. That is, they were no slower to name the color of a letter when it conflicted with their synaesthetic experience as when the letter had no color association. rTMS had no such effect when applied to other regions, including the left parietal cortex and the V1 area of the occipital lobe.

So the binding of colors and shapes in normal healthy adults seems to involve the same brain area that's involved in synaesthesia. Of course, it's unlikely that just one area is involved in synaesthesia; like most cognitive phenomena, it is likely a complex product of interactions among multiple brain regions.

Many questions remain - would the same be true of "associator" type synaesthetics, who associate their synaesthetic experiences with more internal than external sensations? What role might other areas, including the frontal cortex, have in synaesthetic experience? And finally, do synesthetes have altered attentional mechanisms, given the notable attentional deficits of those with Balint's syndrome?

Tomorrow's post will cover the hypothesized role of attention in binding, to gain insight onto the cognitive mechanisms that work to integrate our sensations, whether only sparsely (as in Balint's) or over-zealously (as in synesthesia).

More like this

SUBJECTIVE experience poses a major problem for neuroscientists and philosophers alike, and the relationship between them and brain function is particularly puzzling. How can I know that my perception of the colour red is the same as yours, when my experience of the colour occupies a private mental…
Synaesthesia involves the inappropriate binding of one perception to another - for example, color-grapheme synaesthetes might perceive the letter "h" to be noticeably red, and are actually slower to identify the letter "h" when it is green than when it is red or gray. This "inappropriate binding"…
In the 1880s, Francis Galton described a condition in which "persons...almost invariably think of numerals in visual imagery." This "peculiar habit of mind" is today called synaesthesia, and Galton's description clearly defines this condition as one in which stimuli of one sensory modality elicit…
How do you know what's where? Cognitive psychologists believe this requires "binding" - the ability to associate certain colors or shapes with each other and with certain locations in space, giving rise to a coherent perception of the world. It turns out that at least one unlucky person lacks…

This was a very excellent summary of the work; however, I am a bit confused as to why it would be surprising that disabling the parietal lobes would result in the above symptoms (no time delay). It has been known that the TPO junction is an area associated with higher-synsthesia so isn't this a logical conclusion?

By anonymous (not verified) on 07 Jun 2007 #permalink

Good question - one surprising thing is that the rTMS apparently targeted *just* the synaesthetic color-grapheme mapping and not the "real" color-grapheme mapping in the experiment. Second, I'm not sure that this region (IPS & TOS) is exactly the same as the TPO, but this is what you get when dealing with anatomical names rather than MNI coordinates. Third, the TPO is just one of many regions previously postulated as important for synaesthesia (including fusiform gyrus & V4) but this is all from fMRI, which is a correlational tool. That means you can't tell if one of the regions is causally necessary for synaesthesia or merely correlated with it for some other reason. rTMS on the other hand, can help to establish causal relationships. Finally, the recent Rouw & Scholte dtMRI results show that it is actually a *different* region that is structurally and functionally abnormal in synaesthetics, so this is surprising to some extent (see the discussion here about spurious lateralization in synaesthesia).

Can you tell where the claims about TPO originate? I haven't been able to verify the evidence used in support of that; every blog post and academic paper merely states it as fact. I'm assuming the supporting evidence is from fMRI, but it could be just a "postulate" of the Grossenbacher disinhibited feedback theory.

To test this idea, Esterman et al temporarily disabled the parietal lobes of two synaesthetes

How did they do that? "Disable" the parietal lobe, I mean.

Hey Coin - I describe the "how" in the very next paragraph - repetitive transcranial magnetic stimulation. Some people think it suppresses activity in the targeted region, while others think it actually hyperactivates those regions. I thought it was safe to pick the middle road, and just say that it "scrambles" activity in whatever region it targets.

It's an interesting technology but not to be tried at home (search for "Shakti helmet" to see some scary DIY TMS kits)

I think I have a mild case of synesthesia. I certainly "know" what color each letter of the alphabet is. I wrote these colors down in my journal years ago, and when I returned to the list later, I consistently "knew" the same colors as I had before. P is blue. A is green. O is white. D is brown.

I also seem to be able to see the shape of sounds. If I'm quietly listening to a symphony, for example, I can picture the various shapes intermingling in my mind's eye.

I'd always thought this was a normal understanding that everyone had, but when I made casual mention of it, many people would look at me as though I was crazy. Still, my older brother agrees with me, and so did one of my bosses. It's matter of fact to us.

I've always associated colors with numbers and sounds and words (not so much individual letters for me; just the overall word and its sound). It didn't occur to me that there was anything odd about this until I read an article about synesthesia in Discover magazine when I was 12 or so. Then I started mentioning it to try and feel people out about it.

It does make for amusing / frustrating conversations, depending on which side of the convo you're on. "What did you think of the Symphony piece?" "Too much brown. It needed more red."

Qalmlea, I'd love to be the fly on the wall when two synæsthetes are carrying on such a conversation. I suspect there is some overlap from one individual to the next in these correlations, possibly more than just what is accountable through random factors. Enough for people to hope that they could convince the other of the 'proper' correlation.

Do you think the effect on words is an aid to spelling for you?

By JohnnieCanuck, FCD (not verified) on 10 Jun 2007 #permalink

Yes, this is a really interesting discussion !

It turns out that there are some consistent trends between individuals in different forms of synaesthesia. In sound-color synaesthesia, higher pitches are usually associated with brigher colors. In color-grapheme synaesthesia, the letter A tends to be red, the letter B tends to be blue, etc. Of course, it's hard to recruit a lot of synaesthetes so who knows whether these differences would hold up cross-culturally etc.