Failures in Cognitive Training: Practice and The Stroop Task

Training high-level cognition or "executive function" is not always successful. Interestingly, some of the least robust training effects come from one of psychology's most robust paradigms - the Stroop task.

The Stroop task is simple to describe, but difficult to complete. In its simplest form, subjects must name the ink color in which a word is written but ignore potentially conflicting information from the meaning of the word itself. This typically results in three types of trials: neutral trials (where word meaning is irrelevant to the ink color"LOT"), incongruent trials (where word meaning is in conflict with ink color: "RED") and congruent trials (where word meaning is compatible with ink color: "BLUE").

These trial types generally yield different average reaction times, such that incongruent > neutral > congruent. The difference between incongruent and neutral trial types - the "Stroop interference" score - may reflect the ability to resolve interference quickly (perhaps relying on anterior cingulate regions). In contrast, the difference between congruent and neutral trial types - the "Stroop facilitation" score - may reflect "goal neglect", or the extent to which subjects benefit from the word reading processes they shouldn't actually be using. Stroop interference and facilitation are thought to reflect different mechanisms, although both correlate with gross measures of mental capacity (e.g., operation span).

Clearly, this is a paradigm where training effects should be most robust: interference and facilitation effects occur precisely because we have more experience with reading words than we do with naming ink colors. Therefore, we are slowed when we must overcome this highly-practiced habit of reading.

Intuitively, then, interference and facilitation should both be reduced if subjects simply practice naming colors, particularly if they are paired with incongruent words. As MacLeod describes in his famous 1991 review paper, however, Stroop training effects are decidedly inconsistent.

First, only the interference score seems capable of showing a training effect. But the conditions under which even that occurs are very unclear. A few examples:

Temporary effects. In 1935, Stroop himself demonstrated a reduction in interference on color naming, and a clear but temporary increase in interference on word reading, with 8 days of color naming practice; the latter effect disappeared after a second posttest. Others have since replicated this basic result and shown that this "reverse" Stroop effect is vanishingly temporary. It is unclear whether the benefits of color-naming practice can be maintained over time, but it would seem unlikely based on the transience of the reverse Stroop effect.

Does practice just influence the general speed of responding? One might get a disproportionate reduction in incongruent trial RTs, relative to neutral trial RTs, if the benefits due to practice with "color naming" processes is most visible when those processes competitively interact with word naming. This is consistent with work showing greater benefits for color naming than word reading after training, regardless of whether benefit is measured in absolute (ms) or relative (%) terms. Is training really improving some aspect of high level cognition, such as focused attention, or merely how quickly subjects can name colors? (Alternatively, I note that some Stroop training has been associated with a slowing of word reading, as assessed on neutral trials, far from what we'd like from a successful attention training paradigm!)

Practice effects may be highly specific. A variety of work suggests that practice in the Stroop task does not generalize beyond the particular stimuli trained. For example, practice with word versions of the Stroop task does not transfer to object version of Stroop (e.g., in which bananas are blue, strawberries green, etc). Stoup & Flowers showed that practice effects are evident only when color & word stimuli are unintegrated (e.g., the word red written inside a green rectangle) and only when subjects must sort stimuli rather than orally name them. Adding to the confusion, MacLeod more recently showed more pronounced training effects with integrated stimuli. Other evidence suggests that practice improves manual responding more than oral responding, and that better transfer of training is observed when the stimuli are words relative to numbers. More recently, Clawson et al. have found item-specific practice effects in Stroop - meaning that training-related benefits are due not to general improvements in attention but to the representations of specific words and colors.

Ambiguity in the time required for training effects. Some have suggested that Stroop training effects asymptote within the first 5 trials; others find effects across the first one hundred trials; yet others find that performance asymptotes after 400-500 trials, relative to the next ~500 trials; and in yet other cases, many thousands of trials are required to show a training-related effect.

Why should training have such weak, heterogenous, and low-level effects in the Stroop paradigm? Here are two speculations:

1) It is unlikely that interference can ever be truly eliminated. Therefore, training-related improvements in interference might be masked by a floor effect, since Stroop interference scores can sometimes be quite small, and variability in reaction times can be quite large.

2) Training might influence subcomponents of Stroop interference in different ways. Stroop interference evidently reflects some combination of fluency with reading words, naming colors, the ability to separate word and color information, and the ability to resolve competition between these sources of information. Perhaps training initially allows for more efficient disintegration of color and word information (as supported by MacLeod's work), and thereafter just generally increases attention to the task, yielding broad improvements in reaction time, but relatively small effects on the interference score.

3) To the extent that elimination of the interference effect is possible, it may actually reflect slowing of congruent and neutral reaction times rather than a selective benefit to the incongruent condition. This is consistent with work from Dulaney & Rogers suggesting that word reading actually becomes less automatic with practice.

In summary, despite the long history of research in the Stroop task, experiments which use training as an experimental manipulation cast doubt on the idea that Stroop can be used to promote broad and stable improvements in attention. The reasons for this are unclear, but may have to do with floor effects, the process impurity of Stroop, and general effects of training on reaction time (whether speeding or slowing).