Cronbach & Executive Function Training: Updating Training Shows Only Near Transfer (at the group-level)

In a recent issue of Science, Dahlin et al report the results of an executive function training paradigm focused on the process of mental updating. "Updating" is thought to be one of the core executive functions (as determined through confirmatory factor analysis), is thought to rely on the striatum (as determined through computational neural network modeling), and provides the dynamic gating capacity to working memory which may allow for "perceptual filtering" in which some items are attended and others ignored (as confirmed with neuroimaging).

24 subjects matched for age, education, depression, vocabulary, and mental speed were randomized to an experimental group undergoing 15 45-minute sessions of computer training across 5 weeks, or a control group undergoing only the pre- and posttests of updating ability. The computer training included six tasks all of which involved updating; 5 of the tasks required subjects to report the last 4 items encountered, where items differed by task (letters, numbers, spatial locations, colors) and difficulty was parametrically increased by lengthened the lists for any subject scoring above 80% on the previous length. The sixth task involved classifying each of 15 serially-presented words into multiple categories and remembering the last word classified into each category; difficulty was manipulated by increasing the number of categories.

The results showed that there were remarkable training-related improvements (relative to the control group) on the criterion updating task (letter memory) but this is not surprising, as this was one of the trained tasks. More interestingly, the n-back showed significant training effects relative to the control group, although that task had not been explicitly trained. Finally, the Stroop task showed no significant improvements (although Dahlin et al. did not look at incongruent vs. neutral RTs, as is traditionally done, instead calculating incongruent vs. congruent; further, their Stroop task had only 2 possible responses).

The neuroimaging data showed that the tasks which showed training effects were precisely those that activated the left striatum during pretest, and this same area of the striatum showed a training-related increase in activity in both tasks. In contrast, Stroop did not coactivate striatum although a similar frontoparietal network was involved in all tasks. Interestingly, whereas the striatum showed a training-related increase in activity, involvement of the fronto-parietal network in updating tasks decreased in activity after training, perhaps suggesting that improved updating and filtering functions "lessened the load" on the frontoparietal working memory network.

A second experiment replicated the basic results with a sample of elderly adults, but the elderly did not show the same benefit of training (their performance asymptoted at that acquired by young adults after only 2 weeks of training, and roughly similar to that of the young adult control group) nor did they show any transfer of training to the n-back task. On the other hand, the proportional improvement of older adults was roughly the same as that of younger adults, and it's not immediately clear whether numerical or proportional improvement is the appropriate criterion for group comparison.

One of the interesting things about this article is that they failed to show transfer to Stroop among young adults, and to 3-back among older adults - but they never perform a power analysis, nor do they report alternative methods for calculating improvement (such as the proportion change mentioned above, the use of neutral rather than incongruent trials, etc). They also do not report looking at individual differences in training-related improvements to see whether some subjects might indeed have shown a benefit - for example by calculating differences in the slopes of the two groups' performance between pre- and post-training. Finally, they cite many papers showing a lack of transfer in training, but do not cite those papers which do show transfer - even far transfer - such as Diamond et al., and some interesting papers which I will be reviewing this week.

The paper is also an interesting example of the lingering disconnect between "the two disciplines of scientific psychology" - as Cronbach argued in 1957, the analysis of group-level effects is often conducted independently of the analysis of effects due to individual differences, despite the obvious possibility that the two interact strongly. In this case, the authors showed only that a) the same regions of left striatum showed similar training-related changes on 3-back and letter memory, and that b) there was transfer between these tasks. Their conclusion thus relies on this group-level similarity between behavior and neuroimaging. In other words, they've done a correlation with a sample size of 3: "2 tasks showed similar striatal change and those same two tasks showed transfer; a third task showed dissimilar striatal change from the others and likewise showed no transfer; thus striatal change drives transfer"

What they didn't show is a within-group correlation; they didn't show that the amount of transfer an individual gets is *predicted* by the amount of change they show in the striatum. The difference is in a between group vs. within-group analysis, and to my mind the latter is more diagnostic, whereas the former just shows that some brain region was influenced by training in the same way in two tasks.

This highlights the problematic and implicit assumption made by Dahlin et al: must brain regions which mediate transfer of training show the same hemodynamic profile in those tasks? It seems at least possible that training might influence a particular brain region such that it is engaged differently in two different tasks - say, more selectively - and that difference in the way the region is engaged might translate into improved performance on both tasks. This is the kind of thing a within-subjects test could have revealed, even if at the group level the brain regions change in different directions as a function of training on the two tasks.