Parallel Search in Rule Space Enabled By Prefrontal Hierarchies

Hierarchical views of prefrontal organization posit that some information processing principle, and not just task difficulty, determines which areas of prefrontal cortex will be recruited in a given task. Virtually all information processing accounts of the prefrontal hierarchy are agreed on this point, though they differ in whether the operative principle is thought to be the temporal duration over which information must be maintained, the relational complexity of that information, the number of conditionalities necessary to consider in behaving on that information, or the inherent abstraction of the underlying representations.

All of these ideas seem to mesh with an intuitive sense of what it means for something to be "difficult," such that one might as well say that more difficult things activate more anterior regions of the prefrontal cortex. Ruling out difficulty as an explanation has thus been a pernicious problem for much of this work. Typically, the attempted arguments rely on the behavioral expression of difficulty (long reaction times or high error rates) not following the same patterns as neural activation. But these arguments are complicated by the fact that the behavioral expression of difficulty might be muddled if subjects recruit additional regions of prefrontal cortex in response to perceived difficulty.

Thankfully, this issue has been definitively addressed by a 2010 Neuron paper from Badre, Kayser & D'Esposito. In the process they've provided numerous insights into the way an integrated prefrontal-striatal circuit might support an information processing hierarchy and "parallel search" in service of the discovery of rules for governing behavior.

Theoretically speaking, Badre et al adopt a relatively specific view of the information processing principle that defines the functional architecture of the prefrontal cortex: policy abstraction. This is a machine learning phrase typically used in discussions of reinforcement learning, and refers to how a "first-order" policy for determining response (e.g., if you see a circle press the left button; if you see a square press the right button; or e.g. if you see a small object press the left button, if you see a large object press the right button) can be made conditional on a more abstract "second-order" policy (e.g., if the item is red, press according to shape; if the item is blue, then press according to size). What makes the second-order policy more abstract is that it shapes a subordinate policy, instead of directly specifying a response based on a stimulus (that would be a first-order policy!).

By Badre et al's theory, second-order policies should recruit more anterior areas of the prefrontal cortex. Thus, they designed a task in which variously-shaped and variously-oriented objects surrounded by various colors each required one of 3 button presses. In one case, the rules determining which combination of shapes, orientations and colors lead to each of the three button presses was devoid of any consistent rule. That is, it was a "flat" rule structure, where the association of button presses with shape/color/orientation combinations had to be rotely memorized. In another case - the hierachical condition - the color determined whether orientation or shape determined the appropriate response. In both the flat and hierarchical condition, there were three orientations and three shapes, but only in the hierarchical condition were they mapped consistently onto responses (a first-order policy) in a way that was determined by a higher level rule (a second-order policy).

And the kicker: they never told subjects the rules! Subject had to discover the appropriate policies for responding.

The results indicated that subjects recruited a somewhat more anterior region of the prefrontal cortex (pre-PMd) in the hierachical case than the flat case; moreover, this was not attributable to difficulty, because subjects performed better, and better sooner, in the hierarchical condition, and did so in a way that is reminiscent of insight: learning curves showed abrupt "jumps" in performance (as confirmed through analysis of first and second derivatives in learning), as though subjects were figuring out the second-order policy as enabled by activation of pre-PMd. Numerous significant brain-behavior correlations confirmed this result in the pre-PMd, in contradistinction to a more posterior region, PMd.

Subjects also activated pre-PMd in the flat case, at least early on, but eventually reduced the activation in this region as though "giving up" on finding the higher-order rule, while nonetheless maintaining the activation in the more posterior PMd region. Critically, performance remained lower in this flat condition than in the hierarchical case, despite reductions in pre-PMd activation, indicating that pre-pMD is not brought online merely to compensate for perceived (or experienced) difficulty. Thus, Badre et al elegantly and conclusively rule out difficulty explanations, as well as those related to the total information conveyed by stimuli, as factors in determining the recruitment of the more anterior pre-PMd relative to the more posterior PMd. Moreover, because all stimulus features were presented simultaneously, the distinction between PMd and pre-PMd cannot be attributed to the temporal duration over which stimuli must be maintained.

The results also show remarkable specificity, in that even more anterior regions of the PFC did not discriminate between the hierarchical and flat rules, indicating that second-order policies are selectively associated with this putatively "second-order" area of prefrontal cortex. The results may also rule out relational complexity accounts of these regions, because relational complexity (i.e., the number of stimulus dimensions that must be considered in order to arrive at an appropriate response) is actually lower in the hierarchical condition than the flat one, and yet a more anterior region was recruited in the former case. (Notably, even more anterior prefrontal regions did not distinguish between these rules, perhaps as predicted by relational complexity accounts of those areas).

Given that subcortical areas in the striatum are thought to be corticotopically connected with the prefrontal cortex, one would expect similarities in prefrontal and striatal activations. However, Badre et al observed a few surprising discrepancies: striatal areas showed an increase in activation in the hierarchical condition but not the flat condition, including left putamen and bilateral caudate. Effective connectivity revealed more surprises: both PMd and pre-PMd were (Granger) causing bilateral caudate activation, and that bilateral putamen activation was (Granger) causing PMd and pre-PMd activation. Moreover, these effects did not depend on the type of rule set subjects were learning.

Thus, the striatal activations support the rather mixed picture of the existing literature, in which the striatal-prefrontal circuit seems bidirectional. The results do not clearly support a strictly corticotopic type of information processing in striatum, nor do they particularly clearly support a crossed or cascaded version of that kind of architecture. Clearly, identifying the computations peformed by these striatal areas in the service of rule discovery is an important direction for future work.

I have more to say about this paper but it will need to wait until a follow-up comes out...