All-Nighter Ho! -- Brain and Behavior: Neuroeconomics

For some reason I have been seeing lots of neuroeconomics articles lately. Maybe it is because people enjoy using that prefix. This article caught my eye because I have been reading off the reservation -- a history of economics by Mark Skousen -- who in spite of a rather lengthy "Keynes is a complete yo-yo" rant writes a darn good history.

What I have learned from this extensive reading is that people -- at least in aggregate -- seek out the best deal. I know...utterly shocking that. Unless you are an economist. If you are an economist, you tend to write 1,400 page books and then be pissed when people don't immediately understand all the words you made up -- sort of like Mises.

(As the reader begins to wonder whether Jake will be discussing neuroscience or economics, Jake abruptly changes directions...)

Anyway, Daw et al in this weeks Nature try and explore the neural machinery that one uses to seek out the best deal. They accomplish this in their study by challenging people to play a game where the ideal strategy is not at all moments readily apparent. The subjects play four slot machines that have changing pay out rates such that at times one of the four is the best payout. Under a strictly rational model, the player would find the best and stick with it until it changed, but for this task that is rather difficult. The uncertainty promotes a "check out the other merchandise strategy" -- you know...like when you had that girlfriend you hated. The participants will stick with the best slot machine most of the time, but periodically explore the other slot machines. A review by Daeyeol Lee in the same issue summarizes this better than I am:

Standard economic theories tend to assume that people always make the decision that gives them the greatest reward -- in terms of its subjective value or utility to the person. In reality, however, people often do not know which option will produce the best outcome, as even familiar environments change continually, so decision-making strategies must be adjusted through experience2. In reinforcement-learning theories3, estimates of future rewards expected from alternative actions are referred to as their value functions, and these must be updated appropriately to reflect the actual rewards. Choosing the action with the maximum value function is referred to as exploitation. Because value functions are only estimates of future rewards, however, it is often desirable to try out actions that might seem to have suboptimal value functions, and this is known as exploration.

To see how people deal with making decisions in changing circumstances, Daw et al.1 asked their subjects to play a computer game where they could choose among four slot machines. Each machine was assigned a different payout level, and each time the subject played, the payout they received from their chosen machine was determined randomly on the basis of the mean and the standard deviation of the assigned payout. In addition, the mean payout of each slot machine drifted randomly over time, preventing the subjects from ever knowing accurately which slot machine would reward them the most.

Daw et al. considered the ways in which people might resolve the exploration-exploitation dilemma. One possibility, known as the alt epsilon-greedy rule, is to select the action with the maximum value function most of the time, but to choose randomly among the remaining options with a small probability (alt epsilon). Alternatively, people might explore more frequently if the differences among value functions of the various options are relatively small. This behaviour is described by the 'softmax' rule, which states that the probability of choosing a particular action is given by the Boltzmann distribution of the value functions (analogous to the Boltzmann distribution in physics that describes the behaviour of particles according to their energies).

It turns out that when you actually measure the propensity of people to explore the other slot machines, it follows the Boltzmann distribution related to the difference in payouts between the two slot machines.

This is not why this article is cool, however. The researchers do one better. Using an fMRI they look at the activation in the subjects brains during the decision to explore as opposed to exploit (stay with the same). They discover that the orbitofrontal cortex lights up regardless of the decision (consistent with some earlier work that you need that to appropriately value decisions), but during the decision to explore another part of brain called the frontopolar cortex lights up.

I liked this article for a couple reasons:

1) We really don't know that much about how the prefrontal cortex works. We know it makes decisions. Economics is fundamentally a decisions science, thus it is highly applicable to helping us solve this problem.

2) In the teeth of many of my friends opinions (several of whom are becoming economists), I still have a gut instinct that man is at core non-rational is several definable ways. I am curious how this is going to play out. Will neuroscience show that economic principles are to a degree hardwired or will it show that economics is wrong about how people act in very specific ways? Who knows...but I think the collaboration will be fruitful.