Monty Hall strikes again!
Today's New York Times has this article, by John Tierney, about the latest wrinkle in the Monty Hall problem. According to M. Keith Chen, an economist at Yale University, the results of certain psychological studies are called into question by a sytematic error in their methodology. And the error, it seems, is rather similar to the one found in the classic wrong solution to the MHP:
The Monty Hall Problem has struck again, and this time it's not merely embarrassing mathematicians. If the calculations of a Yale economist are correct, there's a sneaky logical fallacy in some of the most famous experiments in psychology.
Here are some of the details:
The Yale psychologists first measured monkeys' preferences by observing how quickly each monkey sought out different colors of M&Ms. After identifying three colors preferred about equally by a monkey -- say, red, blue and green -- the researchers gave the monkey a choice between two of them.
If the monkey chose, say, red over blue, it was next given a choice between blue and green. Nearly two-thirds of the time it rejected blue in favor of green, which seemed to jibe with the theory of choice rationalization: Once we reject something, we tell ourselves we never liked it anyway (and thereby spare ourselves the painfully dissonant thought that we made the wrong choice).
But Dr. Chen says that the monkey's distaste for blue can be completely explained with statistics alone. He says the psychologists wrongly assumed that the monkey began by valuing all three colors equally.
Its relative preferences might have been so slight that they were indiscernible during the preliminary phase of the experiment, Dr. Chen says, but there must have been some tiny differences among its tastes for red, blue and green -- some hierarchy of preferences.
If so, then the monkey's choice of red over blue wasn't arbitrary. Like Monty Hall's choice of which door to open to reveal a goat, the monkey's choice of red over blue discloses information that changes the odds. If you work out the permutations (see illustration), you find that when a monkey favors red over blue, there's a two-thirds chance that it also started off with a preference for green over blue -- which would explain why the monkeys chose green two-thirds of the time in the Yale experiment, Dr. Chen says.
If I'm understanding this correctly, then the original scientists thought the monkeys valued all three colors equally. If they were right about this, then the monkey's choice of red over blue did not indicate some inherent preference for red M and M's, but merely some effectively random selection. But then in two thirds of the trials the monkey's went on to select green over blue. This would be hard to explain by random choice alone. A possible explanation is that having already rejected blue once, the monkeys reject it a second time as a form of rationalization. I was not wrong to choose red over blue, thinks the monkey, I just don't like blue!
But if we assume instead that the monkey does have a definite, but slight, ordering of preferences among the colors, then the monkey's initial choice of red over blue is also telling us something about his relative preference for green over blue. If the monkey prefers red to blue, then it is twice as likely to prefer green to blue than the other way around. Just list all the possibilities (or consult the nifty diagram included in the article).
Interesting stuff. I recommend also looking at John Tierney's blog entry, which provides some further details, or the working paper by Dr. Chen, which provides still more.
I confess I'm a little uncomfortable applying notions like cognitive dissonance to the workings of monkey brains. But it's fascinating stuff nonetheless.
- Log in to post comments
I just sat down to write a comment here on how stupid the Monte Hall effect is, and while I was writing it, I realized that it is not stupid at all, and is certainly correct.
Reflective writing and learning experience .... what are the chances of that happening ... This is all very interesting...
The Monty Hall Problem makes an appearance in the current movie 21. A student in a math class at MIT impresses his prof by getting it right. The movie is otherwise not particularly great.
Well, I am not familiar with the experiments in question, but I would imagine the investigators probably ran some internal controls? For instance, they must have checked that, in separate experimental runs, the first choice of red vs blue for each monkey was more or less 50:50, as predicted. And they should have also varied the protocol, running series in which the first choice was green vs. blue, or red vs green, and confirmed that those were 50:50 too, in the absence of previous "bias".
Basically, I understand how the Monty Hall problem could have applied, but from an experimental standpoint, it should have been obvious if it did using very simple controls. (And if the authors did not check, it will be easy to do it now by re-analyzing the raw data.)
Well, Jason, it looks like you may need to add another chapter to your book!
While it's probably fairly common for an individual experiment to have subtle statistical errors in its analysis, it would be amazing to have an entire field of study turn out to be a statistical artifact (not that Chen is making that broad of a claim).
Andrea-
I'm still working my way through the working paper, but I think Dr. Chen's claim is that the controls that were run were not adequate. In his abstract he writes:
Still, you have to wonder if it's possible for a person's preferences to be subtle enough to be overlooked by experimenters in their controls, yet powerful enough that they explain the statistical data produced by these experiments.
Kurt-
Truer than you know! I was debating with myself whether it was worth having a short “Conclusions and Summary” type chapter at the end of the book, or whether such a thing would end up being a boring rehash of everythign I said previously. But how can I resist starting such a chapter with, “Even as I write this, the Monty Hall problem continues to make news...”
No student at MIT is going to impress a professor by getting the Monte Hall problem right. I am quite sure MIT filters out nearly every potential candidate who would get the Monte Hall problem wrong at the end of high school.
On a different note, I used to walk down the halls at Washington University in St. Louis and identify errors like this on nearly every paper in the Psychology department. It made my girlfriend at the time, a Psych major, angry with me to no end. All it takes to find errors like this in a soft science is a mathematician willing to take the time...
The paper seems to suggest the typical "preference test" in the first stage consists of assigning some sort of arbitrary scale to items, and assuming that ones which get the same score have the same value. Part of the argument here is that people aren't very good at weighting value, they're much better at ranking things.
Even if you did construct a situation where there really was no preference (say, brand new coins that look identical, placed to the left and the right), it's unlikely that the subject will pick randomly---mostly because people are really really bad at being random. So there could be a measurable preference not of the objects themselves, but in a subject's "arbitrary selection" criteria (i.e. pick the one on the left, pick the new one, pick alternating, etc).
izmir evden eve izmir nakliye evden eve izmir evden eve nakliyat
I'm Louisa Egan, the lead author on the monkey cognitive dissonance study. Paul Bloom and Laurie Santos and I have just published a paper in the Journal of Experimental Social Psychology that circumvents Keith's critique. We find that even with preference-blind choices, monkeys and children still devalue options they reject, which is consistent with the previous paper. Keith's argument that our results were based on prior preferences cannot explain this.
You can find the new paper on my website:
http://www.kellogg.northwestern.edu/Faculty/Directory/Egan_Louisa.aspx#…
Or through the website of the Journal of Experimental Social Psychology.
Best,
Louisa Egan
You may be interested in this new paper: Are birds smarter than mathematicians? Pigeons (Columba livia) perform optimally on a version of the Monty Hall Dilemma.
Coturnix -
Thanks you for sending me this paper. It looks interesting.