Steven Levitt writes about the difficulty of judging wine:
On Tuesday afternoons we had wine tastings. I asked if I could be allowed the opportunity to conduct one of these wine tastings “blind” to see what we could learn from sampling wines without first knowing what we were drinking. Everyone thought this was a great idea. So with the help of the wine steward I selected two expensive bottles from the wine cellar and then I went down the street to the liquor store and bought the cheapest bottle of wine they had made from the same type of grape.
I thus had two different expensive wines and one cheap one. I tried to make things more interesting by splitting one of the expensive bottles into two different decanters. Thus, in total the wine tasters had four wines to taste, although in reality there were only three different wines, with one sampled twice by each taster. I gave them a rating sheet and each person rated each of the four wines.
The results could not have been better for me. There was no significant difference in the rating across the four wines; the cheap wine did just as well as the expensive ones. Even more remarkable, for a given drinker, there was more variation in the rankings they gave to the two samples drawn from the same bottle than there was between any other two samples. Not only did they like the cheap wine as much as the expensive one, they were not even internally consistent in their assessments.
I’ve written a lot about the tastiness of cheap wine and the subjectivity of wine critics. Of course, the fallibility of amateur wine tasters really shouldn’t be too surprising, given everything we know about the role of expectations in shaping perception. (Cheap wine served in an expensive bottle tastes expensive for the same reason that generic aspirin is less effective at relieving pain than brand name aspirin.) We are gullible creatures; the brain wasn’t built for objectivity. For more, check out this new economics paper.