Wine Ratings Are For Suckers

Everytime I walk into a wine store, and see that collage of numerical stickers (This Chianti is a 91! This Pinot Grigio is an 88!), the neuroscientist in me wants to tear them all down an go on a long rant about unconscious biases. The idea that the human olfactory system can reliably decipher the difference between a wine worth 90 points and a wine worth 89 points is patently ridiculous. And yet the trend shows no signs of abating.

"On many levels [rating wines on a numerical scale] is nonsensical," Joshua Greene, the editor and publisher of Wine & Spirits, said. He has been using the 100-point system to judge wines in his magazine for about a dozen years.

Mr. Greene's ratings, especially when he awards a 90 or higher, often figure prominently in newspaper advertisements and promotional materials. Still, he said of the 100-point scoring system, "I don't think it's a very valuable piece of information." To Mr. Greene, The Number is an unfortunate remnant of a time long past, when America was only starting to appreciate wines sold in something other than a green glass jug -- akin to a set of training wheels that should have been removed years ago.

Yet Mr. Greene continues to use the 100-point system because he believes that he has no choice; to do otherwise is to court potential financial disaster.

The rest of the article is full of similar quotes by wine critics. My one complaint about the article is that it failed to discuss two of my favorite experiments in wine tasting. They were done by Frederic Brochet, of the University of Bordeaux. In the first test, Brochet invited 57 wine experts and asked them to give their impressions of what looked like two glasses of red and white wine. The wines were actually the same white wine, one of which had been tinted red with food coloring. But that didn't stop the experts from describing the "red" wine in language typically used to describe red wines. One expert praised its "jamminess," while another enjoyed its "crushed red fruit." Not a single one noticed it was actually a white wine.

The second test Brochet conducted was even more damning. He took a middling Bordeaux and served it in two different bottles. One bottle was a fancy grand-cru. The other bottle was an ordinary vin du table. Despite the fact that they were actually being served the exact same wine, the experts gave the differently labeled bottles nearly opposite ratings. The grand cru was "agreeable, woody, complex, balanced and rounded," while the vin du table was "weak, short, light, flat and faulty". Forty experts said the wine with the fancy label was worth drinking, while only 12 said the cheap wine was.

What these wine experiments illuminate is the omnipresence of subjectivity. When we take a sip of wine, we don't taste the wine first, and the cheapness or redness second. We taste everything all at once, in a single gulp of thiswineisred, or thiswineisexpensive. As a result, the wine "experts" sincerely believed that the white wine was red, and that the cheap wine was expensive. And while they were pitifully mistaken, their mistakes weren't entirely their fault. Our brain has been designed to believe itself, wired so that our prejudices feel like facts, our opinions indistinguishable from the actual sensation. If we think a wine is cheap, it will taste cheap. And if we think we are tasting a grand cru, then we will taste a grand cru. Our senses are vague in their instructions, and we parse their suggestions based upon whatever other knowledge we can summon to the surface. As Brochet himself notes, our expectations of what the wine will taste like "can be much more powerful in determining how you taste a wine than the actual physical qualities of the wine itself."

This doesn't mean that Gallo Hearty Burgundy is equivalent to a Romanee-Conti. It just means that we should be skeptical of any attempt to impose objective scales onto our subjective senses, especially when we are trying to quantify something as romantically elusive as the taste of wine.

[Self-Promoting P.S.: I discuss these experiments some more in my book, Proust Was A Neuroscientist, which will be out next year.]

More like this

The numerical ratings are kind of useful to make sure the wine in question isn't awful when you're bargan hunting and don't have any other info about the selections. A rating means somebody drank it and didn't come to the conclusion that it tastes like Kool Aid and acetone, which is a realistic fear in the $12 a bottle range.

What a superb essay..totally spot-on!

Jonah, if I agree to buy, Proust Was A Neuroscientist, can I do some topically-relevant self-promotion? Great, thanks.

The inaugural post of my Sb Friday fun-blogging feature, The Friday Fermentable, provides a nice example of wine value that came from horizontal blind tasting of 1985 Bordeaux - you could get seven-to-eight bottles of the $13 second-ranked chateau for the price of one bottle of the winner.

The rule for me is that I rarely buy anything I have not first tasted together in the context of other similar wines of various prices. Even with a minimum of tasting experience, you will find that your palate beats the hell out of any expert's rating system. Robert Parker Jr. is one of only a handful of neurosensory "supertasters" in the biz whose ratings I can trust and interpret.

The lesson here for the grad student/postdoc reader (and the modestly-compensated journalist-blogger!) is that there are incredible values to be had that sneak under the ratings radar.

Hope to meet you on my next visit to Seed and, perhaps, share a bottle of my most recent find!

Thanks, I enjoyed reading this and I agree completely. Rating wines is in about the same as rating art; you can do it, and maybe it will help someone, but it's ultimately pointless. Not everything in the human experience can be broken down by science ( yet ) ... for those wine guys, don't feel too bad - electromagnetics is in the same pickle. We use magnets every day but no one can really define a magnetic field either.

The same delusion exists in the audiophile community. I can't post the link (yes, I am spamming), but if you do a google search with "audiophile power cable abx test", it's the first hit.

Most of your points are quite correct. However, I fail to see why you and other continue to bash the 100-point system by saying that nobody can distinguish between an 89 and a 90. Of course not!

But in practice, the 100 point scale is really only about 20 points or so - 75-95. Actually, most wines are probably 85-95, so its only a 10 point scale.

Secondly, most wine critics are fully aware of the price-difference between and 89 and a 90 wine, and they rate accordingly. I bet that if you did a statistical stdy, you will find far fewer 89's than if the distribution of scores was random. In other words, critics subconsciously adjust to the human bias for numbers.

Ultimately, the 100-point scale reduces in practice to about 5 or 6 bands (eg. <70, <80, 80-85, 85-88, 90-92, 93-95, 95+). Surely nothing wrong with that!

By Vish Subramanian (not verified) on 15 Aug 2006 #permalink

very interesting post. that subjectivity extends greatly into the concept of branding. i love ravenswood wine and buy a lot of it, but blinded i would probably enjoy wine from a box as much. well maybe not.

I know little about wine ratings (I try not to buy anything over 15$ or rating under a 25/100), but I expect that they are based on summing a series of 5 or 10 point ratings (clarity, viscosity, depth of colour, duration of aroma, etc.) Each of these ratings should be fairly reliable. However, the overall sum will include a number of things that are important (depth of flavour, adherence to flavour profile for that type of wine), and others that are less important (informativess of the label, for example). For most wine drinkers, we're interested in only about 3 or 4 of the component scales, the rest are irrelevant. I can imagine a wine that has no taste whatsover still scoring a 50, while another with a delicious taste and arome, but coloured a cloudy blue, with chunky bits in it, scoring a 60. Overall scores can be quite misleading if they combine what we highly value with what we don't value as much.

I'm not so sure that it's an omnipresent subjectivity, but that the role of top-down processing is under-estimated in our sensory (or otherwise) perception. As to the distinction between an 89 and a 90 wine, I don't believe there's a pretence of reliable distinction there (although if you examine some other wine literature you might be surprised). It's very normal to take measurements with a greater deal of precision than is implied. I don't think a student with 90% in an exam is reliably better than 89% etc.

Also, in response to Stewart's post, the rating out of 100 is based on 3 components colour, aroma, and flavour (usually weighted 15%, 50%, 35%). However, most judges make a global (gestalt?) judgement, rather than working it out that way.

By James Green (not verified) on 06 Nov 2006 #permalink

Yawn.

But can you differentiate between a 100 and an 80?

I'm not going to lead you in baby steps to where this argument leads: it should hopefully be clear.

Right then, all motion is an illusion-- back to Parmenides, eh Jonah?