Feelings of Knowing

Clive Thompson has a wonderful article in the NY Times Magazine on Watson, the supercomputer programmed to excel at Jeopardy. Thompson delves into the clever heuristics used to generate singular answers to ambiguous questions. (Watson relies on massive amounts of parallel processing, so that "he" is running thousands of Google searches simultaneously.) While Watson's performance is certainly impressive, I thought the most interesting part of the story involved the failings of the machine. It's easy to rhapsodize about the ever escalating speed of microchips, but it turns out that Watson is often too slow at ringing the buzzer:

In more than 20 games I witnessed between Watson and former "Jeopardy!" players, humans frequently beat Watson to the buzzer. Their advantage lay in the way the game is set up. On "Jeopardy!" when a new clue is given, it pops up on screen visible to all. (Watson gets the text electronically at the same moment.) But contestants are not allowed to hit the buzzer until the host is finished reading the question aloud; on average, it takes the host about six or seven seconds to read the clue.

Players use this precious interval to figure out whether or not they have enough confidence in their answers to hazard hitting the buzzer. After all, buzzing carries a risk: someone who wins the buzz on a $1,000 question but answers it incorrectly loses $1,000.

Often those six or seven seconds weren't enough time for Watson. The humans reacted more quickly. For example, in one game an $800 clue was "In Poland, pick up some kalafjor if you crave this broccoli relative." A human contestant jumped on the buzzer as soon as he could. Watson, meanwhile, was still processing. Its top five answers hadn't appeared on the screen yet. When these finally came up, I could see why it took so long. Something about the question had confused the computer, and its answers came with mere slivers of confidence. The top two were "vegetable" and "cabbage"; the correct answer -- "cauliflower" -- was the third guess.

To avoid losing money -- Watson doesn't care about the money, obviously; winnings are simply a way for I.B.M. to see how fast and accurately its system is performing -- Ferrucci's team has programmed Watson generally not to buzz until it arrives at an answer with a high confidence level. In this regard, Watson is actually at a disadvantage, because the best "Jeopardy!" players regularly hit the buzzer as soon as it's possible to do so, even if it's before they've figured out the clue. "Jeopardy!" rules give them five seconds to answer after winning the buzz. So long as they have a good feeling in their gut, they'll pounce on the buzzer, trusting that in those few extra seconds the answer will pop into their heads. Ferrucci told me that the best human contestants he had brought in to play against Watson were amazingly fast. "They can buzz in 10 milliseconds," he said, sounding astonished. "Zero milliseconds!"

This anecdote highlights one of the most impressive talents of the human mind. We don't just know things - we know we know them, which leads to feelings of knowing. I've written about this before, but one of my favorite examples of such feelings is when a word is on the tip of the tongue. Perhaps it occurs when you run into an old acquaintance whose name you can't remember, although you know that it begins with the letter "J." Or perhaps you struggle to recall the title of a recent movie, even though you can describe the plot in perfect detail.

What's interesting about this mental hiccup is that, even though the mind can't remember the information, it's convinced that it knows it. We have a vague feeling that, if we continue to search for the missing word, we'll be able to find it. (This is a universal experience: The vast majority of languages, from Afrikaans to Hindi to Arabic, even rely on tongue metaphors to describe the tip-of-the-tongue moment.) But here's the mystery: If we've forgotten a person's name, then why are we so convinced that we remember it? What does it mean to know something without being able to access it?

This is where feelings of knowing prove essential. The feeling is a signal that we can find the answer, if only we keep on thinking about the question. And these feelings aren't just relevant when we can't remember someone's name. Think, for instance, about the last time you raised your hand to speak in a group setting: Did you know exactly what you were going to say when you decided to open your mouth? Probably not. Instead, you had a funny hunch that you had something worthwhile to say, and so you began talking without knowing how the sentence would end. Likewise, those players on Jeopardy are able to
ring the buzzer before they can actually articulate the answer. All they have is a feeling, and that feeling is enough.

These feelings of knowing illustrate the power of our emotions. The first thing to note is that these feelings are often extremely accurate. The Columbia University psychologist Janet Metcalfe, for instance, has demonstrated that when it comes to trivia questions, our feelings of knowing predict our actual knowledge. Think, for a moment, about how impressive this is: the metacognitive brain is able to almost instantly make an assessment about all the facts, errata and detritus stuffed into the cortex. The end result is an epistemic intuition, which tells us whether or not we should press the buzzer.

The second important feature of these feelings of knowing is their speed. As Thompson makes clear, it's the speed of these inexplicable hunches that allow the human contestants to defeat Watson. Although our meaty computer only requires 12 watts of electricity - we are a damn efficient information processing device - we're still able to react before the supercomputer, which requires a massive air-conditioner to cool itself down. In the human brain, these primal emotions have been bootstrapped to self-awareness, so that many of our feelings are short, speedy summaries of our own vast hard drive. They are what urge us to raise our hand, or keep on trying to remember a name, or press the buzzer.

The larger point is that we won't get a genuinely "human" version of artificial intelligence (not to mention more energy efficient computers) until our computers start to run emotion-like algorithms. What Watson needs isn't a bigger hard drive or some more microchips - he needs to develop feelings of knowing, which will tell him that he probably knows the answer even if he's still drawing a blank.

For decades, we've assumed that our emotions interfere with cognition, and that our computers will outpace us precisely because they aren't vulnerable to these impulsive, distracting drives. But it turns out that we were wrong. Our fleeting feelings are an essential aspect of human thought, even when it comes to answering the trivia questions on Jeopardy.

Update: Vaughan Bell has more.