There was no sentence in How We Decide that I regretted more than this one, which was first written in the fall of 2007, when Vince Young was the starting quarterback for the Tennessee Titans:
Vince Young ended up excelling in the pros.
I was discussing the statistical disconnect between a QB's score on the Wonderlic intelligence test - an abbreviated version of the IQ test - and their performance in the pros. The league requires that every player in the draft take the Wonderlic. The test is twelve minutes long and consists of 50 questions, which get progressively harder as the test goes along. Here's an example of a easy Wonderlic question:
"Paper sells for 21 cents per pad. What will four pads cost?"
And here's a hard Wonderlic question:
"Three individuals form a partnership and agree to divide the profits equally. X invests $9,000, Y invests $7,000, Z invests $4,000. If the profits are $4,800, how much less does X receive than if the profits were divided in proportion to the amount invested?"
The underlying assumption of the Wonderlic test is that players who are better at math and logic problems will make better decisions in the pocket. At first glance, this seems like a reasonable conjecture. No other position in sports requires such extreme cognitive talents. A successful quarterback will need to memorize hundreds of offensive plays and dozens of different defensive formations. They'll need to spend hours studying game tape of their opponents so that, when they're on the field, they can put that knowledge to use. In many instances, quarterbacks are even responsible for changing the play at the line of scrimmage. They are like a coach with shoulder pads.
As a result, NFL teams start to get nervous whenever a quarterback scores too far below the average for the position, which is 25. (In comparison, the average score for computer programmers is 28. Janitors, on average, score 15, as do running backs.) But here's the funny thing: the Wonderlic test, at least for QBs, is famously useless; the NFL is full of quarterbacks who achieved success despite bombing the test. Dan Marino scored 14. Brett Favre's Wonderlic score was 22, while Randall Cunningham and Terry Bradshaw both scored 15. All of these quarterbacks have been, or will be, inducted into the Hall of Fame. (In recent years, Favre has surpassed many of the passing records once held by Marino, such as most passing yards and touchdowns in a career.) Furthermore, several quarterbacks with unusually high Wonderlic scoresâ¯players like Alex Smith and Matt Leinart, who both scored above 35 on the test and were top ten picks in the 2005 NFL draftâ¯have struggled in the NFL, largely because they make poor decisions on the field.
All of which leads me back to Vince Young, who reportedly scored a 6 on the test. (Subsequent reports indicated that he scored a 16 on his second test.) After How We Decide went to press, Young behaved somewhat erratically and was benched. I got numerous emails mocking my NFL analysis. But now I'm delighted to say that Mr. Young is back. Yesterday, the QB with the low Wonderlic dismantled the QB with a score of 35:
Almost four years after Vince Young led the Texas Longhorns to a national-championship victory over the Matt Leinart-led USC Trojans, those two college stars -- the first two quarterbacks selected in the 2006 draft -- finally faced each other again.
Last time, Young scored the winning touchdown on fourth down at the end of the game.
This time?
You guessed it.
Young threw a fourth-down touchdown pass on the game's final play to lift the Tennessee Titans to a 20-17 victory over the Arizona Cardinals, quarterbacked by Leinart.
The point is absolutely not that the IQ test doesn't matter. Measures of intelligence remain extremely relevant for predicting academic success, and remain one of the most predictive measures of real world achievement. (Although it's worth pointing out IQ scores are still remarkably unimportant, and consistently account for less than 5 percent of individual variation in non-academic achievement.) Instead, I think the disassociation between QB performance and IQ performance is a reminder that many tasks are driven by non-cognitive factors, which rely on a very different set of mental skills. Here's how I describe it in the book:
The reason there is virtually no correlation between the results of the Wonderlic and the success of quarterbacks in the NFL is that finding the open man involves a very different set of decision-making skills than solving an algebra problem. While quarterbacks need to grapple with complexityâ¯the typical offensive playbook is several inches thickâ¯they don't make sense of the football field the way they make sense of questions on a multiple-choice exam. The Wonderlic measures a specific kind of thought process, but the best quarterbacks don't think in the pocket. There isn't time.
Instead, I'd suggest that QB decision-making requires a surfeit of emotional intelligence - the athletes must develop their unconscious pattern-detecting abilities - which isn't measured on the Wonderlic. For too long, we've pretended that making decisions in the pocket is like doing a math problem, when it's actually much closer to swinging a bat in a baseball game: the velocity of the game makes deliberate thought impossible. Until we come up with a test that reliably measures the sort of EQ required by QBs, I'd suggest that NFL teams stop taking the Wonderlic into account. Sean Jones, a member of the Oakland Raiders' personnel department, said it best when asked if Young's low Wonderlic score might hurt his chances in the NFL draft:
"All I need to know about Vince Young is that he came up with one of the greatest performances ever in the Rose Bowl. In the fourth quarter, I saw (USC coach) Pete Carroll throw every kind of blitz at Vince. I saw Vince read the blitz and beat the blitz. I don't care what his Wonderlic score is. The only score I care about is 41-38."
- Log in to post comments
"The reason there is virtually no correlation between the results of the Wonderlic and the success of quarterbacks in the NFL is that finding the open man involves a very different set of decision-making skills than solving an algebra problem" - or pushing a broom, for that matter.
"Measures of intelligence remain extremely relevant for predicting academic success, and remain one of the most predictive measures of real world achievement." What other kinds of worlds have you in mind? The question is, is the world of academic success part of the "real" world.
" Yesterday, the QB with the low Wonderlic dismantled the QB with a score of 35:"
Except that's not what happened at all. One team dismantles another team, and if a quarterback led offense dismantles a non-quarterback led defense, that's hardly the same as dismantling the opposing quarterback.
And "on any given day" is not in that sense equivalent in football to "on every given day."
In any case, the type of predictive calculus used in football is far from that used in everyday problem solving. Think experience based algorithms.
This study says wonderlic works, http://sabermetricresearch.blogspot.com/2008/05/does-wonderlic-test-pre…
The Wonderlic is a quasi-IQ test that teams hope will give them some insight about the prospects ability to learn the playbook. Profootball people recognize that it's not about what you're going to do on the field. That's why they also have scouting reports.
mind you alex smith might be back as well! maybe his decision making skills are strengthened when he operates in a spread offense. all a matter of comfort? under center he is apparently uncomfortable and therefore makes bad decisions. amazing it took the 9ers 5 years to figure this out...possibly they need to screen with the wonderlic.
I would love it if Randall Cunningham were elected in the the Hall, but it's not going to happen...
The link says there is a very small relation between the Wonderlic score and performance unless you throw away QBs with less than 1000 yards in a season. That's as bad as a sample selection bias I've seen. What's the implication for GMs? Draft a QB with a high score and he will perform well, unless he is terrible and doesn't play at all. Garbage.
It is an interesting result though. It seems to imply that the variation in QB performance increases with Wonderlic scores. High scorers tend to be either much better or much worse.
Young is finally "great", because during his five game winning streak he has been playing either against teams that have been average at best, or subject to recent severe injury problems. My prediction is that the Titans lose three of their final five games.
I have actually heard from people who work at Wonderlic that teams are somewhat nervous about taking players with too high of a score. To them, that indicates that an individual has other options (could be a doctor, a businessman etc.) and they might not want to put 110% into the NFL. (That is, they don't want to kill their body in what is a fairly violent game.) I don't know if that is true, but that is what I heard.
Also being the back-up QB and holding a clipboard on the sideline is not a terrible gig. Many of them make a few million dollars a year. To me it is very smart to perform just well enough in practice to stay in the league but not well enough to start and get beat up.
Young looked absolutely abysmal yesterday, like he was in totally over head.
"Young looked absolutely abysmal yesterday, like he was in totally over head."
Actually he had a fairly good game, just not the outstanding performance he needed to win and we expect. One dropped pass and two field goals that should have been taken instead of going for it were the difference in this game.
I've always wondered about the correlation between scholastic aptitude and athletic prowess....especially pertaining to cognitive-heavy positions such as quarterback.
I'm not "entirely" sold on this study but it definitely makes me ponder some things. I'll have to keep reading and then blog some of my own thoughts :P
A lot of anecdote here.
Where are the correlations holding other things constant?
IQ is a not-bad predictor of off-field troubles: going to jail, car accidents, dropping heavy objects on your bare foot, forgetting to stay in shape, getting shot, that kind of thing.
Most NFL quarterbacks are above the national average in IQ. This suggests that, yes, there is, on the whole, a positive correlation between IQ and making it to the NFL as a quarterback when measured across the whole population.
Among NFL quarterbacks, however, there isn't much of a correlation between success and IQ, but that's what's called a restriction of range bias in the analysis. NFL quarterbacks don't include guys who were too dumb to learn the college playbook, guys who were too dumb to stay out of jail when they should have been going to college, and so forth.
Similarly, NFL starting quarterbacks who were first round draft picks don't perform all that much better than NFL starting quarterbacks who were lower round draft picks, or went undrafted. But a higher percentage of high draft picks become NFL starting quarterbacks than low round draft picks or undrafted college quarterbacks. Unless you are Malcolm Gladwell, you can grasp that this doesn't mean the NFL guys can't predict better than random.
Do you want more interesting?
Joe DeCamillis has been around a few kickers that have played a long time in the NFL Draft (Jason Elam, Morten Andersen) and he believes Nick Folk will be one of them, too.
The Cowboys cut Folk Monday after missing 10 field goal attempts this year and signed Shaun Suisham.
"He works his craft the way you're supposed to work it, so it was a difficult decision obviously," DeCamillis said. "You don't want to see anybody struggle and he was struggling. I think he even knew it. I've got the utmost respect for him, wouldn't surprise me if he started kicking well again the league. He just kind of went into a little funk and unfortunately we couldn't get him out of it. It's disappointing on my part because you hppe to do something to revive it and get him going in the right direction and it just didn't happen."
Merry Christmas!And have a good time!
vince young is the man, he got in there and started making changes in the team since day one. he changed the team
I'm very happy to read this. This is the type of manual that needs to be given and not the random misinformation that is at the other blogs. Appreciate your sharing this best doc.