In the latest New Yorker, Malcolm Gladwell has a thought-provoking article on the difficulty of figuring out what sort of person is best suited for a particular job. He begins by discussing the challenge of choosing college quarterbacks, a topic that I've written about a few times before (and cover at length in my forthcoming book):
All quarterbacks drafted into the pros are required to take an I.Q. test--the Wonderlic Personnel Test. The theory behind the test is that the pro game is so much more cognitively demanding than the college game that high intelligence should be a good predictor of success. But when the economists David Berri and Rob Simmons analyzed the scores--which are routinely leaked to the press--they found that Wonderlic scores are all but useless as predictors. Of the five quarterbacks taken in round one of the 1999 draft, Donovan McNabb, the only one of the five with a shot at the Hall of Fame, had the lowest Wonderlic score. And who else had I.Q. scores in the same range as McNabb? Dan Marino and Terry Bradshaw, two of the greatest quarterbacks ever to play the game.
Why is the Wonderlic so useless at predicting which quarterbacks will succeed? The reason is that finding the open man during an NFL game involves a very different set of decision-making skills than solving a problem on an IQ test. While quarterbacks need to grapple with complexity - the typical offensive playbook is several inches thick - they don't make sense of the football field the way they make sense of questions on a multiple-choice exam. The Wonderlic measures a specific kind of thought process, but the best quarterbacks don't think in the pocket. There isn't time.
So how do quarterbacks do it? How do they make a decision? It's like asking a baseball player why he decided to swing the bat: the velocity of the game makes thought impossible. What recent research in neuroscience suggests is that quarterbacks choose where to throw the ball by relying on their unconscious brain. Just as a baseball player will decide to swing at a pitch for reasons he can't explain (he' is acting on subliminal cues from the hand of the pitcher), an experienced quarterback picks up defensive details he's not even aware of. Although he doesn't consciously perceive the lurking cornerback, or the blitzing linebacker, the quarterback's unconscious is still able to monitor the movement of these players. And then, when he glances at his receivers, his brain automatically converts these details into a set of fast emotional signals, so that a receiver in tight coverage gets associated with a twinge of fear, while an open man triggers a burst of positive feeling. It's these inarticulate emotions, and not some elaborate set of rational calculations, that tell the best quarterbacks when to let the ball fly. The pocket, it turns out, is too dangerous a place to think.
Gladwell goes on to argue that the failure of the Wonderlic and the dismal track record of NFL scouts suggests that drafting a QB is ultimately a crapshoot:
We're used to dealing with prediction problems by going back and looking for better predictors. We now realize that being a good doctor requires the ability to communicate, listen, and empathize--and so there is increasing pressure on medical schools to pay attention to interpersonal skills as well as to test scores. We can have better physicians if we're just smarter about how we choose medical-school students. But no one is saying that Dan Shonka is somehow missing some key ingredient in his analysis; that if he were only more perceptive he could predict Chase Daniel's career trajectory. The problem with picking quarterbacks is that Chase Daniel's performance can't be predicted. The job he's being groomed for is so particular and specialized that there is no way to know who will succeed at it and who won't.
I'm not that pessimistic. I think the real problem is that, for the last several decades, we've just been measuring the wrong things. We've assumed that passing decisions are rational decisions when, in fact, there's nothing rational about them. Obviously, it's a bit more difficult to measure the unconscious, but that doesn't mean it can't be done.
I'd also argue that this new emphasis on the learned expertise of the unconscious - as opposed to the innate intelligence measured by the Wonderlic - suggests that an essential determining factor of how a QB will perform in the NFL is how they are trained and taught. Look at the Patriots. After Tom Brady was injured, Matt Cassel - a QB who hadn't started a game since high-school - was able to become a surprisingly good substitute. (This doesn't mean the Pats will make the playoffs, but that's another story.) Why did Cassel succeed when so many other high-profile QB's fail? I think part of the answer is Bill Belichick's emphasis on situational practice.
What separates Patriots training camp is the attention to detail and emphasis on situational football. Two-minute drills, four-minute drills, and coming-out-of-your-own-end-zone scenarios are situations in which the Patriots preach what they practice.
"The most important thing that I saw that he did in his training camps, the thing that he does that in my opinion makes that team better is that he has specific practices set aside for situational football," said former Patriots tight end Christian Fauria.
Fauria, who played in New England from 2002-05 and also has played for the Seattle Seahawks, Washington Redskins, and Carolina Panthers, said in his first camp with the Patriots, Belichick created a situation: It was late in the fourth quarter and the Patriots had the ball at their 17 with 1:21 left and no timeouts, needing a field goal.
Sound familiar? Belichick recreated the situation that led to Adam Vinatieri's field goal in Super Bowl XXXVI.
"They redid that situation. I remember that sticking out in my mind," said Fauria. "Ultimately, those situations can make the difference between winning and losing, being able to make those small, quick decisions that the other guy might not know."
That is how you train the unconscious: not with dry recitations of the playbook, but with realistic simulations. As a result, players are able to practice decision-making - what should I do on fourth and short with a corner blitz? - just as they'd practice throwing a deep pass.
- Log in to post comments
Gladwell's article left out some major arguments against his focus on innate ability. You rightly point out that realistic training for NFL style games might help quarterbacks perform better in NFL games. His leap, from the implication that great quarterbacks are born, not made, to the idea that good teaching skills are innate seems even more of a stretch.
Clearly, the problem here is that our training programs for teachers (and NFL quarterbacks) is inadequate, not that excellence in these fields can not be taught. In the article, he devotes a fair bit of time describing how easy it is to separate good practices from bad practices when viewing tapes of teachers in the classroom. There is even a discussion in a case when a teacher gave a child an adequate, but not inspired, response as to what a better response would have been. In other words, the path to improvement was obvious to everyone viewing the tape.
It takes very little imagination on my part to envision a program where prospective teachers are taught classroom skills, perhaps by viewing tapes such as the ones described in the article, or by extended mentoring under successful teachers. Yes, intuitiveness is helpful, but communication skills can be learned with varying degrees of success.
And a little more success in the classroom might be all we need. Gladwell references Eric Hanushek, a Stanford economist, who concludes that the US can raise its academic performance level "...simply by replacing the bottom six percent to ten percent of public-school teachers with teachers of average quality." This does not seem an insurmountable bar.
I'm sure there is quite a literature on this in fields like training pilots in emergency maneuvers, etc. Look for domains where responses to crisis have to be made in real time, the costs of a failure are very high, and where there's good institutional support for analyzing failures and regulating training.
A very different domain to analyze training and control over cognitive issues is civil engineering, which is at the other end of the time scale of responses, but where again the costs of failure are high and the institutional controls are strong. Here the emphasis is on long term sound practices and social controls.
Ship navigation is more or less in the middle of this range. Some interesting ethnological work has been done on that but I can't generate a reference offhand.
I find it interesting that there are very few or no effective institutional controls and learning mechanisms, even within institutions, for financial activities. In that domain both short term and long term behaviors have potentially very high costs (and benefits) but for some reason the kinds of institutional mechanisms that work well in airline safety, ship navigation and building bridges are not present. I wonder why.
Someone please help me, what is Payton Manning's IQ? I think what he does in a game more correlates with the what many reliable and valid measures of cognitive ability actually test for.
I quickly did some research on the Wonderlic and the only area that I see germane to being a Pro QB would be testing for the ability to understand and follow directions (tied in with Verbal Ability). I think what would be better predictors of a QB's performance would be more executive functioning tasks ( i.e. Delis-Kaplan-Executive Functioning Scales or Wisconsin Card Sorting); combined with tests for visual fluid reasoning (i.e. Raven's Progressive Matrices), also tasks of cognitive fluency may be beneficial.
Getting back to Payton, he calls all his own plays and EXTENSIVELY audibles. His ability to read defenses and fluidly adapt is unprecedented. So, although I agree with Jonah that "in-the-moment" decision making of who is open is unconscious and can be increased by practice (where fluid reasoning ability is innate and non-trainable) there are a lot of rational cognitive factors that a QB with a high IQ could employ that would allow for a higher frequency of people being open, thus making them better QBs. So I agree: the NFL is looking at the wrong things. Specific cognitive strengths are different than having an overall high g.
I teach an integrated circuit design class. My course project this semester was a very simple neural network that is a quarterback network, and has a single output, the decision to throw the ball. While I vastly oversimplified the problem in order to have designs that would be small and fast, and of reasonable size, the problem was interesting enough to motivate the students. In teaching design I believe strongly in teaching design approaches and strategies, then having the students carry out design problems in succession, with each one having a small change, twist, complication. Repetition with variation enhances learning, I think. Then in the heat of exams, the Rose Bowl, the presidential debates, continued practice manifests itself in unconsciously knowing what to do or say.
The blog is great, Jonah!
Alice Parker's comments brought back a memory from engineering undergraduate school where Jon M., a fraternity brother, always seemed to struggle with his grades in the electrical engineering curriculum. Yet Jon could seemingly always fix a broken audio tape recorder (if video existed, it was not in our house), or rig up an early warning system that would alert us about a forthcoming "raid' from the pledges. Jon graduated and went to work at Cape Canaveral at a level barely above that of a technician but soon rose to some prominence because of an uncanny ability to identify and effect the fix of electrical malfunctions that caused "holds" in the countdowns to liftoff.
I would also be interested if anyone knows if studies have been done on the effectiveness of pilot selection by the Navy and Air Force particularly in the case of single seat aircraft pilots. My son was an F-16 pilot and spoke to me during training about something like "situational overload" that trainees had to cope with in view of all the systems on the aircraft.
One of the sections in the article highlights the success of direct feedback in the classroom. This is a good memory trigger: which teachers do you remember most vividly and does that relate to being singled out for individual abilities or talents? Early recognition, even at the crayon stage, can prompt confidence to explore. Not crushing young minds with bare bones yes-no answers permits all the varieties of learning styles to live and flourish for all the decades that follow, no matter what professions are chosen at different points in life.
Gladwell's article about football taught me much I didn't know about that sport. But he fumbled his central points. Unlike the difficulty in replicating the NFL quarterbacking experience in college, their IS a clear possibility for new teachers to be observed in whatever real-world teaching situations they aspire to. Here in Washington State it's called student teaching, 3 months worth in a real school classroom with real kids, under the mentorship of a real teacher. If the mentor and sponsoring college fail to observe and predict in this situation, it is a huge failure on their part, not because there is some mysterious unpredictability involved. Just make sure the newbie is teaching in the age and social setting he/she intends ultimately to teach, for crying out loud! Gladwell's article shows that this level of pre-professional experience is not possible in the quarterbacking profession, so any analogizing between the two is a bad call.
The techniques of great teaching are known, identifiable in advance, and teachable to certain identifiable personalities - Gladwell shares some of them in his article. Bag the football analogy and share more of these education play-by play analyses, please. EVERYONE should know what made their favorite teacher so great, and it's as easy to understand as football. But many teachers (and certainly their unions) will welcome the aura of mysterious unpredictability Gladwell casts because it prevents the kind of empirical accountability for teachers that we expect of the over-tested kids.
The codification and teaching of these skills in the Teacher Ed factories is not always pursued, again because it would force many Ed teachers into critically examining themselves and each other as well as their students. The certifiably worst teacher I ever had was in a college Music Education course. By the way, the Education teachers who are as smart, goal-oriented (more football wordage!) and thick skinned as a top financial-advice manager aren't getting paid their worth either, certainly not according to how many great teachers they graduate.
But Gladwell kicks a touchdown (see what I learned?!) in recognizing the role of great teaching (along with great parenting, I might add) as a prime factor in learning. Give my kid one great teacher and she wont need any stinking tests. Bring on the great teachers and pay'em what they're worth! They're out there and identifiable in advance, Malcolm.
p.s. to reflect current demographic reality Gladwell's reference to "wiggly Janes, Lucys, Johns, and Roberts" should include some wiggly Rishalbs, Aquibs, Shoys, Xin Wangs, Ahneekas and Vishaals.
I think the similarities between choosing good teachers and choosing successful quarterbacks are overstated here. In the latter case, one is trying to select someone who is in the 99.999 percentile for relevant skills, out of a pool of applicants in the 99.99 percentile. A screening mechanism that fine grained may simply be infeasible. For teaching, we're mostly trying to identify and screen out applicants who would perform in the bottom quintile or quartile, given the current distribution of performance. That at least sounds much easier.
There is a whole seperate but related issue with regards to teachers: the school bureaucracy does not reward excellence, in fact it tends to punish it. Most if not all school principals, esp. elementary principals, do not want highly excellent teachers that will make all the other teachers look bad. This creates all sorts of problems with parents (who want to know why their child's teacher is not doing the same thing that little Johnny's teacher is) and other teachers (who tend to be female and easily threatened and made insecure by others that clearly outshine them).
As a grade school teacher I can attest: The issue was never knowing which teachers were awful, we ALL knew them. We could name them, and every teacher (save the guilty) would name the same names. The issue is always not having the guts or capability to fire them. The system is broken: I should be able to fire a non-performing teacher in 3-6 months. I should need agreement from some kind of internal peer council, but it should not take a year to determine results: 3 months is sufficient. By the same token, there can be no blacklisting, perhaps a training program is necessary to provide 2nd chances. But a teacher cannot be allowed to fail his/her way through a career, there needs to be a limit, because some people simply cannot succeed at being a teacher, and that's a fact.
Memorization learners by nature - of everything - are those who excel in sports. Those with the repetitive eye of the camera.
While not a fan of NFL ball (at least up until now) I am familiar with "Monday morning quarterbacking" which is something we all do after the fact, especially if we 'lost the game'. So we listen to 'alternative plays' that we might have used from a well-meaning friend or therapist or mentoring teacher and ideally we enact them those plays in some sort of situational training before the next 'game'. Simply mentally rehashing 'plays' is all left brain; to influence your unconscious non-thinking right hemisphere invovles doing, whether it's QBing, dancing, teaching in a classroom, or going one on one with a client.
Yes it is important to do this situational trainiing and to do any left brain prep like mastering the content of your lesson plan if you are a teacher, but whether you are going to be able to access any of this training on a given day is going to depend on your state of mind: are you relaxed, confident in the ability of your unconscious mind, present and in tune with the situation? Are you able to drop any extraneous mental stuff that might distract you, like memories of your performance the last time you 'played this opponent'? The only training that cultivates this state of mind is meditation as far as I know, though I'm sure that there are aspects of athletic training that approach
this. Athletes spend long hours doing simple physical repetitive actions and try and stay present I assume, as opposed to those of us who watch TV on our treadmills. Sportsman of the year Michael Phelps was able to visualize (another form of doing)what he had to swim stroke by stroke to win at the Olympics and then he enacted it. That's a well-trained mind and body!
I think the similarities between choosing good teachers and choosing successful quarterbacks are overstated here. In the latter case, one is trying to select someone who is in the 99.999 percentile for relevant skills, out of a pool of applicants in the 99.99 percentile.
Yes, discrimination at the tails is subject to greater error.
sohbet
thank`s:)
Hi guys. You always pass failure on the way to success. Help me! Looking for sites on: Stock trading for dummies. I found only this - internet stock trading. Williams, on the day emotion of trading, stock trading. It will extract whether you are a future, $200 or corporate industry, stock trading. Thank :o Carlisle from Hungary.