Larry Moran thinks I have the wrong idea about teaching evaluations and "thin slicing":
Unfortunately, Dave Munger seems to draw the wrong conclusions from this study as he explains in an earlier posting [The six-second teacher evaluation]. In that article from last May he says ...
So we do appear to be quite effective at making judgements about teaching ability even after viewing only a total of 6 seconds of actual teaching, and without even hearing the teacher's voice.
This is dead wrong. Students are good at evaluating something after six seconds but it sure as heck ain't teaching ability.
If you'd like, you can check out my (and PZ Myers') responses over there, but I did want to add a few words here. First of all, in a way, Moran is right. Student evaluations of teachers are not the only way of measuring teacher performance, and they very likely aren't the best way. However, if you read my entire post from last May, you'll see that wasn't my point.
In the Ambady and Rosenthal study, teachers weren't just being evaluated by students. Principals also evaluated the teachers, and those ratings correlated with the 30- and 6- second "thin slices." Also, the "30-second evaluation" is a bit of a misnomer, because the 30 seconds were actually 10-second snippets taken from three different parts of a class. That's a whole lot more information than just a single 30-second snippet, because teachers were probably doing different things in each 10-second snip. This is what correlated significantly with end-of-course and principal's evaluations of teacher performance.
Also, there's a bit of verbal maneuvering (perhaps unintentional) in Moran's post. I say we can make "judgements" about teaching ability in 6 seconds, but he changes that to "students are good at evaluating," which implies a higher level of analysis. In fact, I reported that along certain dimensions of teaching ability, the judgements that can be made in 30 seconds are correlated with end-of-semester judgements: dimensions such as "confidence" and "optimism" correlate at an impressive level of r=.82 and .84 respectively. But when we evaluate a teacher, we're doing much more than offering an impression of his or her character; we're assessing how well he or she can help students learn. Obviously confidence and optimism are good traits for a teacher to have -- and it's impressive that our 6-second judgements about of these traits aren't much different from 6-month judgements, but in the end, that isn't what we mean when we're talking about a good teacher.
My take-home message from the post I wrote last May was simply this: that end-of-semester student evaluations aren't much better than initial impressions, so if you're a student trying to decide whether to take a class, you might as well just visit on the first day; it's just as effective as poring over those guidebooks of the student evaluations that are distributed on many college campuses. (Even more important: are you interested in the content of the class -- you'll spend more time studying on your own than you will in class!)
The point of the "Blink" methods now being applied in the classroom post was to show how results like Ambady and Rosenthal's are, well, being applied. At least in the case of the Charlotte-Mecklenburg schools, it's not true that teacher evaluations are being placed in the hands of students; rather, principals are using thin-slicing to evaluate a larger number of classes than was previously possible. I'd certainly rate this as an improvement over the system that was in place during my brief tenure as a teacher, which is that the principal didn't visit classes at all.
But if you find yourself agreeing with Moran, or if you think I'm misrepresenting him -- or myself -- please let me know in the comments.
- Log in to post comments
A 6-second evaluation judges nothing more than looks.
In my 2 years of casually studying charisma and how people make very rapid judgements about other people I've become a little frightened by how much of it is extracted from looks. Studies have shown, for instance, that a majority of the outcomes in political races can be predicted by having people do nothing more than compare still photos of each of the candidates.
I once had a colleague, born & raised in the American Midwest, who married a Hindu from India & adopted quasi-traditional dress. Her student evaluations before her conversion never mentioned the quality of her spoken English, but after changing the way she looked, students began to remark on her evaluations how difficult it was to understand her English with the strong accent. No matter how you "slice" it, those evaluations were, not to put too fine a point on it, xenophobic, racist crapola. The reason you get a correlation between "thin slice" evaluations & those by a principal or by students at the end of the semester, is that all three evaluations share the same kind of bias, which is often, as Steve S. writes above, based on looks, gender, race & other factors that have nothing to do with teaching. All the thin slice proves is that people's prejudices are quickly formed & rigid after they are formed.
I have to ask: why does everyone keep insisting on referring to the old-fashioned concept of "judging a book by its cover" as "thin-slicing?" Lets be realistic, Gladwell hasn't invented anything new here other than a trendy name.
I agree that physical attractiveness can have a big impact on on people's impressions of others. However, Ambady and Rosenthal controlled for attractiveness in their study.
I agree with Tommy. Gladwell just made up a new term and described preexisting research. What happened to terms like "intuition", and why did Gladwell choose to use new terminology rather than being consistent with past thinkers? I'm glad it's getting increased popular attention, but it could've been done better.
Dave--
Interesting and persuasive post. I tend to buy it. (Would have to delve into the evidence a lot more to be sure.)
I wonder how the 30 sec approach interacts with prejudices.
Let's take an area with less ideological baggage than most. Let's take for example a really seriously overweight and not otherwise very good looking teacher -- who was for example objectively an excellent one, as say evaluated by the prior college from which he/she has moved over to the new one where he/she has little widespread popular reputation.
Are there any research results showing or even hinting at how the 3 x 10sec slices method would work in that situation?