This is kind of sticking with the "Journal of Unsurprising Results" theme, but Inside Higher Ed today reports on a new study of student evaluations finding, well, more or less what you would expect:
One explanation could be that good students are earning good grades, and crediting their good professors for their learning. The Ohio State study, however, provides evidence for the more cynical/realistic interpretation -- namely that professors who are easy (and aren't necessarily the best teachers) earn good ratings. The way the Ohio State team did this was to look at grades in subsequent classes that would have relied on the learning in the class in which the students' evaluations were studied. Their finding: no correlation between professor evaluations and the learning that is actually taking place.
There are some problems with this approach-- subsequent classes will involve new material as well as old, and it's possible that students can get lousy grades because of problems with the new stuff, rather than deficiencies in their background. It's not worth five bucks to me to see whether they tried to control for this, though.
- Log in to post comments
One of the faculty at my graduate institution made a scatterplot showing average Teaching Assistant evaluation score versus average score of that TA's students on the final exam. The correlation, if anything, was slightly negative: the worse you thought your TA was, the better you did in the class.
My own interpretation of this is that, for a variety of reasons, the 'good' TAs are the ones who can give answers to student questions in ways where the student thinks s/he understands, and 'bad' TAs are so 'useless' to students that the students have to figure stuff out on their own. There isn't enough incentive, training, freedom, time, or direction on the part of individual TAs, for the most part, to really make sure the student gets to the point where s/he could figure out a problem on his/her own.
I am not sure how the evaluations were phrased in Ohio State, but they are usually variations on the question "would you have beer with this guy" (e.g. "Did the instructor appear interested in teaching?"). There must be better ways to determine the effectiveness of our teaching, not to mention providing feedback and actually trying to improve it.
On the other hand, the whole electoral process revolves around that question, maybe I should not underestimate it.
We've known for decades that student evaluations are at best uesless and at worst counter-productive. Why do we still use them?
Isn't it time to find a better way to engage students in a joint effort to improve university education? The student evaluations encourage the "us-versus-them" mentality that poisons the university atmosphere.
We've known for decades that student evaluations are at best uesless and at worst counter-productive. Why do we still use them?
They're the worst option except for all the others.
People want to have some way of evaluating teaching, and student evaluations provide a crude measure that doesn't require a great deal of effort on the part of faculty and administrators. I do think there are valuable things to be learned from written student evaluations (the bubble-sheet kind, not so much), but they become a problem when they're taken too seriously as a "quantitative" measure of teaching.
I would note, by the way, that our tenrue and reappointment reviews don't rely on the student bubble sheets much at all. The numbers are given to the review committees, but they're also required to have in-person interviews with at least twenty students selected at random from the classes taught by the faculty member under review. I've had good numerical evaluations over the last few years, but I had much more confidence in the tenure review process knowing that they would be directly interviewing students, and not just looking at anonymous responses to vague questions.
The problem is, interviewing a random selection of students from every class is a little too labor-intensive for regular evaluation.
I can't speak to my engineering programs as I don't think students had access to the reports, but in my MBA program the evaluation reports were extremely strongly correlated to professor quality. The best predictor in my experience was, "Would you recommend this professor to others?". Results generally grouped (1-5 scale) into ~1.3, ~1.8, ~2.3, ~3.0. Translating those into A, A-, B, C, I had about 16 profs and about 13 were dead on and the other 3 were off by one grade.
Of course, the student population were professionals in an evening MBA program and the median age was 27-29, so it's not necessarily a good counterexample, but it does demonstrate that they can be useful tools in the right environment.