Female Science Professor has been talking about student evaluations lately. (here are some other Student Eval posts - post 1 post 2 post 3). I had some ideas on student evaluations, and here they are.
One Question
A friend of mine likes to say that student evaluations should just be one question: "Do you like this instructor?" Maybe that is the only reliable information you can get from a student. Perhaps that can even be useful. Here is an indication of the problem. We have on our evaluation form (which is filled with useless questions) the following question:
Agree-Disagree: Instructor starts and ends class on time (1 - Strongly disagree....6 - Strongly agree)
I pick this question because it has a definite answer. Essentially, everyone should agree on this answer (including the instructor). For the past few years, I have been very careful to EXACTLY start and end class on time (and I mean down to the exact second). It has kind of become a game of mine. So, everyone should agree with the starting and ending class question, right? Well, no. I don't get 6's on this question. The scores I get are similar to the scores for the more subjective questions. Odd.
My friend from above suggests that many students treat the evaluation as though it were just one question.
The Scores
Here is the other problem. The average scores for the University are on the order of 5 out of 6. Why is this a problem? This is bad because it gives very little room for discrimination between scores. It makes it really difficult to get an above average score when the average is 5.2 and the max is 6.
This could possibly solved by giving some explicit instructions (and training) to students. How about a 1 means the instructor is in the bottom 2% (or some other value) of all instructors this student has seen and a 6 means top 2%. That would make the 6 and the 1 scores mean a lot more.
What to do with Student Evaluations?
I have to evaluate faculty that go through the tenure processes. We are supposed to use the student evaluation scores, and surprisingly I do. The important thing to realize is that the student evaluation scores are just one way to try and assess someone's teaching ability. The problem is when departments use this as just about the only measurement. For me, I really think of the evaluation scores as an indicator. If a faculty member has really low or really high scores, then I need to look at that class more carefully. Could be good, could be bad. It is just an indication that something could be wrong.
- Log in to post comments
I love the start/end on time thing. Sadly, I can't use the same test, because I'm terrible about ending on time. One student comment a few years ago said "There is no time for discussion, as his lectures are perfectly timed to take one hour and thirteen minutes" (Our lecture classes are an hour and five).
As a student these evaluations agitate me. Two of the questions we have are "Would you recommend this professor?" and "Would you recommend this class?" Both with yes/no answers. The recommending the class one is a little weird, since most classes I'm taking are required for my major, so it's not as if anyone would have a choice. Or even if I hate the class, I might recognize the information to be useful. Or if I love the class, the information might be useless. So there's a lot of ambiguity in the yes/no. And would I recommend the professor? Another toughy. I don't think it would be just a yes/no. That's a little easier than the class, but still not straightforward.
We have the same scale for starting/ending the class on time. And yes, I wish it was more explicit. Say your professor always lets out 5 minutes early. Would you then put a 1 as to not starting/ending on time even though those 5 minutes are incredibly convenient to get to your next class and the professor might have a well-timed 45 minute lecture instead of a 50 minute lecture? Or say the professor always starts 2-3 minutes late. Again this can be really convenient when you are rushing in from across campus. Versus say a professor that always goes over, and then admonishes students for leaving during the passing period when they have another lecture to get to. Mostly this is my university's idiocy in not having a decent "passing period" inbetween scheduled blocks.
And by the time I finish all these inane questions, I'm usually too irritated to provide much in one of the commentary areas unless I feel very strongly about something. Maybe it would be best to have the "Would you recommend this professor/class?" questions and then have two big fat blocks for comments. Our evaluations are open to students and I've usually learned the most from the comments rather than the arbitrary questions.
You learn very little from student evaluations other than what they like and dislike. I team taught with a very popular and very superficial colleague who was extremely entertaining, but the information content was minimal. Then I taught the very same material in the very same course in the very same way, but team-teaching with other colleagues, and my evaluations jumped up over 1 full point on a 5-point scale. With my former colleague, analysis showed that my exams were determining the grades for 4 out of every 5 of the students, so I was the "bad guy".
However in more advanced classes, if you use tailor made evaluations, you can find out some interesting information about assignments and exercises that "worked" and those that didn't, but these are usually ignored because the university wants standardization. You finally get to the point where you file them away without reading them.
I am a high school teacher, so I fill out a few recommendation forms. I like the idea of the columns being labeled like a recommendation form. Really, if someone is in the bottom half does it matter how far into the bottom half they are? Five categories: One of the best ever, top five percent, top twenty-five percent, top fifty and bottom fifty.
When I was in college a fellow student looked over at me as I darkened a 1 circle on and evaluation form. A 1 meant that the assignments had been unreasonable. This professor would assign 200 pages of reading due in two days with reading quizzes that said thing like, "in the diagram on page 457 the force vector is what color?" I thought unreasonable perfectly described these assignments. The student that looked over at my forms said, "You can't give a 1." I responded that I thought it was a perfect description and that because of all the other things this professor did I intended to give him a 5 as a professor even though I gave him a 1. I also said it was only going to be the fourth 5 of my time as a student. The whole class heard this discussion, and I looked around and saw erasers changing bubbles everywhere.
"Really, if someone is in the bottom half does it matter how far into the bottom half they are?"
Yes. Half of your faculty are below the faculty average in teaching ability. Horrid teaching is presumably a good reason not to keep someone around. Unless you're ready to dismiss half your faculty, there's value in distinguishing the truly horrid from those who are good enough, but don't match the best.