Academic Poll: Correlation and Evaluation

The always fraught question of student course evaluations has come up again on campus. In discussions, the correlation between "expected grade" and "overall evaluation" has once again been noted-- that is, students who report expecting a higher grade are more likely to give a good overall score to their professors than students who expect a lower grade. Which, of course, does not indicate a causal relationship between those two things, but that doesn't stop us from spinning hypotheses. Thus, a poll question:

Failure to respond to this poll will result in failure in the course, so please vote early and often.

More like this

There's a paper in the Journal of Political Economy that has sparked a bunch of discussion. The article, bearing the snappy title "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors," looks at the scores of over 10,000 students at the US Air Force Academy over…
A former Texas public school teacher has sent me some stories from their career there. It's not pretty. The situation is what I also recollect from my long-ago days in a Yankee high school, though, so I don't know that we can just blame Texas, but it's true — the system is often set up to give…
One of the perennial problems of teaching intro physics is getting students to do their homework, so I was very interested to see Andy Rundquist on Twitter post a link to a paper on the arxiv titled "How different incentives affect homework completion in introductory physics courses." When I shared…
Amongst the other TAs and the lab coordinators in my department, I have a reputation of being a tough grader. At the end of the semester, when the course admins calculate grades, my students invariably get a few points added to their lab scores -- this is done to bring lab scores more in line with…

"Students who expect a good grade usually do so because they're happy with the quality of the instruction."

Not sure which button to click for that sentiment.

Students who expect to receive high grades give more positive evaluations to their professors, independent of learning.

Students who expect a high grade tend to suck up more.

Honestly, I think you can't make a generalization. One teacher can be excellent at conveying information, making it understandable to the students, while another could just be trying to keep the students happy. Based on most questions in evaluations, there's little to discern which it is.

One of my favorite deans had this policy for evaluating student class evaluations:

He would subtract the average expected grade (on a 4 scale) from the evaluations (on a 4 scale).

The highest possible score for a faculty member would be a class that gave the highest evaluation marks despite the fact they were all flunking.

I thought that was a pretty good way to balance things out; obviously I am highly skeptical of the first option on the poll.

By Anonymous Coward (not verified) on 11 Feb 2011 #permalink

This poll needs to be check-boxes not radiobuttons, as the correct answer is clearly a combination of the above.

Slightly modified digitalbusker:

Students who expect a good grade are only very rarely seriously unhappy with the quality of the instruction. (They may explicitly link the two, or they may not, but they're unlikely to be thinking "I'm going to get a good grade despite being taught by this useless eejit".)

[x] tickybox

Oh, I meant to say also:

People always like to assign bad things to an external cause; it's so much easier to cope with a failing grade if you 'know' that it wasn't your fault. You could have done so well if only you'd had a different professor, or Pluto hadn't turned up in your dorm and conjured everyone to Hades for a party weekend, or....

All of the above?

Student evals are worthless because the admins keep the crap profs.

I suspect Chrisj is right -- we usually like to find reasons why it's not our fault when things aren't going well. If you're getting a bad grade in the class, it's more likely you're going to look for a reason other than either your lack of effort or your lack of ability to explain it.

I have heard anecdotally that student evaluations are not correlated very well with how much students learned-- which brings into question the whole reason for doing them in the first place, of course. (monson @#9: this is the *real* reason that student evals may be worthless!) I did a bit of Google Scholar trolling to see if there are articles about how well student learning correlates with student evaluations, and it's hard to find. I did find this: http://jmd.sagepub.com/content/31/1/16.short -- The abstract, at least, suggests that the correlations are present but very weak, and only present sometimes.

The interesting thing about the correlation is that it disproves the contention that evaluations are randomly generated.

I think it is important to note that different people prefer different lecture styles. In some cases the correlation might be attributed to this alone. However, most likely it is an intermixing of several effects.

I actually have thought that I was going to well despite the professor. But this is only in cases where I learned the material in a previous class, and I felt that it would have been much more difficult to learn the way this new prof. presented it.

It looks like most of your respondents are academics in denial and are looking for other places to place blame.

I see that my choice - "some third factor" - ties for last place with "beings of pure intellect." But I think there really is a third factor other than ice-cream. There are different teaching styles and different learning styles. When the styles are compatible it is only to be expected that the evaluation would be higher, more learning would take place, and the expected grade would be higher. When the styles are less compatible the opposite occurs.

I can think of several ways of testing the hypothesis, but none that doesn't either potentially violate the anonymity of the evaluations or make a prior assumption that actual grade performance and expected grade are highly correlated.

Our student evaluations have two components; a scored set of 9 questions and the opportunity for written comments. The scored questions are useless but the comments are very valuable. I have made changes to the approaches I use in class based upon these comments.

By Hugh Miller (not verified) on 16 Feb 2011 #permalink

Before voting, I believed it was a combination of 2, 3, and 4, although I eventually pressed 4.

By the way, I'm giving student evaluations in my Statistics class tonight. I love you for publishing this!

By Young CC Prof (not verified) on 16 Feb 2011 #permalink

Why is there such a strong tendency to look at the students who would place some of the blame for poor performance on the professor? Cetainly, some of these students are making excuses, but everybody here seems very inclined to immediately assume the professor could not possibly be a bad teacher. This makes no sense to me.

I was, in the past, a very very bad student. I acknowledge that, and I accept the responsibility for the grades which resulted from it. But I've also become a much better student, and have had experiences with both very good and very bad professors. I've seen this issue from both sides, and while I may have had some tendency in the past to shift blame off of myself, I feel very insulted that people would assume bad grades are always the fault of the student.

Bad teachers DO exist, just as surely as bad students exist.

Clearly I did not proofread the first sentence of that post, but I think my meaning is understood :(