Stuff That Doesn't Belong on Student Evaluations

This was a good week for "Chad bristles at side issues of massively reshared stories," with the Vox and gender bias stories, and also this PBS piece urging parents to tell their kids science stories. That probably seems surprising, given what I do around here, but while I fully endorse the end of that piece, the opening section in which Wendy Thomas Russell explains why she never liked science mostly makes me think that she's an awful person. She attributes her lack of interest in science to bad teaching, and provides a series of examples ending with:

Later, at the University of Nebraska, I was able to avoid math and science for the most part (the journalism department was kind to me). I did take one astronomy class — and was pretty excited about it! — until I realized that the teacher was a very old Japanese man whose heavy accent destroyed any chance I had at making sense of the universe.

He pronounced “star” like this: “stah-waaaah.” I barely scraped by with a C-.

Seriously? You know, there are a bunch of valid criticisms that can and should be made about uninspiring science teaching. Complaining about people's foreign accents is not one of them. Openly mocking said accent is well over the line into Not OK.

There's a bit of irony to this, as another of the pieces being massively reshared around the same time was this interactive chart comparing RateMyProfessor evaluations for men and women. This provides a nice illustration of biased language used in evaluating faculty-- see the write-up at the NYT for examples if you can't think of stuff on your own. Among the not-okay things cited as being mentioned more frequently for women are comments about appearance and personality (though as the NYT article notes, these are less frequent than you might think). Those are definitely on the list of things that make faculty say "I can't believe I have to deal with this horseshit."

Right up there on that list with "needs new clothes" is "has a thick accent." That sort of thing always makes me roll my eyes when it turns up in student comments. But "He pronounced 'star' like this: 'stah-waaaah'" goes past eye-rolling, to "This student's evaluations should be disregarded because the student is a bigoted asshat."

Bad instruction is a real problem in science, and there are valid criticisms to make about poor teaching turning people away from the subject. "The class was presented in a confusing manner" is perfectly valid. "The lectures were extremely abstract and boring" is an appropriate complaint. "The professor talked funny" is not. That has no place on an anonymous student evaluation form, and it's completely inappropriate in a major media outlet.

More like this

My take on Ms. Russell's comments is that this was something she needed to declare for her to come to grips with her "failure" and move on. It isn't so much a criticism of the teacher/teaching as it is a cleansing. That is not to say that accents are not a problem. I struggled with accents, my own and others, throughout university. Is coping with such an unreasonable challenge of education? I am still unsure today. Nonetheless, in an idealistic sense, students should not critique teachers for behaviors and temperaments that are fixed and unchangable, or will be ignored. But how do students know that? And does that serve their needs?

By Bruce Fowler (not verified) on 08 Feb 2015 #permalink

There are areas where language can be a valid issue-- that happened to me in grad school, with the professor teaching Quantum I. But the problem wasn't just that she had a thick accent, the problem was that her English was sufficiently bad that she couldn't or wouldn't re-phrase any of her explanations. What she presented in class was almost word-for-word what was in the textbook (down to faithfully reproducing the two jokes in the book), and if you asked her to clarify something, she would repeat exactly the same thing she had just said, a little slower.

That's a language barrier that is a valid and appropriate issue to raise (and it's why I switched to the other section for Quantum II). Funny foreign pronunciation, on the other hand, is something you need to learn to deal with, unless your intended career path is "Faintly Racist Stand-Up Comic."

I think the opening is intended to be mildly self-deprecating, to set up the story stuff later on. But I think it's badly miscalibrated in that-- had she stopped at just mentioning the heavy accent, I would've sighed and moved on. Phonetically reproducing that accent as a joke crosses a line, at least for me.

The reviews are subjective, and there is a definite self-selection bias in the modern, on line methods. When they were done during the last class, on paper, I would guess there was less, or at least a different, self-selection bias, in favour of those that at least showed up.

Also, in the sciences and math, there is likely a strong disparity between those in field and out-of-field. The out-of-field may be more likely to walk in with a negative bias, and things that in-field students handle bother outsiders more.

A better, though impractical, indicator might be to get reviews a year, or several years, down the line. I have found that many of the students that curse me out the most are the ones that come back and thank me after a few years when they see what they did learn,and where they are relative to their peers.

These evaluations always have been, at best, a minor indication of `quality' in teaching. Way back when I took first level modern physics, my section leader, at a school in the US with the uniform instructional language english, spoke only greek. She was quite knowledgeable, and anything that didn't require discourse, she was awesome. The other section leader was male, personable, and didn't know the material. Guess who got the higher ratings. I went to her section because I wanted to learn.

The self-selection definitely rules out using online sources like RateMyProfessor for any kind of serious study of student responses. This was raised by somebody or another on Twitter, who mentioned a class discussion where they asked students how many looked at RateMyProfessor, and almost the whole class raised their hands. The follow-up, "How many of you post ratings there?" got almost nobody. Which illustrates the point pretty well.

This is an issue that can be dealt with in official evaluations by providing some incentive for completion of the evaluations. I've talked to people who use on-line systems for course evaluations, and they get nearly all of the students to complete the on-line survey by not letting students see their final grades for the semester unless they've filled out all the course evaluations. The counter-argument is that this doesn't make them take it seriously, but my counter-counter-argument to that is that if you think all the students are putting deep and careful thought into filling out paper evaluations in class, you clearly haven't been paying attention.

As a Nebraska alumna, I can say the student got the nationality of the teacher wrong. He was born in Hong Kong. And had been speaking English since he started school. Which sticks out at me: she paid enough attention only to note the professor was an East Asian man with an accent*.

My stepmother is a chemist and apparently her student evaluations comment on her fashion sense.

* I suppose it's possible that she went to UN Omaha or UN Kearney and they had a Japanese professor with an accent, or that I spent four years as an astronomy minor and no one mentioned a previous hire.

By Becca Stareyes (not verified) on 08 Feb 2015 #permalink

If you have any questions about why the recent string of measles cases are associated with a theme park rather than unvaccinated children, read no further than her thanks to the journalism school for not requiring more than a random astronomy class. Innumeracy is a close second.

I am struck by the fact that a journalist never even thought of the alternative to lecture called "reading the book". She could have done that during class, in the back row where kids used to do crosswords and read Playboy and now play games on Fb or view porn.

Which leads to my suggestion, for her if not for you. Maybe, just maybe, what we need are well-written science story books for the elementary grades (and middle grade books that can be read for fun). Yes, that would be similar to segregating African-American literature in its own "reader" that gets pulled out every February, but better that than the alternative. The teachers might even learn something they can put to use in the classroom.

By CCPhysicist (not verified) on 08 Feb 2015 #permalink

Nope! If students can't understand the instructor's accent after a couple classes, they're not going to learn effectively. It's unfortunate as hell, but that instructor is not a good fit. Do you suggest they just sit through the class, get a bad grade, fail to learn much, and say "Ah well, too bad I couldn't understand the instructor, but that's just the way the cookie crumbles."?

(None of which is to say that it's OK to make fun of accents.)

By Tim McCormack (not verified) on 08 Feb 2015 #permalink

I agree with you, Chad, and I want to observe that Wendy Thomas Russell is not unique. I occasionally run into a certain type of person who is comfortable with simultaneously proclaiming that we need more diversity in science (agree 100%) and that a big part of the problem is that science is full of foreigners with funny accents. There is a germ of truth there, in that a scientific community that was more diverse would offer a wider range of role models for students to relate to, but they need to frame it in terms of who is missing, not the "heelarious accents of those Asians amiright?". The framing that they are using is part of the problem, not part of the solution.

You think not being able to understand the instructor is not a legitimate complaint? The mind boggles.

By Robert Jacobson (not verified) on 08 Feb 2015 #permalink

There's a difference between a "funny accent" and an accent that is actually difficult to understand. I've come across students who will complain about a colleague's "thick" accent when they really mean "noticeable." I've had many a colleague whose accent was noticeable but clear, and whose command of grammar and vocabulary far exceeded that of the student voicing the complaint.

I agree with Alex @10. Several foreign-born colleagues of mine have gotten complaints on student evaluation forms about their accents. (Not just Asians, either.) In most cases, the real complaint is that the foreign accent is noticeable and the student doesn't feel he should have to adjust for it. Remember that most American college freshmen have little or no experience with non-American accents, and much of the experience they do have is with the occasional comic-releif foreign character on a TV show. That doesn't excuse failing to adjust for a foreign accent when they do encounter one, but it explains why it happens more often than it should.

I do know some people whose accents are thick enough that I, who have considerably more practice than the average freshman at deciphering foreign accents, have to concentrate to understand what they are saying. But most of the people I know who fit that description don't have teaching positions, at least not in English-speaking countries.

By Eric Lund (not verified) on 08 Feb 2015 #permalink

Since this journalist knew that "stah-waaaah" meant "star," she obviously did understand what the professor was saying.

By Ori Vandewalle (not verified) on 08 Feb 2015 #permalink

This is an issue that can be dealt with in official evaluations by providing some incentive for completion of the evaluations. I’ve talked to people who use on-line systems for course evaluations, and they get nearly all of the students to complete the on-line survey by not letting students see their final grades for the semester unless they’ve filled out all the course evaluations.

That approach was suggested by our faculty when the administration went to having evaluations done on-line. It was turned down as it "punished students".
The alternative suggestion from the administration: Put a notice in your syllabus that evaluations will be posted on line during the final 3 weeks of the semester, offer students some points (20 was the suggested amount) if they filled out the evaluations and turned in a screenshot or print out showing they had done so.

Our group (math and stat) refused to pay students for doing evaulations: the results show.

I use fairly trivial incentives to get students to do diagnostic pre- and post-tests for the intro courses, with reasonably good results. Offering to drop the lowest homework grade, or adding a standard in an SBG system for completing the tests gets them to do it, without significantly altering the overall grade distribution.

The way she phrased it was poor, but shouldn't excellent communication be one of the primary requisites for a teaching position? Unhindered fluency in the language should be (I think) almost as important as expertise in the subject matter. Otherwise, how can a student go to the teacher for help or guidance? How can they understand the lesson at all?

The university I attended was notorious for sponsoring research by brilliant minds--nearly all of whom had accents so thick, no one could understand a word that was said. They were teachers so their research was paid for, which was funded to maintain the university's reputation. The whole arrangement was an open secret, but it didn't serve the students in 300+ attended intro engineering courses, as the microphones and speakers further distorted an already incomprehensible session.

It wasn't bigotry that drove my compatriots towards American-taught classes; it was a need to understand the lesson and gain an education.

By Ashley Stryker (not verified) on 09 Feb 2015 #permalink

from the NY Times piece:
"..are more likely to focus on a woman’s appearance or personality and on a man’s skills and intelligence."

I am by no means saying that there isn't a problem (see below), but I'm not finding that the tool supports that statement. The charts for "sexy", "cute", and "looks like" show that those words/phrases are used more for men than women. "Hot" doesn't show an obvious difference. "Clothes" and "outfit" also don't show a clear difference. Traditionally gender-linked terms for attractiveness (e.g., "handsome" and "beautiful") break the way you'd expect.

Positive words that I can think of related to intelligence do show a clear bias in favor of men. "Genius", "smart", "intelligent", and "brilliant" are clearly more common as descriptors of men. Negative words like "dumb" and "stupid" don't show a clear difference.

Most of the words I tried that were related to personality, whether positive or negative, show up more for men than for women. (e.g., "bossy", "mean", "angry", "nice", "helpful", "sweet", and "happy"). On the other hand, "funny", "hilarious", "boring", and "dull" show up more often for men.

Sorry, by "supports that statement", I meant "the first part of that statement."

Also, although the author's phonetics do make me wonder whether it was her bias or a real accent problem, I do think that pronunciation is relevant. A truly thick accent can be a problem in a job where a large part of the job is oral transmission of information.

In my present life, I am not strongly bound by student evaluations, but I am bound by an evaluation system that is a 'data based' and 'evidence guided' formal system. Unfortunately, the 'data' is essentially random numbers. Though the means and medians between the evaluators are nearly dead on over all evaluations, which the administrators that handle the data proudly note, there is poor correlation amongst evaluators with regard to the person being evaluated. This mirrors the system when student evaluations were heavily weighted.

My last cycle, I had, on the 1-4 scale, which the admins don't understand is a three unit scale, a mean of 3.0. On the evaluations where I had multiple evaluators examining the same criteria, the reviews varied from 2 to 4. Uniformly. A 2.5 minimum is needed for contract, to insure a 'uniformly high standard'.

I will reiterate that, when looking at a process where the long term outcomes are relevant, the only evaluative system that can be relevant is one that looks at long term outcomes. The end of the term is not long term.