I've spent the last two days discussing the problems with value-added teacher evaluation, and I thought I would turn it over to the readers, since there has been some really good discussion. At the end, I'll revisit some statistical and methodological issues, but I want to address a good question raised by becca--and, in other posts about education, this question has been raised:
But do you really think it's easier to fix poverty than it is to get good teachers?
Two responses to that. First, it's not clear to me that teacher quality plays anything more than a minor role--and a transitory one at that (as I noted yesterday). If teacher quality had an effect on par with that of poverty, we would have detected it rather easily by now. But the other issue is one raised by Mokele:
It's not an issue of "easy", it's an issue of whether it'll work at all. IMHO, better teachers won't accomplish much, maybe raise things a few percentage points, tops. As is so often the case, the *real* solution which would lead to major improvement is also the most difficult, expensive, and time-consuming. We need to stop looking for quick fixes and band-aids, and start addressing the real issues.
At this point, people will say it's hard to 'solve' poverty. And it is. But there are programs that can ameliorate its effects. Summer reading programs that counteract the increased academic decline among poor children would help, along with other summer enrichment activities*. For that matter, we could make sure that kids have adequate nutrition year round. There are other interventions that already have had society-wide effects: the removal of lead from the environment being the most noticeable. In Tennessee, one school district is starting the school day a half-hour later, and anecdotally (it's too early to do any rigorous analysis), the students are performing better.
This isn't to say that a child with parents for whom English isn't a first language will start at the same place as a child with two very-well educated parents. But we can do much more to provide safe, healthy environments for children.
The flip side of this is that improving teachers isn't trivial. Consider this estimate of what it would take to remove 'bad teachers' from a school system:
When they ran the numbers, the answer their computer spat out had them reviewing their work looking for programming errors. The optimal rate of firing produced by the simulation simply seemed too high: Maximizing teacher performance required that 80 percent of new teachers be fired after two years' probation....
Their point isn't that we can or should fire 80 percent of new hires, but that their work should be seen as a "thought experiment" on the extreme measures that would be required to really improve American education, provided we can't figure out how to find better teachers at the get-go or develop reliable methods of improving teachers once they're in the system.
While I've called for ongoing teacher training (and it's something Massachusetts emphasizes relative to many other states), none of the educational 'reformers' have talked about how we're supposed to find all of these available teachers--or, for that matter, how we're supposed to make poorly performing ones better. Maybe there is a surplus pool of teachers waiting in the wings, but I doubt that, certainly not without some good salaries and benefits.
Finally, some methodological issues. First, John Hawks, along with Tim Bartik, argue that there's a regression to the mean problem. Bartik:
For example, he finds that the FIFTH-grade teacher you are assigned is correlated with your FOURTH-grade "value added" gains. Different teachers get assigned different types of students. Furthermore, there is a strong indication in his data of reversion to the mean in test score levels. That is, students who have high value-added gains in 4th grade tend to have lower value-added gains in 5th grade. Therefore, 5th-grade teachers who for whatever reason happen to get assigned a great many students who had high value-added gains in 4th grade will tend to have lower value-added gains. These lower value-added gains are obviously not solely due to the 5th-grade teacher.
And John Hawks:
The cases that feature in the article all look consistent with regression to the mean -- an award-winning teacher with high scoring students has a slight decline, one with very low scoring students has a marked increase. If those were selected for the story because they represent the extremes, there's no story there.
I've never seen that these statistics are actually showing anything other than the effect of sample size. They keep saying that "having the right teacher makes more difference than the right school", but a classroom is a much smaller sample than a school, so naturally there's more statistical variance among them.
If students are shuffled year-to-year, the classrooms ought to start with something nearer the mean, and the variance among classrooms ought to increase by year end, if the teachers are actually doing anything at all.
Sounds testable to me. Finally, there's a potential curriculum effect:
I'd be interested to know if the v-a methodology takes into account that California standard curriculum introduces new material to students in the odd grades (1st, 3rd, 5th, etc) and (more or less) repeats the material in the even grades, making it easier to appear to be effective.
This definitely wasn't mentioned in the article.
I've said it before, but I'll say it again: some states do very well compared to other countries, and much better than expected given their poverty rates. Copy what those states are doing, instead of trying out untested ideas.
At this point, people will say it's hard to 'solve' poverty.
Lots of things are hard:
"We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win"
"I can't pay no doctor bill,
But whitey's on the moon."
I taught high school math and science 35 years. My observations about teacher quality:
Find teachers who:
a. Know what the hell they are talking about
b. Know how to engage students so they pay attention
c. Don't let their egos get in the way
d. Hold kids accountable for reasonable expectations concerning effort, behavior and honesty
People with these qualities are hard to find, and hard to hold.
I was a typical teacher, but several years ago I was a famous and celebrated teacher. I was recognized because I had a great group of students. I got a lot of their reflected glory. They were going to be great students whether or not I was around. I tried to make sure my head didn't get too big. I also had several students who tried my patience and made me look like less than a saint.
Everyone has a friend who has skill in some area and "makes it look easy". Few people know how much work goes into making it look easy. There is not a large crowd of people in math and the sciences waiting in the wings to take over for the bad teachers. Training teachers would be a lot more cost-effective than constantly looking for new blood
I taught for many years. I don't know if I was a "good" teacher or "bad." Some of my students did well and seemed to like me and others didn't. The way I see it is that if a student is interested in the subject matter he or she will study and master it regardless of what the teacher says or does. If the student isn't interested then (s)he won't study and master the material regardless of what the teacher says or does. Some might say that the job of the teacher is to generate interest in the subject matter for the student. I would contend that such interest is intrinsic; that the student either generates it him or herself, or doesn't. No amount of compensation could get me back into the classroom under the current political environment.