I’ve lost track of who on social media pointed me to this, but this blog post about testimony to the Michigan Legislature is a brilliant demonstration of what’s so difficult about teaching even simple subjects. Deborah Ball, the Dean of the education school at the University of Michigan gives the legislators a simple grading exercise from elementary school math. The video here is worth a watch:

(This also includes one of the greatest failed SNL references ever. It flops badly enough that the guy responsible is nearly as embarrassed as he ought to be…)

The problem she’s illustrating is one that plagues graders at all levels: while there are a finite number of ways of arriving at the correct answer, the number of ways to do even simple questions incorrectly is essentially infinite, and some of them are perversely ingenious. And one of the key tasks of an educator is to provide useful feedback, which in many cases means trying to figure out what the hell the student did to get the wrong answer they got.

This is one of the many reasons why I try to get students not to plug numbers into their calculator from the very start of a problem. If I get a problem set that works things through algebraically, I can usually spot the error fairly easily, but when students are manipulating nine-digit decimals, going from one to the next via their calculator, it’s nearly hopeless. The occasional arrogant student employing the Feynman Method (“First you write down the problem. Then you think very hard. Then you write down the answer.”) really is hopeless.

This is also a tricky problem for exam design. When I have time to make up exam questions at leisure, I try to go through some of the obvious wrong methods as well– punching angles in radians into a calculator set to degrees, using sine instead of cosine, etc.– to make sure that the wrong answers are clearly distinguishable. Sadly, most of the time, I don’t really have the time to do that, and on occasion I’ve ended up with questions where one of the common wrong paths leads to an answer that’s within 10% or so of the correct answer. That’s a grading nightmare, as it becomes really hard to sort out genuine math and physics errors from idiosyncratic rounding behavior.

I will admit that I’m not as good about this as I probably should be– I frequently punt and write “Math Error” in the margin, meaning “You’ve ended up with the wrong numerical values, and I don’t understand why.” In my current standards-based grading scheme, I’ve also added a separate standard for “Correctly performing calculations to get numerical answers,” that shows up on any question with a numerical answer. That gets me out of trying to figure out what weird combination of calculator keystrokes could lead from correct equations to wrong numbers.

(I had a student once who was off by an order of magnitude on all his calculations, which I initially guessed must be a units issue. He was adamant that he was right, though, and we spent fifteen minutes going over his calculations before I realized that when he had to enter a number like “6×10^{7}” into his calculator, rather than hitting “6 EE 7″ he was putting in “6 x 10 EE 7″ and thus actually calculating 60×10^{7}. The calculation in question involved multiplying three numbers that were in scientific notation, and dividing by the product of two other numbers in scientific notation, which is how he ended up with all his answers off by a factor of 10…)

The infinite variety of wrong answers and the need to figure out the process behind them is a constant and under-appreciated source of angst for faculty at all levels, and it’s nice to see it demonstrated so clearly in a public forum. And it’s something to keep in mind when people start ranting about what a cushy job teachers have and how anybody could do that.