I just finished grading my final exams--see here for the problems and the solutions--and it got me thinking about a few things.
#1 is that I really really really should be writing the exams before the course begins. Here's the plan (as it should be):
- Write the exam
- Write a practice exam
- Give the students the practice exam on day 1, so they know what they're expected to be able to do, once the semester is over.
- If necessary, write two practice exams so that you have more flexibility in what should be on the final.
The students didn't do so well on my exam, and I totally blame myself, that they didn't have a sense of what to expect. I'd given them weekly homework, but these were a bit different than the exam questions.
My other thought on exams is that I like to follow the principles of psychometrics and have many short questions testing different concepts, rather than a few long, multipart essay questions. When a question has several parts, the scores on these parts will be positively correlated, thus increasing the variance of the total.
More generally, I think there's a tradeoff in effort. Multi-part essay questions are easier to write but harder to grade. We tend to find ourselves in a hurry when it's time to write an exam, but we end up increasing our total workload by writing these essay questions. Better, I think, to put in the effort early to write short-answer questions that are easier to grade and, I believe, provide a better evaluation of what the students can do. (Not that I've evaluated that last claim; it's my impression based on personal experience and my casual reading of the education research literature. I hope to do more systematic work in this area in the future.)
Comments
I have taught large sections with multiple guess tests. I would make up questions covering the lecture before giving the lecture. After the lecture, I would get notes from my student assistant, so I would know what I actually lectured about, then revise the questions as needed, and incorporate them into the test file. I found it instructive to record one of my lectures and listen to it after a day or two had passed. I had forgotten a great deal of what I had lectured about.
Posted by: Jim Thomerson | February 11, 2010 8:42 PM
most of principles of psychometrics that I know try to achieve higher internal consistency among individual items (so high correlation between items), since these questions are designed to measure a single concept, which is different from an exam.
and one way to increase cronbach's alpha is to increase the number of questions.
Posted by: wei | February 23, 2010 11:15 AM