Kate recently signed up for Facebook, and I was talking to her earlier about some of the options for wasting tons of time entertaining yourself with Facebook, and mentioned the ever-popular trivia quizzes and "personality tests" and the like. Of course, I had to caution her that most of the quizzes are really lame, because the people making them up don't know how to make a good quiz.
Making up good questions is a skill that takes time to master. The key elements that the people behind most Facebook quizzes are missing are good distractors-- the plausible-sounding wrong answers that lead people astray. If you want to make a test that actually tests people's knowledge, rather than their ability to parse the question carefully.
For a silly example, here's a music trivia question:
That's written to have reasonably good distractors. You've got four similar-sounding names and one wild card. Even a person who knows the correct answer will have to stop and think a bit. Someone who only knows a tiny bit about music is likely to jump for the obvious wrong answer, and someone who knows just enough to know that the name isn't obvious will more or less have to guess.
What you tend to get on Facebook trivia quizzess is more like:
This is practically an insult to the quiz-taker. Two of the answers are solo artists, not bands, and two of the bands on the list weren't recording hits in 1988. You don't have to know much of anything about pop music to get the correct answer-- you can easily eliminate all the wrong answers as being transparently dumb.
The same thing carries over to writing physics questions. For example, in an introductory mechanics class, I might very well use a question like this:
A car with mass m is moving along a test track with speed v. The driver slams on the brakes, and the car comes to a stop in a distance L. The driver re-starts the car, increases the speed to 2v, and repeats the test.
(This is still kind of an easy question, but I don't want to give away any of my really good material on the blog.)
The key thing here is that the wrong answers provided with the question are the sort of thing that students arrive at through common misconceptions. When they get the question wrong, you learn not only that they don't know the right answer, but you also learn something about how they got it wrong. You get information about the train of thought that led them to the wrong answer, which is critical information if you're trying to teach the key concepts of classical physics.
And, as a bonus, the correct choice of wrong answers makes it difficult for students to bluff their way through a quiz or test with only a tiny bit of knowledge. And that's something that trivia-quiz writers could stand to learn from.
(The correct answers to all of the questions in this post are trivially Google-able, but that would be cheating...)
- Log in to post comments
More like this
Your first quiz has "Norman Greenberg" as an option twice.
Your second question is a trick question since it could be all of the above.
But in general you're right - quizzes should capitalize on misconceptions. It would be interesting if a quiz capitalized on subtle scientific misconceptions. For example:
Why did the dinosaurs go extinct?
A) They didn't
B) Not enough information
C) The Second Law of Thermodynamics
D) Earthquakes
Anyway, thank for the interesting post.
NS
The correct answer to the physics question is "Not enough information. Because you provided no uncertainty information about the velocity of the car, we must assume that it's velocity is known perfectly. As such, we have absolutely zero knowledge about its position, and it could stop (in the same reference frame in which its velocity is specified) anywhere in the Universe. Never mind how long it takes."
Of course, if a student wrote all that, you'd probably smack them.
When I wrote my "how well do you know Rob Knop" quiz on facebook, I tried to put in lots of plausible distractors. Even my wife only got 66% on it...!
Your first quiz has "Norman Greenberg" as an option twice.
The second one should be "Herman." And will be soon.
You get the idea, though.
Your second question is a trick question since it could be all of the above.
How do you figure?
Jimi Hendrix was dead in 1988, which pretty much rules him out... John Lennon was also dead, which makes the Beatles an unlikely choice as well.
Regarding that first quiz: I don't know if I was looking at the edited version, but I am seeing "Herman Greenberg" as both the first and the fifth choices. I suspect you intended one of them to be a "Greenbaum", which would up the distraction factor even more.
notedscholar #2: You would have to have been in a complete pop music bubble to think that any of the listed choices could have recorded that song (or any song, for that matter) in 1988. Chad explained why Hendrix and the Beatles were ludicrous choices. The other two wrong answers aren't that much better: one is a solo artist who had his best work behind him by then, and the other didn't really become known until a few years later. Pretty much anybody who was born between 1960 and 1980 should know that none of the incorrect choices is remotely plausible, even if they weren't fans of that particular band (which I am not--IMO the front man of that band has one of the worst singing voices ever to hit the pop charts).
Rob #3: You're definitely overthinking the physics problem. Chad specified an "introductory mechanics" class, so he's not expecting the quiz takers to be familiar with the Heisenberg Uncertainty Principle.
I try not to underestimate even introductory mechanics students' ability to overthink things. I'd at least add something like "assuming the brakes exert the same (constant) force in both cases" to the physics question. But that's irrelevant to the question of distractors. Regarding options, I might be inclined to replace L with L/2; it's still easy to eliminate, but might at least make them think at least briefly about why it doesn't make sense for it to be less than L. Or more likely, I'd give them L/2 instead of "not enough info", because I tend to be a little wary of that sort of answer in situations that might admit the possibility of plausible arguments about velocity-dependent friction and similar. That makes the question easier, but I'm something of a softie anyway.
What's also interesting is to create a quiz where every multiple choice question has the same letter answer. I did it once by mistake, and haven't repeated it, but analyzing the results of who changed what answer because multiple choice tests "don't do that" was fun.
What's also interesting is to create a quiz where every multiple choice question has the same letter answer. I did it once by mistake, and haven't repeated it, but analyzing the results of who changed what answer because multiple choice tests "don't do that" was fun.
My father used to do that once a year with his sixth-grade classes (actually, he did it on a true-false quiz, which is even more evil), and I've done it a few times in intro physics classes. It's never tripped up many people, though, because most of the students legitimately miss enough questions that they never notice the pattern.
Facebook quizzes - any Facebook applications really - are a bit dodgy. When you add an application, by completing a quiz for example, you are giving that application access to the information in your profile. What happens then? What happens if the company making the quiz engine decides that it is now in the data mining and targetted advertising business?
Facebook has so far been pretty bad at answering these questions, so you should think twice before signing up for a load of applications.
Plus, quizzes are possibly even more annoying than zombie bites and FarmTown.
I agree with both Erics on the physics question. The first thing though my mind was the (extraneous and completely stupid) thought "But what if it's raining? Or the tires are bald? Or it's a hot dry day? Or the bearings are bad?" Leaving that "not enough information" answer in there gives way too much leeway to a student with an incorrect answer!
Seriously though...net quizzes are great to screw with people. You should always have at least one answer in a foreign language, one in gibberish, one translated into it's numerological equivalent (the answer "42" is also acceptable) and one that's almost right. This works best when you never do actually provide the correct answer.