Answers Matter More than Questions

The smart-people blogosphere is all abuzz about questions from the French college entrance exams, with comments from Matt Yglesias, Dana Goldstein, and Kevin Drum, among others. The general tone of the commentary is summed up by Goldstein's question:

Could you ever imagine the SAT or ACT asking students to write an essay on such complex, intellectual topics?

The answer is "Sure. The answers would suck, but you could ask them."

And that's the important thing, here. What matters is not whether you ask ostentatiously intellectual questions of your students, but whether the answers they give are any good. It's very nice to ask students to write essays on the topic "Does language betray thought?," but it's really easy to imagine getting a whole slew of responses that strain to reach the level of dorm-room bull sessions.

The form of the questions may indicate, as most are supposing, that the French are really doing a better job of teaching their students to think deeply about things. Or, it may mean that they're teaching their students to traffic in pompous bullshit. There's no way to know from just the questions-- you also need to know what constitutes an acceptable answer.

The fact that the Alex Massie link above claims that the questions "Is it absurd to desire the impossible?" and "Are there questions which no science can answer?" come from the Science portion of the exam makes me vastly less impressed with the whole thing. This may well be a selection effect-- the questions are a translation of an excerpt published in a French paper, as far as I can tell-- but if their science standards are based around this sort of thing rather than, you know, asking students to answer questions about science, then I don't have a great deal of confidence that their system is all that superior.

(And then, of course, there are a whole host of issues about tracking and earlier specialization, and the organization of the educational system, which all affect what you can expect students to do. I know next to nothing about the French system, so I can't really comment, save to say that I wouldn't jump directly to the conclusion that they're objectively better based on a handful of showily Intellectual questions reported on a blog somewhere.)

More like this

There's a slightly snarky Review of Leonard Susskind's book on string theory (The Cosmic Landscape: String Theory and the Illusion of Intelligent Design) in the New York Times this week. Predictably, Peter Woit is all over it. The central issue of the book, and the review, and Woit's whole blog is…
In comments on and earlier post, I mentioned that I no longer take extraordinary measures in anticipation of students taking an exam in an earlier sitting passing on information or answers to students taking the same exam in a later sitting. Commenter Martin wondered if I wasn't being naïve:…
As I said last week, I recently wrapped up a term experimenting with "active learning" techniques in the two intro courses I was teaching. The diagnostic test results were a mixed bag-- one section showed really good improvement in their scores, the other was no better than the same class with…
Keeping the week's unofficial education theme, Kevin Drum posts about the latest "kids these days" study, namely the just-released NAEP Geography results. Kevin makes a decent point about the 12th grade questions being fairly sophisticated, but includes one comment that struck me as off base: I…

These are questions from the philosophy exam, specifically. Keep in mind that the French generally have philosophy as a high-school subject. The questions in the article you pointed to are intimidating, but I expect the students have dealt with similar matters before.

By Johan Larson (not verified) on 19 Jun 2009 #permalink

Or, it may mean that they're teaching their students to traffic in pompous bullshit.

Put me in this column.

Daniel C. Dennett, Breaking the Spell: Religion as a Natural Phenomenon, pg 405 -

John Searle once told me about a conversation he had with the late Michel Foucault: "Michel, you're so clear in conversation; why is your written work so obscure?" To which Foucault replied, "That's because, in order to be taken seriously by French philosophers, twenty-five percent of what you write has to be impenetrable nonsense." I have coined a term for this tactic, in honor of Foucaultâs candor: eumerdification....

By Pierce R. Butler (not verified) on 19 Jun 2009 #permalink

They're not that different than the english language proficiency exam questions I drew entering university. Of course, those were just supposed to be interesting enough that we could write a few pages of discussion so they could assess our writing skills.

I taught high school in Florida for a couple of years. On their standardized end of grade tests (FCAT) there was both a short and long essay section. This was a major component of the student grade, and the grades (and money) the schools received.

The result of this was that the 5 para essay question was beaten into the ground by the time students got to high school. They could write them in their sleep. Great for standardized tests, but mostly worthless for anything more than that. They were very formula driven, with the same words over and over. It took very little thought or insight. I'm not sure that the end result was really any better, except that those are the kind of questions you see on college admission/scholarship applicaitons, so maybe it was useful ;)

Brian's comment reminded me of an intentionally nonsense essay for the SAT writing exam which I stumbled across a year or two ago. I've long since lost the link (it may have even been from this blog), but I still have the essay. It managed to earn a top score despite being chock full of incoherent revisionist nonsense such as this:

In the 1930âs, American businesses were locked in a fierce economic competition with Russian merchants for fear that their communist philosophies would dominate American markets.

The "point" of the paragraph whence the quoted sentence came was that President "Franklin Delenor [sic] Roosevelt" encouraged cooperation rather than competition, allowing the American economy to pull out of the Depression, and we thereby defeated the evil Commies.

If comparable answers to the questions on the French exam are enough to get a student into the program of his choice at the Sorbonne, then the questions are not evidence of the French system's superiority. And the questions themselves sound as though the examiners might be happy with something on this level--it would be hard, under time pressure, to do much better.

By Eric Lund (not verified) on 19 Jun 2009 #permalink

Monday 15 June 2009, I took my exam at UCLA Extension in the U.S. Constitution and California Government, which was the very last of my requirements for the full time Secondary School Teaching Credential. 75 questions. One needs 80% or more right to be certified in writing as having passed. The proctor was amazed: I got only 4 wrong, the best she'd ever seen. Several were inadvertently trick questions. I know the RIGHT answer, but have to model the test-writing committee to deduce what THEY think that the right answer might be. In general, the secret of "test sophistication" for the very smart and/or very educated is to "Impedance Match" yourself to the writers of the exam, i.e. dumb yourself down optimally. I paid $80.00 for the experience of taking the 1-hour exam, which I handed in after 25 minutes, as any more time would have had me second-guessing myself.

Kurt Godel famously saw the flaw in the U.S. Constitution which would allow legally established dictatorship. He was STRONGLY advised not to mention this at the Naturalization Exam. He started to mention this, and his friend (who'd driven him there) literally put his hand over Kurt's mouth to silence him.

To the naive observer, all philosophy looks like pompous BS. I'd suspect that only most of it looks that way to a philosopher.

Question 2 for the Philosophy/Science option is a pretty standard topic in any gen-ed science class and a core issue in the philosophy of science. One could write for hours if you analyzed the results of using science to answer religious questions, versus using religion to answer science questions. If they have good graders, I suspect that is what they are looking for. If not, they probably just read the first sentence and then count the number of words you write.

A note from a certain frog-eating country. I was rather surprised to see that these exam questions were making the rounds in the Scienceblogs community. For anyone who's been to a French high school and taken the Baccalauréat exam, they are what you generally expect to get. I must specify that the final grade is a weighted average, with the weights depending on your class. In the literary classes, philosohpy has a high weight and math a low one. In the scientific classes, it's the opposite, and the social-economic classes go somewhere in between.

Now, there's a great truth in this article's title. Not long ago, while visiting a friend who is a philosophy teacher, I took a look at the Baccalauréat essays she was correcting (I know, I wasn't supposed to...). It was rather appalling. Many of them were just a collection of somi-coherent ramblings peppered with common sense remarks, not very appropriate mentions of a few philosophers, and sad attempts at pontificating. From my own high school days, I remember having a very hard time just trying to understand what I was expected to do. But after a while, I finally learned to define every term before using it, to explore all the aspects of a question, and to not confuse a philosophical problem with a simple question waiting for a yes or no answer. That's not so bad.

By Christophe Thill (not verified) on 21 Jun 2009 #permalink

It's true that the SAT and ACT don't ask such questions, but college applications that require an essay ask questions that perhaps sound a little less ostentatiously intellectual, but are likely to gather equally bad responses. I would argue that one thing that makes them less ostentatiously intellectual is a lack of a standard national program of study.

The essay questions from the common application are:

1 Evaluate a significant experience, achievement, risk you have taken, or ethical dilemma you have faced and its impact on you.

2 Discuss some issue of personal, local, national, or international concern and its importance to you.

3 Indicate a person who has had a significant influence on you, and describe that influence.

4 Describe a character in fiction, a historical figure, or a creative work (as in art, music, science, etc.) that has had an influence on you, and explain that influence.

5 A range of academic interests, personal perspectives, and life experiences adds much to the educational mix. Given your personal background, describe an experience that illustrates what you would bring to the diversity in a college community, or an encounter that demonstrated the importance of diversity to you.

6 Topic of your choice.

While, I'm sure some of these essays are wonderful, I think lots of them are likely pretty bad. Partly because the writers are 17 and may have limited life experiences to draw on. I found my own entrance eassys while cleaning out files at my parent's house and they are cringe worthy, fairly self-important, and a little arrogant. However, since I got in and my high english teacher liked them, they couldn't have been that bad compared with what over 17 year-olds were writing.