Blogger SAT Challenge update (the question revealed!)

The data-collection phase of the SAT Challenge is complete. By any measure, this was the most successful Casual Friday ever. We maxed out the generous 500 responses I allotted for the challenge, the most ever responses to a Casual Friday study -- despite the fact that participants were warned the task would take up to 21 minutes.

The survey required participants to enter at least their name before moving on to answer the essay question. The most popular name was "asdf," but no one claiming the name asdf actually wrote an essay. Clearly plenty of participants only "participated" in order to see the question (you'll see it soon enough -- I'll give it below the fold). So, of our 500, how many wrote essays? Just 155. Of those, 20 opted out after completing their response. Of those remaining, just 109 finished in under 21 minutes.

So 109 responses will be sent to our volunteer graders. Each essay will be graded twice according to the official SAT guidelines. After grading is complete -- hopefully a week from Friday -- we'll present a preliminary analysis and make the responses public. Then everyone will have a chance to rate the essays themselves. If you weren't among the 500 people who at least looked at the essay question, I'll present it below.

The essay gives you an opportunity to show how effectively you can develop and express ideas. You should, therefore, take care to develop your point of view, present your ideas logically and clearly, and use language precisely.

Important Reminders:

* Since this is an online version of the test, you will get 20 minutes instead of the usual 25
* An off-topic essay will receive a score of zero.

Directions: Think carefully about the issue presented in the following excerpt and the assignment below.

'I have learned that success is to be measured not so much by the position that one has reached in life as by the obstacles which he has overcome while trying to succeed.'

-- Booker T. Washington

Assignment: What is your opinion on the idea that struggle is a more important measure of success than accomplishment? Plan and write an essay in which you develop your point of view on this issue. Support your position with reasoning and examples taken from your reading, studies, experience, or observations.

What do you think of the question? If you took the test, do you want to say anything about the process of responding? Now's your chance -- but if you saved your response, please don't reveal it now; we don't want to bias the graders.

More like this

So, the Blogger SAT Challenge has officially run its course, and Dave has posted the question to Cognitive Daily. I'll reproduce it below the fold, and make some general comments. What were the results like? We had 500 people at least look at the survey question, and Dave gives the breakdown: The…
As of yesterday, readers had made an astonishing 3,878 individual ratings of the essays in the Blogger SAT Challenge. The average rating was 2.76, compared to 2.9 from the expert judges. Averaging the most popular rating for each essay comes up with an even lower number, 2.51. Anyone who thought…
Two weeks ago, after reading the New York Times Article which judged the best high school writers harshly, Chad Orzel came up with an idea that was so good it just had to be tried: Somebody ought to get a bunch of bloggers together, and give them the writing SAT under timed conditions, and see what…
Visit the Official Blogger SAT Challenge site. After the grading was finished, a few of our volunteer graders made general comments about the essays they read. One thing that really jumped out at me about this was the way that the problems they described sounded like exactly the sort of thing you…

The question is ok. It sounds very typical of SAT questions. I did not participate in the challenge, but I think if I had I wouldn't have minded writing the essay for this particular topic.

What I'm wondering is why the SAT doesn't move to make its essay like the GRE "analysis of an argument" task. I feel like that particular type of essay shows more about logical thinking skills and might even level the playing field for demonstrating writing skills. A student's ability to perform well on an opinion-based essay can depend on how much background knowledge or experience he/she has with the topic (some are pretty history-heavy, for example), so certain individuals might end up getting lucky. The "analysis of an argument" type of question has everything you need to discuss right there in front of you. Or perhaps I'm just showing my own bias towards the "analysis of an argument" essay.

I agree with Katherine. The question really just tests how thoroughly you can BS and guess what the questioner wants. There are all manner of ways to answer that question, and I think it is a better test of how well a high school student is going to do in college to give them something to read, understand, and then analyze. That's the first step before setting out your own opinions, anyway.

If anything, that question is more interesting than the SAT questions I saw (I took the SAT II writing test only two years ago). But for a student who has been writing 5-paragraph essays for three or four years, most of them in under a half an hour, an open-ended question like that is a piece of cake.

I didn't participate, but I should have. That's summer laziness for you. I can't wait for the results though.

I entered, but I didn't fully realize that I was supposed to time myself from when the instruction page came up so I didn't think of actually finding and looking at a watch. Consequently I had to try to rush through it without knowing the actual time (and I may well have overextended it; I just don't know).

I'm the same boat as Janne. Answered the question, but expected some sort of automatic shut-off after 20 mins. I think I went over a little bit. Now I'm bummed out that my answer won't be graded. Lousy standardized tests.

By rick green (not verified) on 21 Sep 2006 #permalink

Sorry about that, Rick and Janne, but I think the instructions were actually pretty clear on that point. I wish we had a better way to indicate how much time was left, but that would have required a much more sophisticated testing system.

btw, Janne, you made the cut, Rick, you didn't.

I thought it was a pretty good question, and I'm just sorry I didn't get a chance to answer it - I got interrupted by something more important about five minutes in and had to stop.

For those of you who are not deep into the world of SAT coaching and tutoring, I can give an idea of the strategy many students are armed with as they prepare for this test. When I work with students, I encourage them to pick four topics they know really well. Mine are usually something like: psychology, bridges, classical music, and product design. One of my student's topics were: noir film, cooking, baseball, and US history. We practice taking any SAT question and writing a five paragraph essay, using three of the four topics to build the three middle paragraphs.

As an example, I might respond to the question above by writing a paragraph introduction, one paragraph about the challenge of overcoming mental illness, a paragraph about the collapse of the Tacoma Narrows Bridge and how it influenced later bridge design, and a paragraph about a pianist with a chronic nerve problem, which caused him to change focus and become a successful composer (I can't remember if the pianist is Schubert or Schumann; the essay graders wouldn't dock me for getting it wrong). Then I'd write a paragraph conclusion, and bingo! done. It's the only way I've found to consistently answer these types of questions in 25 minutes.

I'm fairly certain I made it through in time (about 18 minutes or so by my watch) and found it to be much more difficult than expected. I find that my writing is often dictated by structure first and foremost, and that with only a 20-minute timeframe and a preset question that I was forced to just write "whatever came to mind". I'll be thoroughly surprised if I do well here, since my essay is very choppy, not-well-substantiated, and may even contain a factual error or two.

I'm thoroughly interested in seeing the final results, though, and think it's a fascinating experiment.

Other than the timing, I didn't find the actual writing difficult. The timing made it difficult because it was hard to concentrate on the subject. I have never really written to pass tests but write what I actually feel, even when I was at an age where I had to take tests. It's always fun when halfway through writing something I realize that maybe I don't think what I thought I did and end up having to start over. I should have just bsed.

My solution to the time awareness issue was to move my cursor to the bottom of my screen to make the control bar pop up with the clock in the end. It made it a bit more difficult to hold onto a train of thought when I kept checking the time, but I think I got in under the limit.

I don't think I went over on time, but I might have.

In any case, my entry ended up quite a bit shorter than I'd have liked. I only got to spend about 10 minutes actually writing it.

Hypercycloid: I don't mean this as a criticism of you personally, but that approach to test preparation puts a bad taste in my mouth.

Really the whole idea of professional test prep is problematic, but.

By Mike Bruce (not verified) on 21 Sep 2006 #permalink

"Really the whole idea of professional test prep is problematic"

Sure, it's problematic. But if you are going to have a high-stakes test, that's the inevitable result. I'm not sure the fancy-schmancy prep services really have much of an advantage over a diligent student who buys a couple $20 books and maybe a CD, though. Perhaps this might be a separate Casual Friday topic....

There are a lot of people who can't afford two $20 books and a CD, either. I mean, that's among the least of the academic advantages money can bring, but still.

Anyway! I'll be interested to see what the results of the experiment are.

By Mike Bruce (not verified) on 21 Sep 2006 #permalink

Let this 46-year-old, BA of English (1984) just say: the two classes that gave me the best-skills for coping were typing and speech-and-debate.

I used the test as an impromptu speech; ...wasn't a current event though.

...at least it wasn't math! I've learned I like to stargaze (but not count craters); I've learned I like to re-enact history (but not catalogue archeology fragments).

I've also learned that after 23 of technical writing, I don't like technical writing very much. But, it's a living.

Robert Schumann. (The pianist who ruined his fourth finger trying to strengthen it mechanically--before there was an anatomical understanding that the fourth and third fingers aren't independent--and who then became a composer.) Married Clara Wieck, also a composer and a concert pianist. Franz Schubert remained a bachelor, IIRC.

Um . . . sorry. Easily sidetracked.

Did my essay make the cut? I was the one who got the "The page cannot be displayed" and fought it for a while and then e-mailed the essay. I enjoyed doing it . . . I think I should go back to grad school and be a perennial student. Too lazy for a Real Job. *grin*

Someone wrote: "The question really just tests how thoroughly you can BS and guess what the questioner wants. There are all manner of ways to answer that question."

But that is what makes it a good question! We're not a fourth-grade teacher's aide with an answer sheet. We're looking for your original ideas and your thoughts on the subject. If you spend time trying to guess what the questioner wants, you're already on the wrong path (if they disagree but you build a good argument, you should still score well.)

I disagree that it's BS fodder. It's an open-ended question, and that's a FANTASTIC test of ready writing. I think ready writing is a good measure of whether a person is ready for college, where you learn to think (rather than just taking in information without building that mental knowledge base and then flushing the buffer after the exam.) Open-ended questions present the writer with an opportunity to show what he or she can do when given a specific topic to think about. Most people will respond with examples from their reading and with personal anecdotes. I think the essay Hypercycloid proposes would be a good answer. An impromptu speech-making capability (try Toastmasters if you're not near a debate society) is a valuable skill to have. Persuade, inform, describe . . . you know the drill.

I thought this was a good question. I kind of twisted it towards "What is courage?" because I had been thinking about that topic earlier in the day. That's the beauty of an open-ended question--if you can relate your preferred topic back to the one they ask about, you're ready to write.

Shalanna, it's an open-ended question, which is fine. But a tight time limit really means it's just a test of your ability to quickly "extemporize" (or BS, as the term is nowadays) on the theme. There really is no time for any reflection, no careful thinking through of anecdotes or finding a theme. You grab what comes to mind and run with it.

If a real essay is an extensively researched, carefully crafted and polished conference presentation, this is a few quick and disjointed reflections at the conference dinner, extemporized by an unprepared stand-in after the intended speaker suddenly got taken ill with the crab canapés.

Of course, the ability to BS - to make the impression of insight and expertize at a moments notice - should not be underestimated, nor sneered at. Many a job, many a career is depending on this ability. But don't fool yourself into thinking this is a test of something it isn't.

Mike, I totally agree with you! I have found a method that helps students to do well on this test, but I really don't believe that the SAT is necessary, fair, or useful. I tell students, pretty much, that this is the only time they will need to write like this, and that the SAT is just a hoop they have to jump through to get through the college application process. My job is to help them do well on it, then move on to more relevant and gratifying learning.

I was delighted to hear about the SAT grading problems that occurred last winter, in which thousands of tests were scored incorrectly. It was my hope that a big error like this might finally bring the SAT empire down a notch and cause schools to lose faith in the SAT score as a one-dimensional assessment of students' academic potential.

As far as test preparation, whether it works, research has shown that the SAT is totally coachable. There are definitely tricks that are helpful to know, and information about the scoring of the test that is important to use (like when to leave questions blank versus when to guess). But in my opinion, the best way of preparing for the SAT is to do lots of practice tests, correct them, and get help understanding the missed questions that aren't just careless mistakes. Tutoring and test prep services are helpful to some students because, honestly, most high school students have full lives and don't have the discipline/desire to set aside time to take practice tests outside of a structured environment.

"There are a lot of people who can't afford two $20 books and a CD, either."

I suppose, but if that's the case, they probably won't be able to afford $100 college textbooks and $2000 (at the absolute cheapest) tuition. That's most certainly a shame, a travesty, and a condemnation of our higher education system, but my point was that students don't have to choose between a $2500 tutoring program and bombing the SAT.

Hypercycloid's comments may be more to the point -- kids are so overextended, that really what you're buying with test-prep services is discipline. The information can be gained much more cheaply.

I took the test and thought my answer was decent given time constraints...but didn't research SAT grading standards at all beforehand. So I suspect that mine will be a good parallel to a high school student who isn't "coached"-- I was unaware that there is an ideal number of examples to give, that the accuracy of facts doesn't matter, etc.

And, boy was it hard to respond on the computer and not bring up 10 tabs to do some quick research. That may be one factor that separates bloggers from other kinds of writers-- we rely heavily on quick but thorough research (or at least, I do!).

By KC Culver (not verified) on 22 Sep 2006 #permalink

Last year, my son took the new SAT with the essay. He's a good writer, and has won high school story writing contests. However, these are stories that he thought about and worked on for weeks.

When he started practicing the 25 minute essay for the SAT, he got very frustrated at the time limit and the lack of time for thought. We discussed it for a while and talked about different ways to approach it.

All of a sudden, the light bulb obviously went on in his head. His eyes lit up, and he exclaimed, "It's fast food writing!"

After that, he was able to do the practice tests, and he got a good score when he took the SAT.

And he's gone back to taking weeks to write thoughtful, well-edited, and well-crafted stories.

I disagree that the 25 minute SAT is applicable to college however. In an essay exam in any college class, the students will be writing about material that they have immersed themselves in for days, if not weeks (if they're good students). College courses should teach students how to think about and analyze the material, not just write whatever comes off the top of their head.

My son just read this and added that the topics on the SAT and the practice tests were actually interesting topics: "You could have written a good essay on them if you had *any* time at all to really think about them!"

Reading the question now and attempting to just jot down some thoughts about how I would proceed, I was surprised by how emotionally loaded this kind of question could be. I think the main issue with something like this, where people are asked to draw from their "studies" and "experience" but not aksed about specific subject matter they've studied or given much more direction beyond that, is that depending on the person's background and point of view, what's expected as a valid answer could vary hugely. For some people, this is just your standard boring essay, inspring an essay decently-formed, not especially profound but reasonable ideas. For someone for whom race/class/struggle is a major emotional issue, this could be really loaded though, and I can easily see someone reading the prompt about "experience" and getting over-focused on personal experiences.

I wonder how typical test graders react to that - whether they just penalize students who are clearly emotionally invested in the answer for getting too "off-topic" or if they're from a diverse enough background and open-minded view to grade the content and intelligence behind the article without undue biases about what's inappropriate. I agree, analytical writing would probably make a lot more sense and probably do a lot to assauge the this kind of problem.

Curious, the graders don't read the essays closely enough to have anything about the writer's emotional involvement register. Essays are scored in just over a minute per essay. Scorers look to see whether the writer understood the assignment, took a position, and gave relevant examples. After that, organization and sophistication of language separate okay essays from highest scoring ones.

And, in a 25 minute essay that is at the beginning of a four-hour standardized test, the students don't have room for an emotional reaction either.

"There are a lot of people who can't afford two $20 books and a CD, either."

"I suppose, but if that's the case, they probably won't be able to afford $100 college textbooks and $2000 (at the absolute cheapest) tuition."

Actually, Dave, as one of those people who couldn't afford the test prep books, much less the courses, I'm happy to tell you that college (yes, room/board and misc. costs included) can be quite affordable. By combining outside scholarships, federal and state grants, university scholarships and work study, I managed to get a BA from an Ivy League school without ever paying for my own highly-priced textbooks. Granted, it's *hard* to get enough financial backing for higher education, but that's the difference between prepping for college and college itself - you *can* get money for the latter, but it is nearly impossible to get any help in paying for the former.

Clia,

You should be proud of your achievement -- impressive, to say the least.

I'm not saying low-income people aren't at a disadvantage in preparing for the SAT, but to suggest that the prep books are the main source of the disadvantage strikes me as a bit naive. What about checking out the books at the library?

I took it and managed to finish something half-well. I remember a similiar question when I took the GRE some years back, and I nailed it; then again, I was a bundle of nerves and sugar during that test, and relatively calm during this one. I wonder if I'd be more confident if I'd taken a formal progymnasmata-style approach. I just wrote in my usual wandering web style - which has structure, though not as easily detectable.