Last week I created a survey that was truly humbling. The goal was to find out if time and money invested in preparing for the SAT and other standardized college admission tests is worth it. The first thing I learned from the study was that Cognitive Daily readers are incredibly smart — much smarter than me, for example. Take a look at this graph of high-school class ranking among survey respondents:
As you can see, nearly half of survey respondents were in the top 4 percent of their high school class, and over 70 percent were in the top 10 percent. Only 15 percent of respondents ranked lower than me.
Of those who gave their SAT or ACT scores (the two major college admissions tests in the U.S.), their average percentile ranking was 93, which means they scored better than 93 percent of those taking the test.
The other reason this study was humbling for me is the massive quantity of data it generated. What was I thinking? (Clearly I wasn’t thinking very hard, because as I’ve pointed out, my readers are much smarter than I am.)
But, undaunted, I’ve spent all day sorting through the data, and I have found some interesting results. The primary question we wanted to address is whether test-preparation helps. But we were interested in more than just increasing your scores — we wanted to know if that increase (if it exists) actually helps you get into a better college or university, and whether you in turn had a more satisfying career.
First things first: How does test-prepping affect your scores? Take a look at these two graphs:
There’s no relationship between scoring well on the tests and either how much time was spent preparing, or how much money was spent on preparation. This doesn’t necessarily mean that people don’t improve through preparation, though. People who initially scored lower on the tests or in practice might be more motivated to study and spend money to get better. In fact, there is a significant positive correlation between improvement on the SAT after the first time you take the test and both how much money and time you spend preparing for the test. But the correlation is small: r = .11 for our sample of 582 test-takers.
To find a larger effect, we need to take a look at how much people believe they improved:
The blue lines show the relationship between respondents’ ratings of how much their scores improved as a result of their preparation, and how much time and money they spent preparing. In both cases, the relationship between preparation and perceived improvement was significant and positive, with rs above .2.
But as prep-time and money increased, so did both stress in taking the test and pressure from parents and teachers to do well. Stress about the test was in turn correlated with lower scores, r = -.18. Not a pretty picture.
And how did all this stress and pressure pan out in the long run? Not so well. Take a look at this graph of job satisfaction compared to test scores:
There’s no relationship. There’s no relationship between job satisfaction and how much time or money you spent on test-prep, either. Spending more money and time on test-prep isn’t more likely to get you into your first-choice college, either. One factor that might be mitigating all this is the fact that more recent high-school grads have been spending more time and money on test-prep, and they’re less likely to be happy with their jobs, possibly just because they haven’t yet reached their career goals and are still in near-entry-level positions.
So what good is test-prep? It does mean one thing: The more hours you spent on test-prep when you were in high school, the less likely you are to be unemployed right now.
Why? It’s hard to say, but one guess is that high-schoolers who spent more time on test-prep are just harder workers, which is one thing that may help you hang onto a job during difficult economic times.