Last week’s Casual Fridays study was inspired partially by reports we see in the mainstream media so often, proclaiming that Americans are ignorant about the rest of the world. If the rest of the world really is better than Americans at geography, then maybe they can beat us even on our home turf. Our quiz compared non-Americans’ knowledge of U.S. states to Americans’ knowledge about the rest of the world (you can see the maps and answers here).
I randomly chose ten countries and U.S. states, then created a multiple-choice quiz where a map was displayed and five plausible choices were given. (For example, the distractors for the map of Ukraine were nearby Poland, Romania, and Belarus, and similar-shaped Kazakhstan.) Non-Americans were given the U.S. State quiz, while Americans took the Countries of the World quiz. Both groups had the option of completing the other quiz when they finished. So, did non-Americans demonstrate vastly superior ability?
Ummm… not exactly. In our test of 500 individuals, Americans performed significantly better on the Nations quiz than non-Americans did on the U.S. State quiz.
But as many commenters pointed out, this isn’t really a fair test. Why should Europeans learn the internal divisions of the U.S.? A better comparison would be a direct one — how do Americans compare to the rest of the world when it comes to knowledge of national boundaries? Considering only the people who chose to take both tests, here are the results:
It does appear that non-Americans have a slight advantage, and given our large sample size, that advantage actually verges on (but does not attain) significance. Does this mean if we tested a larger sample, we’d find a significant advantage for non-Americans? I don’t think so: I believe I have found the source of that slight advantage. Take a look at these results for the single quiz question on Denmark, the only Western European country on the test:
As you can see, Europeans were nearly perfect on this question, significantly better than Americans and others. But there was no difference in scores on any other question, even the one on the European country of Ukraine. In fact, the European advantage on the single Denmark question explains the entire difference between non-American and American scores on the Countries of the World quiz. So while Europeans may enjoy an advantage identifying (some) European countries, that advantage disappears in the rest of the world.
Clearly, there are some problems with this study. First, non-Americans may have learned the country names in their native language, so they might have been at a disadvantage taking the quiz in English. Second, it’s casual. Participants weren’t randomly selected, but instead are chosen from the biased group of individuals who happen to read Cognitive Daily. In our defense, our readers probably come from similar backgrounds, so if there were underlying differences in the educational systems from nation to nation, our study should have uncovered it. But still, there’s a clear selection bias in our results: our test was only taken by people who chose to take it.
This all brings up an interesting point. Can these results reveal any differences in how people choose to take online quizzes? One thing I was wondering about is whether people who took the test early were better than those who took it later in the testing period. Take a look at this chart of the average scores for the entire test:
The first 100 participants scored significantly higher than the last 100! Could it be that people who choose to participate in online quizzes early on are those who feel they are likely to perform better?
It turns out, probably not. I analyzed the composition of those groups and found that there were 72 Americans in the first 100 participants, but only 58 in the last 100. This is probably due to the time difference — at the end of the sampling period it was Friday night in America, but Saturday morning in Europe, and daytime in other parts of the world. Since Americans did so much better on the State quiz, this pulled the early scores up in comparison. As you can see by this chart which shows only the results from the Country quiz, where everyone did about the same, there are no significant differences in scores, regardless of when the test was taken:
Nonetheless, I think you’ll agree these are interesting results, suggesting that educated, thoughtful people like CogDaily readers have a similar level of geography knowledge, no matter where they’re from.