Cognitive Daily gets a lot of complaints about graphs, mostly from readers who say the graphs are useless without error bars. My response is that error bars are confusing to most readers. But perhaps I’m wrong about that. Last year I posted about this issue, and backed it up with a short quiz about error bars, which most of our readers failed. After another 16 months of Cognitive Daily, maybe they’ve improved. So here’s the test again.
Take a look at this graph. It represents a fictional experiment where two different groups of 50 people took a memory test. The mean scores of each group are shown, along with error bars showing standard error:
Based on this graph, can you tell if there is a significant difference (p<.05) between the scores of the two groups? Let's make this a poll (for the sake of accuracy, please respond as best you can even if you don't know what the error bars represent).
Below I’ve included a similar graph, again testing two different groups of 50 people but using a different type of error bar:
Again, based on this graph, can you tell if there is a significant difference (p<.05) between the scores of the two groups?
I’ll bet that we will still get a large number of incorrect responses to each poll, even though many of our readers are active researchers, and even though I already posted the same quiz last year.
Last year, I offered this wager. I said that fewer than 50 percent of our readers could accurately answer the poll questions without cheating or looking up the information elsewhere first. If we got more than 300 responses to each poll, and accuracy was better than 50 percent for each, then I would add error bars to every graph I produce for Cognitive Daily from there on out (as long as the researchers publish enough information for me to generate them — and as long as the error bars are statistically relevant [more on that later]). If not, then I would get to link to this post every time a commenter complained about Cognitive Daily not putting error bars in its graphs.
What’s a reasonable wager the second time around? Should I still keep it at 50 percent, or should I up the ante, since readers should have learned last time?
If you’d like to know the answers to the quiz, you’ll have to check last year’s post.