The silliest graph I’ve ever seen presented in public looked something like this:
It was an after-dinner talk at a DAMOP meeting a few years back, and the speaker was somebody associated with the Hubble Space Telescope. I don’t recall what was being plotted, but he talked for a while about ho proud they were of this data, and how well it fit the theory, and then he put up this plot. The blueish circle is the data point, and the dotted line is a theoretical fit to the data.
The physicists in the audience all guffawed. He asked “What’s so funny?” and somebody near the front asked “What’s the chi-squared on that fit?” He blinked for a second, and said “Oh, right. Well, the error bars will get smaller once we analyze the rest of the data.”
OK, maybe you had to be there. The point is, he had no idea why we were busting up laughing. I told this story to an astronomer, who recognized what it must’ve been, and said that there’s actually a valid reason for plotting the data like that– the only point they can measure is the inflection point, or something like that. To a room full of atomic physicists, though, that looks utterly ridiculous– if you only have one data point, you can’t draw a four-parameter fit through it, and claim that it means something. It doesn’t matter how small the error bars are.
I was reminded of this yesterday by a graph in an article plugged in the header bar (reproduced below the fold):
My first reaction to that is “You have to be kidding me.” Shelley reports that the correlation represented by the line “remained significant after multivariate adjustment,” but I’m sorry, I have a really hard time taking that seriously.
This is, of course, why I would never make it as a medical or social scientist. On an intellectual level, I know that statistical techniques are very powerful, and can pull small correlations out of large data sets, but my gut reaction to a graph like that remains “What a bunch of crap.”