Comments

1. #1 davey

   March 23, 2010

   Tamino, the proprietor of the Open Mind blog, is a statistician.

2. #2 NickD

   March 23, 2010

   Wow. Good job on being flippant, dismissive, judgmental and wrong all in just two paragraphs.

   Concerning “the wrong” part: Ever heard of quantum mechanics? That’s uncertainty the way we physicists like it – fundamental and profound.

3. #3 Andrew Gelman

   March 24, 2010

   Nick: I learned about quantum mechanics many years ago on the way to getting my undergraduate degree in physics. I agree that quantum-mechanical uncertainty is fundamental and profound. However, the laws of physics (including quantum mechanics) are not uncertain in the way that models in applied statistics are uncertain. Applied statisticians are, I believe, not so inclined to be so sure of ourselves when venturing outside our areas of expertise. We just have too much experience seeing models go wrong. The blogger I linked to appears to have a physicist’s attitude that he or she can’t be wrong. But if you look carefully, the blogger was writing from a position of ignorance of the statistical literature. It’s fine to be ignorant of the statistical literature, but it’s not so fine, I believe, to not recognize your own ignorance.

4. #4 Ian Musgrave

   March 24, 2010

   If you really are discussing the linked article, “Good Bayes Gone Bad”, then you have completely mis-characterized the article.

   The article is about an example in a textbook which apparently shows a discrepancy between frequentist and Bayesian results for the interpretation of a (hypothetical) clinical trial example. The article is not about “certainty” or being right (indeed, the entire language of the post is about plumping for uncertainty), it’s about the sensitivity of Bayesian analysis to subtle assumptions.

   In particular the discussion is of whether the text book authors’ assumption that \theta_A and \theta_B are the same produces an exaggerated statistical significance when compare to a more biologically plausible assumption that \theta_A and \theta_B are different.

   Nothing in the post (or the comments looking at different ways to set the null hypothesis and its prior probability) implied that statistics was easy.

   Now, unless the statistical literature has produced a better way to chose prior probability than actually looking at the biology/chemistry/physics behind those assumptions, I cannot see how it would help in this circumstance.

   So perhaps you can enlighten us on how you would choose an appropriate null hypothesis for the example given, and estimate the prior probabilities. We biological scientists who actually do these experiments would like to know (seriously, it’s a huge topic in my discipline).

   (Unless of course you are actually mean the post prior to the Bayesian one “The Power — and Perils — of Statistics” which debunks a particularly stupid piece by climate change denialists, which means you comment is still a misrepresentation. Then again, I think Ioannidis should be whipped with a wet noodle for not titling his paper “Why most research done on dodgy, un-validated genechips is wrong”, but hey, it wouldn’t be so headline grabbing)

   PS, the textbook author is an information theoretician, not a climate scientist.

   PPS Despite the blog, I’m a biomedical researcher during the day.

5. #5 Ian Musgrave

   March 24, 2010

   Here’s a comment from Tamino (the bloke who wrote the “Good Bayes Gone Bad” post) in the “The Power — and Perils — of Statistics” thread that may have some relevance. He’s replying to two physicists.

   The two of you [the physicist posters] also seem to have this impression that physics is somehow “better” than other fields of science — as though physicists know how to define the proper use of statistics better than statisticians! That’s arrogance.

6. #6 Andrew Gelman

   March 25, 2010

   Ian:

   I may indeed have mischaracterized the blogger. And part of it may be stylistic differences; another post lower down on the page at that same blog concludes with “Are you man enough?”–which is the kind of thing you’d expect to see on a political blog, not so much on a statistics blog!

   My main reaction was that the discussion of Bayes on that blog was occurring pretty much in isolation of the statistical literature. The blogger was discussing a statistics textbook written by a physicist, and he wrote certain things that appear to betray some confusion about the basic principles of statistics; for example, “it’s likely (one could even say ‘statistically significant’) that the treatment is effective, but that doesn’t mean proved conclusively.” Looking at the blog entry more carefully, though, I realize I’m being a bit unfair, as the blogger states right at the beginning that he is responding to a reader who recommended that particular book.

   The sort of very simple example being discussed misses a lot of the point of Bayesian inference, in that it’s an unusual case in which the data model (the “likelihood”) is uncontroversial. Once you move to linear regression, logistic regression, and so forth, any statistician–Bayesian or not–is going to be making a lot of assumptions, and in this context the prior distribution is just part of the larger model.

   In general I don’t think it makes sense to assign positive probability to the even that two treatment effects are identical, but of course it’s difficult to say much more than this, given that the example is entirely hypothetical.

7. #7 Dikran Marsupial

   March 25, 2010

   I don’t think it would be fair to categorize Prof. MacKay as a “just” a physicist; he has made many worthwhile contributions to Bayesian statistics and information theory, so he is an expert on the topic.

   For my part, my objection was to tamino treating a bit of Bayesian inference as if it were a hypothesis test, and using a threshold of 95% to mean significance; when the usual interpretation of Bayes factors would suggest the evidence is weaker (MacKay’s intepretation of the Bayes factor of 99 as being “very strong” is in line with normal practice AFAICS).

   However tamino is now editing and deleting my posts without bothering to consider the content (for instance to explain why he adopted th 95% threshold rather than the normal interpretation scale). So the mixing will probably improve now that an inconvenient mode has been deleted ;o)

8. #8 Ian Musgrave

   March 26, 2010

   Andrew Writes:
   

   And part of it may be stylistic differences; another post lower down on the page at that same blog concludes with “Are you man enough?”–which is the kind of thing you’d expect to see on a political blog, not so much on a statistics blog!

   It helps to know the context. Anthony Watts has been particularly reprehensible in making fact-free claims and ignoring refutations. If you have never had to deal with climate change denialists, creationists and anti-vaxxers before let me tell you it is only a matter of time before you snap when dealing with them. Tamino was quite mild given the relentless ignoring of data practised by Watts (Tamino has just demonstrated a central claim of Watts was false, and had several people independently replicate his finding).

9. #9 Dikran Marsupial

   March 26, 2010

   I would second what Ian says above; purely electronic means of communication tends to bring out the worst in most of us, and anyone arguing against the denialists have had more than their fair share of provocation.

   Tamino has been doing an excellent job debunking some of the sceptic nonsense that has appeared in print on climate change, and he seems to have an excellent graps of frequentist time series analysis (and communicates it very well to the general public).

10. #10 Ian Musgrave

    March 26, 2010

    [sigh] Actually posted a substantial statistical comment as well. It’s somewhere in moderation because it quote a URL.