Brignell, again

Posted by Tim Lambert on February 11, 2005
(19)
More »

John Brignell has an odd response (scroll down to “Hit Parade”) to some of my criticism. He doesn’t link, or dare to even mention my name, so it’s probably rather mystifying to his readers what he is responding to. Brignell goes on the Michael Fumento road, boasting about how the 2,488 hits he got on Monday vastly exceeds the 10 hits he got from me. Trouble is, he got those hits from a link in a comment in a two-day old post, so it’s hardly a meaningful comparison. For what it’s worth, his web counter shows 230k visits in five years, which is less than what I have in two years.

Earlier I wrote:

John Brignell dismisses the [Lancet] study just because:
A relative risk of 1.5 is not acceptable as significant.

Actually the increased risk was statistically significant. You won’t find support for Brignell’s claim in any conventional statistical text or paper. To support his claim he cites a book called Sorry, wrong number!. Trouble is, that book was written by. … John Brignell. Not only that, it was drafted by … John Brignell. Brignell is a crank who dismisses the entire field of modern epidemiology as some sort of plot by scientists to scare people.

Brignell’s response is:

Among the charges in the web log were that the author is not an epidemiologist, so not qualified to comment on epidemiology, and that he is innumerate for suggesting the relative risks of 1.5 are unacceptable for observational studies. The first is like saying you have not committed mass murder therefore you are not entitled to write about crime. Critics of observational studies have included great scientifically inclined epidemiologists, such as Alvan R Feinstein, Sterling Professor of Medicine and Epidemiology at Yale. The also great R A Fisher would have no truck with them at all. The second accusation is typically hyperbolic. An innumerate person would not even be able to begin discussing a concept such as risk ratio. There is a substantial body of opinion outside mainstream epidemiology that is critical of such lax statistical standards. Correspondence to Number Watch confirms that many professional statisticians are appalled by what is going on. Besides which, the proof of the pudding is in the eating. The book The Epidemiologists begins with some examples of the many completely contradictory headlines generated by popular epidemiological studies.

Actually I didn’t say that Brignell shouldn’t be commenting on epidemiology because he wasn’t an epidemiologist, but that the only support he offers for his 1.5 claim is his own opinion. He even admits that his view is outside mainstream epidemiology and still has not offered any cite or argument to support his claim. I guess Fisher might well be on Brignell’s side, since Fisher rejected the idea that smoking causes lung cancer, but very few deny this any more. And Brignell’s 1.5 risk ratio principle is innumerate. According to his principle, for example, the observed ratio of male to female births of 1.03 is not significant and we can’t conclude that male births are more likely.

(19)
More »

Comments

  1. #1 winston
    February 12, 2005

    If the Brignells (pere et fils or terrible twins, let’s suppose) weren’t quite so smug and presumptuous and were a little less self-impressed with their ability to add and subtract, they’d be the better for it. Numberwatch is not a bad attempt at imparting knowledge and respect for numeracy to the world. I mean, there are worse out there! It’s only wrong in parts, like its mistaken greenhouse effect page, that cites only the risibly ill-conceived dumbsite as reference. And peroxisome isn’t completely clueless, just wrongheaded (missing the woods for the trees) too often and modest never.

    Pa Brignell’s page on the greenhouse effect was written only two years ago yet today’s atmosphere contains 15% more CO2 than the number Brignell gives there for it. An atmosphere changing that fast would give a wiser man some pause before his next silly sermon. So the page that for the most part reads well is by the second last paragraph falling apart (there’s a gratuitous reference to a “three body problem”, suggesting that he’s heard the term used somewhere once and liked the scientific sound of it, but doesn’t know what it actually historically refers to) and the last sentence is simply stupid and bumptious: “There are other potential greenhouse gases, such as carbon dioxide and methane, but their atmospheric concentrations are so low that they may be ignored (CO2 at 0.033% and CH4 at 0.0002%”. Really? Idiot.

    Anyway that’s my take on the Brignell brothers, they’re nothing special – but don’t tell them that! Oh, Numberwatch would read more like an honest effort if it did better references, not only to scientific sites that actually understand the subject they’re allegedly explaining to folks but also links to other things they’re talking about, like your site Tim. Was this a good or bad review?

  2. #2 Jason Stokes
    February 12, 2005

    Is Brignell /really/ unaware of the distinction between a measure of effect size such as relative risk and a measure of significance? I can’t imagine anyone with even a rudimentary understanding of statistical inference making this error.

    Surely he’s playing dumb. Bamboozle the public with numbers, all that.

  3. #3 not John Brignell
    February 16, 2005

    Tim 1:

    Actually I didn’t say that Brignell shouldn’t be commenting on epidemiology because he wasn’t an epidemiologist,

    Tim 2:

    Brignell is a crank because <snip> he is not a epidemiologist.

    Brignell:

    A relative risk of 1.5 is not acceptable as significant.

    Tim:

    Actually the increased risk was statistically significant.

    Is that called moving the goal posts ?

    Tim:

    Suppose we had perfect records of every death in Iraq and there were 200,000 in the year before the invasion, and 300,000 in the year after. Then the relative risk would be 1.5 and Brignell would dismiss the increase as not significant even though in this case we have absolutely certainty that there were 100,000 extra deaths.

    that’s really good tim. But how about owning up to the fact that we don’t know the variability between years in your example, and you cannot interpret it like you do ?

    JB

  4. #4 Tim Lambert
    February 16, 2005

    Dear per/John Brown/David Bell/M Mouse

    Nice snip job. The full quote that you creatively snipped was this:

    Brignell is a crank because 1. there is no support for his position amongst epidemiologists. 2. he is not a epidemiologist. 3. his notion that the whole field of epidemiology is a scam.

    Yes, the variability between years matters. That’s why I pointed out that the 1.5 number was statistically significant. So, given a 50% increase in deaths that was statistically significant, and assuming no measurement or sampling error is it correct to claim, as Brignell does, that the increase is not significant, just by invoking a principle espoused by no mainstream scientist?

  5. #5 not James Brown
    February 17, 2005

    >Actually I didn’t say that Brignell shouldn’t be commenting on epidemiology because he wasn’t an epidemiologist, but …

    you did call him a crank, and one of the reasons is that he is not an epidemiologist. Do you think that is misleading ?

  6. #6 maybe Jam Buster
    February 18, 2005

    assuming no measurement or sampling error is it correct to claim, as Brignell does, that the increase is not significant,

    I don’t really understand why you are so abusive about Brignell, because you are saying much the same thing as Brignell. Formal statistics isn’t allowing for sample or measurement bias, and I understand that to be JB’s point since these factors can often amount to a 2-fold effect. Many epidemiologists accept that such factors can be important, and equally, it is common ground that small effects can be significant.

    no need for abuse. JB

  7. #7 Tim Lambert
    February 18, 2005

    Dear per/David Bell/M Mouse

    Sample and measurement bias can indeed give you 2 fold effect. They can also give you a 30 fold effect. There is no magic number above which you can ignore them and below which you should ignore the result. Brignell is saying that an effect of size 1.5 is per se not significant. And his claim is nonsense.

  8. #8 not James Brown
    February 19, 2005

    hi tim

    so you are accepting you called him a crank, because he was not an epidemiologist ?

    Brignell is saying that an effect of size 1.5 is per se not significant. And his claim is nonsense.

    So tell me, Tim;

    are you saying that there were no measurement or sampling bias in the Lancet study, and there is no possibility of systematic errors in that study ?

    yours

  9. #9 Tim Lambert
    February 20, 2005

    Dear per/David Bell

    I did not say that Brignell was a crank just because he was not an epidemiologist. You manufactured that quote by creative snipping.

    I did not say that there was no possibility of bias or errors in the Lancet study. What Brignell said was this: “relative risks of 1.5 are unacceptable for observational studies”. Nothing about bias or errors there. Why is it that you keep trying to pretend that he said something else?

  10. #10 not creatively snipping...
    February 21, 2005

    I did not say that Brignell was a crank just because he was not an epidemiologist.

    but you did say he was a crank, and one of the reasons was he was not an epidemiologist.

    it is fascinating to see such hard-fought evasion about what you wrote. Compare:

    I didn’t say that Brignell shouldn’t be commenting on epidemiology because he wasn’t an epidemiologist

    I called him a crank because (inter alia) he wasn’t an epidemiologist.

    A fascinating comparison. I can only draw the conclusion that you think cranks should comment on epidemiology !

    I think I would be quite happy to characterise the Lancet study as an observational study. I also think it is obvious to anyone who knows anything about epidemiology that one of the biggest difficulties for this sort of observational study is excluding bias.
    yours
    per

  11. #11 Tim Lambert
    February 21, 2005

    David, the full version of my quote is up thread. Anyone can see how you doctored the quote.

    Brignell claimed “relative risks of 1.5 are unacceptable for observational studies”. As I wrote earlier, that implies that the 1.03 factor for male births over female births (from an observational study) is not significant. This is nonsense, but you keep defending it.

  12. #12 not creatively snipping...
    February 23, 2005

    I didn’t say that Brignell shouldn’t be commenting on epidemiology because he wasn’t an epidemiologist,<snip>

    Brignell is a crank because 1. there is no support for his position amongst epidemiologists. 2. he is not a epidemiologist. 3. his notion that the whole field of epidemiology is a scam

    So Tim, I have now provided your full quote. Does that make your first comment any less hypocritical ?

    For the record, you know point 1 is wrong, because I gave you the reference of an article in science which gives the name of several prominent epidemiologists who support Brignell’s view. Your point 3 is a mischaracterisation of Brignell’s position.

    hey. These are just facts. Feel free to ignore them.

    yours
    per

  13. #13 Tim Lambert
    February 24, 2005

    David, you did not give me a reference to prominent epidemiologists that support Brignell’s view. Even Brignell admits that his views are outside mainstream epidemiology. They do not. The article took their statements out of context. I posted a letter from epidemoiologist corrected the misleading quote. But you keep evading this point: is the 1.03 ratio of male to female births not significant because it comes from an observational study? Well?

    Brignell says

    “Relative Risk or Risk Ratio (sometimes also called Hazard Ratio or Odds Ratio, but as the meaning of odds is quite different in, for example, racing circles, this is to be avoided) is at the very heart of the dispute between epidemiology and real science.”

    That is, he is saying that epidemiology is not “real science”.

  14. #14 not creatively snipping...
    February 27, 2005

    Dear Tim

    I did give you a reference to an article in Science by Gary Taubes. It contained quotes from Agnell, Temple and Richard Doll; all prominent epidemiologists, and all querying the value of relative risks of <2.

    Your stunning denunciation of this was a letter from another epidemiologist, Trichopoulos. Naturally, that does not have any bearing on the quotes from these other prominent epidemiologists. Trichopoulos actually says that his quote was correct, but he changes his position by adding caveats.

    It seems that sometimes your presentation is a little one-sided.

    yours

    per

  15. #15 Jonathan Dursi
    February 27, 2005

    Per:

    The size of an effect and how well it is measured are two different things, and not always related. I don’t see how you can fail to understand this.

    You seem to believe that it is impossible to measure an effect of less than 50% well; this is clearly nonsense, and I don’t see how you can fail to undertand this, either.

    A difference of 3%, as Tim keeps pointing out and you keep ignoring, can be extremely well measured. Do you really not believe this? Do you really not believe that effects on human health, or in physics or chemistry experiments, cannot be measured if the effect is less than 50%?

    You up the ante in your last point to a factor of 2. If that is true, then why does anyone attempt political polls, since no effect less than a factor of 200%(!!) can be measured statistically? Do double-blind studies that find that a medicine reduces mortality of a disease by a factor of 1.97 mean that the medicine ought always be discarded, since it produces no measurable effect?

    Someone who persists in nonsensical statements, either about epidemiology or about the End being Nigh, is a crank. Brignell keeps pushing this cranky claim, making him a crank. You keep pushing this bogus belief about statistics, making you, too, a crank. At least to within a factor of 2.

  16. #16 Tim Lambert
    February 27, 2005

    Dear David,
    Trichopoulos did not change his position. Taubes left out Trichopoulos’ caveats when he quoted him. That is why Trichopoulos wrote his letter to correct the misleading article that Taubes wrote with out-of-context quotes from epidemiologists.

    Hey, why not apply Brignell’s principle to the global warming debate? It tells us that any rise in temperatures of less than 300K is not significant.

  17. #17 not creatively snipping...
    February 28, 2005

    Dear Jonathon

    I completely agree that it is possible to measure an effect of less than 50% well. The point was that in the Lancet study, that there is the potential of systematic bias and inaccurate data gathering. The potential of these two confounders could overwhelm the borderline “statistical significance” of the study. Note that they had to exclude part of their data to obtain this “significance”; a practice which is a tad suspect.

    I don’t up the ante; three prominent epidemiologists made these statements, and I cited them.

    for the record, the quote that Tim provided about trichopoulos does not support the statements Tim makes. Trichopoulos allegedly writes that the quotes were correct. Tim hasn’t provided any of that letter that says caveats were left out when he quoted him.

    yet another case where there is a disconnect between fact and fantasy ?

    per

  18. #18 Tim Lambert
    March 2, 2005

    David claims that Trichopolous was changing his position when he wrote:

    Taubes writes that I have expressed the view that only a fourfold risk should be taken seriously. This is correct, but only when the finding stands in a biological vacuum or has little or no biomedical credibility. we all take seriously small relative risks when there is a credible hypothesis in the backgeound. Nobody disputes that the prevalence of boys at birth is higher than of girls (an excess of 3%), that men have 30% higher rate of death compared to women of the same age, or that fatality in a car accident is higher when the car is smaller.

    But in the original Taubes article he is quoted as saying:

    [Most epidemiologists] tend to avois causal inferences on the basis of isolated studies or even groups oif studies in the absence of compelling biomedical evidence.

    Also compare the out of context quote from Angell that David provided:

    “As a general rule of thumb, we are looking for a relative risk of 3 or more before accepting a paper for publication.” – Marcia Angell, editor of the New England Journal of Medicine”

    with the full quote in the Taubes piece

    “As a general rule of thumb, we are looking for a relative risk of 3 or more [before accepting a paper for publication], particularly if it is biologically implausible or if it’s a brand new finding.”

  19. #19 not creatively snipping...
    March 3, 2005

    Dear Tim

    I am absolutely delighted that these quotes make my case.

    thanks

    per