“A Tantalizing Hint of Dark Matter?” No.

Posted by Ethan on December 18, 2009
(32)
More »

One of the reasons I write here on ScienceBlogs is because of our associations with the New York Times, a journalistic news source that I’m proud of on most days.

Today is not one of those days.

It isn’t just the Times, either, the BBC is busy botching this story, so is the Telegraph, the Daily Mail, and pretty much everyone else, except for TG Daily, which got it right.

If you go down to the bottom of Soudan Mine in Minnesota, you’ll see an usual site for a mine: a group of giant physics experiments! Why? At the bottom of mineshafts, you have up to hundreds of feet of Earth protecting you from all the particles and radiation coming from the Sun, from space, and from terrestrial sources. This way, when you’re looking for a very rare physical event, you can be confident that what you’re seeing isn’t contaminated by other, mundane processes.

Perhaps the most important experiment going on down there is the Cryogenic Dark Matter Search (CDMS), which searches for Dark Matter particles by hoping that they bounce off of their detectors (above). When a dark matter particle bounces off, you can detect the recoil of other particles.

But there’s a problem here: other things bounce off your detector, too, despite your best attempts at shielding it. You cannot stop neutrinos, you cannot stop radioactive particles inside the Earth from decaying, and you cannot stop some cosmic rays (like muons) from getting through. These all hit your detector also, and create what we call “background” events. You can calculate what your expected background is; anything you see above that is likely your signal.

So what you do is reduce your background (which is bad) as much as you can. You try to catch all the muons and block all the radioactivity, and you find a way to try and tell light, fast-moving neutrinos apart from heavy, slow-moving WIMPs. At CDMS, they got their background down so far that, over two years of looking for these dark matter particles, they expected about one background event.

They saw two. What’s the appropriate response to that, Matt Foley?

That’s right, when you expect to see one and you see two, it doesn’t tell you anything conclusive. On Monday, I went through a hypothetical situation about CDMS, and now here’s the revised version — as best as I can reconstruct it — based on CDMS’ released results.

Looking for dark matter is like playing a one-in-a-million lottery. There are two interesting ways to play it for you: you can either buy a whole bunch of different tickets for one drawing, which gives you the best chance of winning exactly once, or you can buy tickets one-at-a-time for a bunch of successive lotteries.

CDMS, based on how long they waited (two years) and what their detector size was (30 hockey-puck-sized detectors), had the equivalent of about 900,000 tickets. If you could buy 900,000 tickets and you bought them all at once, you’d have a 10% chance of not winning and a 90% chance of winning once. CDMS can’t do that; each particle at each moment is an entirely different lottery drawing. When you work out the math for that, you find that:

  • 40.66% of the time, after two years, you observe 0 events.
  • 36.59% of the time, after two years, you observe 1 event.
  • 16.47% of the time, after two years, you observe 2 events.
  • 4.95% of the time, after two years, you observe 3 events.
  • 1.11% of the time, after two years, you observe 4 events.
  • 0.22% of the time, after two years, you observe 5+ events.

That’s what you expect to see from your background alone. The fact that CDMS saw two events? That doesn’t even count as a “hint” for dark matter; you would’ve needed to have seen four event to scientifically count as a hint, and you would’ve needed about 8 events to have some very good evidence.

The scientists who released these results are good, and they said as much in their release (emphasis is mine):

We estimate that there is about a one in four chance to have seen two
backgrounds events, so we can make no claim to have discovered WIMPs.

That’s right. What we saw? It’s probably background, and there’s no evidence at all that what we saw is anything more than that.

This doesn’t mean dark matter doesn’t exist, and it doesn’t mean that dark matter isn’t made of WIMPs either. But it means that we haven’t detected it yet, either, and stating that these two events mean anything other than this is just unscientific wishful thinking.

(32)
More »

Comments

  1. #1 Daneel
    December 18, 2009

    I had read your previous posts so when I saw the news of those “detections” I’ve already knew it was bullshit. Thanks :D I love your blog!

  2. #2 Brando
    December 18, 2009

    Bah, humbug!

  3. #3 Warren Focke
    December 18, 2009

    “It’s probably background”
    Since when is 22% “probably”?

  4. #4 Treppenwitz
    December 18, 2009

    “It’s probably background”
    Since when is 22% “probably”?

    Since 5% tends to be the standard for considering results statistically significant.

  5. #5 Greg Laden
    December 18, 2009

    OK, so one of them wasn’t dark matter, but WHAT ABOUT THE OTHER ONE!!!??

    But seriously, what is your prediction for the number of dark matter hits this detector should get per unit time? Or is there no way to know that.

    I’ve been down in that mine. It is cold and dark and deep. And has a very interesting history.

  6. #6 MRW
    December 18, 2009

    “Since when is 22% “probably”?”
    You’re confusing the probability of this happening in the case of just background with the probability that this is background.

    If I flip a coin twice, there’s only a 25% chance of getting two heads by chance alone. That’s very different from saying that if I get two heads, probability of the results being chance (noise) is 25%. In fact, the probability that the results are chance is quite high and getting two heads was probably noise.

  7. #7 https://www.google.com/accounts/o8/id?id=AItOawnzYFSRFGBVc7i7Ah_p05XVo1r8TIQ4dn8
    December 18, 2009

    The New Scientist headline will be “Dark Matter Detected – Twice!!”

  8. #8 6EQUJ5
    December 18, 2009

    If instead of 2, or 1, they had seen 0, then no doubt the Kneejerk Times would ballyhoo the ‘death knell of dark matter’.

  9. #9 rni.boh
    December 19, 2009

    “It’s probably background” Since when is 22% “probably”?

    Since 5% tends to be the standard for considering results statistically significant.

    Nooooo….

    22% is the probability of seeing 2 events. To say “It’s probably background” would mean comparing the probability of seeing 2 events if it was only background with the probability if one was dark matter. But if dark matter is very unlikely to be detected, the latter probability is small, (say 2.2%, just to put a number on it). So the probability would be

    P(2 events|background)/(P(2 events|background) + P(2 events|dark matter)

    or

    0.22/(0.22 + 0.022) = 0.91

    (I’ve cut a few corners here, but the basic argument is the same when the corners are added on)

  10. #10 Lotharloo
    December 19, 2009

    A question: are they capable of detecting almost all possible dark matter events?

  11. #11 MadScientist
    December 19, 2009

    I’ll have to trust the CDMS team’s probability calculations since I haven’t got the time to look into how they did the calculations. The funny thing with rare events is, even without an interfering background the counting may sound strange. If you detected 1 event then you know you may have had 0, 1, or 2 events. If you detected 100 events then you know there’s a very good chance you had somewhere between 90 and 110 events. That’s not even considering the efficiency of your event counting mechanism yet. To make matters even worse, throw in many other rare events, some of which are constant when averaged over periods of a few hours to a few years while others may fluctuate wildly at times. The CDMS folks will have to wait an awfully long time and be extremely careful with their data collection and analysis to be able to convince anyone that they’ve found what they’re looking for. For example, how do we know they’re not spotting a variation in the neutrino or cosmic ray flux?

  12. #12 IanW
    December 19, 2009

    Isn’t there something deliciously ironic about being in the dark about dark matter?

  13. #13 bruce
    December 19, 2009

    I work with mortality statistics a lot and we have a rule of thumb that we need about 1000 claims (deaths) in a study before we start to believe the data. That’s 1000 deaths for a group of people with the same characteristics, e.g. the same age & sex. So, if we’re looking at a group of people with about a 1% chance of dying in a year we need to look at about 100,000 people for a year before we believe the results of our study.

    For this dark matter experiment I think I would want to see something on the order of 1000 detections before I started to believe the results meant anything. That would mean either carrying out the experiment for 2000 years or doing the same thing in 1000 different locations, or getting lots of extra detections above the “background” level.

    Can anyone explain why any relatively small number of detections (even 10 or 20 in the 2-year study) would tell us anything statistically significant?

  14. #14 Treppenwitz
    December 19, 2009

    22% is the probability of seeing 2 events. To say “It’s probably background” would mean comparing the probability of seeing 2 events if it was only background with the probability if one was dark matter. But if dark matter is very unlikely to be detected, the latter probability is small, (say 2.2%, just to put a number on it). So the probability would be

    P(2 events|background)/(P(2 events|background) + P(2 events|dark matter)

    Thanks for the clarification. Probability & statistics unfortunately gets little attention in my major at my school.

  15. #15 ijc
    December 20, 2009

    “Can anyone explain why any relatively small number of detections (even 10 or 20 in the 2-year study) would tell us anything statistically significant?”

    The purpose of the 1000 claims you are talking about is to determine the unknown probability distribution function for the population. This is a calibration step, which is analogous to determining the background rate for the CDMS measuring apparatus.

    If you did a sample size of 1 million and found, say, two cases of genetic deformity X, and then went to a village and immediately found 5 people with deformity X, you would instantly know that something statistically significant had occur-ed in that village.

  16. #16 bruce
    December 21, 2009

    Thanks for the explanation ijc.

  17. #17 quantos
    December 28, 2009

    I think the main reason, Mr. Siegel, that you are a blogger on ScienceBlogs is because you could never hope to make it as a media critic, as evidenced in this post.

    You condemn the New York Times and the BBC for their coverage of the CDMS team’s announcement, even though the TG Daily story you praise contradicts itself. (The Daily Mail article is indefensible, so I won’t.)

    from TG Daily:…(CDMS) experiment have produced a set of results that could indicate the discovery of dark matter…[Two paragraphs later]…So far the experiment, which started at the Soudan mine in 2003, has given no evidence for WIMPs…

    The writing of the TG Daily story is blatantly inconsistent, stating that the two recorded events both indicate the discovery of AND provide no evidence for WIMPs. It is unclear why you thought it necessary to commend this unfortunate piece of writing. Is this really what you think is good writing?

    The NYT and the BBC are not incorrect to say that the two recorded signals could be recorded from WIMPs. While it is possible that they are background noise, it is also possible that they are the actual detections.

    Neither story claim these events are definitive proof of the existence of the particles. Indeed, both the NYT and the BBC story strongly emphasize the high level of uncertainty associated with these readings.

    From the NYT’s second paragraph: But, the scientists said, there was more than a 20 percent chance that the pulses were caused by fluctuations in the background radioactivity of their cavern, so the results were tantalizing, but not definitive.

    From the BBC’s fourth paragraph: “The scientists were keen to stress that they could not confirm that what they had seen was definitely dark matter.”

    So it is unclear why you despise the NYT and the BBC articles so much even though they are factually correct.

    Lastly, a quibble, but it is unfair on your part to invent a quote from the articles. In none of the articles, does the phrase “a tantalizing hint of dark matter” appear. Quotation marks are used to indicate a direct quote, taken another source. Don’t use them if you’re inventing a quote, because it unfairly implies that appears somewhere it doesn’t. Many sources do say the word “tantalizing” (because let’s be honest, it is) but nowhere does your invented quotation appear.

  18. #18 hack
    May 10, 2010

    So what you do is reduce your background (which is bad) as much as you can. You try to catch all the muons and block all the radioactivity, and you find a way to try and tell light, fast-moving neutrinos apart from heavy, slow-moving WIMPs. At CDMS, they got their background down so far that, over two years of looking for these dark matter particles, they expected about one background event. james

  19. #19 Jim Karter
    June 20, 2010

    A century slips by when you visit this park. Visitors wear hard hats and journey down 2,341 feet via a “cage.” On the 27th level, the transportation shifts to a rail car for a ride back into the mine as you listen to the stories of the mining days.
    resume help

  20. #20 salsa
    October 3, 2010

    The article is somewhat obtuse about the longstanding moral problem at the center of organ transplantation, which is that the donors aren’t actually dead.

  21. #21 web design uk
    October 7, 2010

    his “country seems more like a panorama of madness and delusion…of devoted family men carefully seeing to it that their children will never be able to afford college or proper health care; of working-class guys in midwestern cities cheering as they deliver up a landslide for a candidate whose policies will end their way of life.”

  22. #22 r articles iRewrite youn just two clicks and 5 seconds - Professional Article Rewriter
    October 10, 2010

    The American people seem to think, not unreasonably, that more jobs would have been created without the stimulus bill than with it.

  23. #23 Scented Candles
    October 14, 2010

    The trick here is to understand the inventory restocking part. Those figures can be played with by the government and the private secto

  24. #24 Irish Culture
    October 31, 2010

    The Bush administration itself would probably have sought other ways to constrain Saddam, bring about regime change, and deal with Iraq’s horrendous human rights violations.”

  25. #25 diet breakfast
    November 10, 2010

    After a precipitous decline following the 2008 financial crisis, the art market has made a solid recovery in 2010, driven in part by a huge infusion of new collectors abroad with fortunes newly forged in oil, gas and gold.

  26. #26 jocuri cu hannah montana
    November 14, 2010

    To make matters even worse, throw in many other rare events, some of which are constant when averaged over periods of a few hours to a few years while others may fluctuate wildly at times.

  27. #27 jocuri cu printese
    November 14, 2010

    Indeed, both the NYT and the BBC story strongly emphasize the high level of uncertainty associated with these readings.While it is possible that they are background noise, it is also possible that they are the actual detections.

  28. #28 jocuri de gatit
    November 14, 2010

    That’s very different from saying that if I get two heads, probability of the results being chance.Thanks

  29. #29 hermoni
    January 7, 2011

    Yes, I looked it up, this new force in the dark sector having mass and giving rise to dark matter has it roots with Nath et al and Feldman et al. Now, in this new paper, these authors, Feldman, Liu, Nath and Peim actually get the dark matter amount correct with multiple parts to the dark matter. It looks like they actually calculated it from first principles. They also get what looks like good fits to the positron data without throwing in fudge factors (boosts) like many other papers do. This is interesting!
    Quality Backlinks

  30. #30 fUrniTURe oCala
    January 7, 2011

    So much for statistical significance- It certainly looks like all they got was some background radiation of sorts – It’s a good thing I’m not a betting man

  31. #31 freelance web developers
    February 11, 2011

    You can be confident that what you’re seeing isn’t contaminated by other, mundane processes, when you’re looking for a very rare physical event

  32. #32 HTC One S deals
    May 4, 2012

    what would they do with the Dark Matter particles if they captured it? anyway, thanks for posting this. i love it. :)