Good Math, Bad Math

Big Numbers and Air Travel

As you’ve surely heard by now, on christmas day, some idiot attempted to
blow up an airplane by stuffing his underwear full of explosives and then
lighting his crotch on fire. There’s been a ton of coverage of this – most of
which takes the form of people running around wetting their pants in terror.

One thing which I’ve noticed, though, is that one aspect of this whole mess
ties in to one of my personal obsessions: scale. We humans are really,
really lousy at dealing with big numbers. We just absolutely
have a piss-poor ability to really comprehend numbers, or to take what we
know, and put it together in a quantitative way.

The context in which this comes up is in many articles written by
people criticizing the pants-wetters. The point that they’re trying to
make is that airport security is actually pretty damned good. When
you consider the number of flights, and the number of people on each flight,
and then put the number of terrorists who’ve managed to get onto a plane
and try something – successful or not – the number who get through are
amazingly small. For example, here’s one of my favorite ranters,
from youaredumb.net:

And the second part of the deal is that there aren’t that many
people trying. Even if you assume that for every one attempt that makes the
news, there are ten attempts kept secret, which is a wildly unlikely ratio in
a world where everyone’s carrying six cameras and wants to be on teevee?
That’s still less that 50 attempts on tens of thousands of flights over nearly
a decade. There simply aren’t that many people trying to blow up planes.

His argument is perfectly good. He’s right that the new airport security
measures are aimed at making you think that they’re doing more to
protect you than they used to, rather than actually doing more. But
ten of thousands of flights over the last decade? Ummm…. No.

Tens of thousands seems like a lot. But it’s so far off
that it’s laughable. It’s off by more than 4 orders of magnitude. It’s
smaller than the square root of the actual number of flights.

Let’s take one busy airport: Newark Airport in NJ. Newark is in full operation
for about 14 hours per day. Based on my time watching planes take off and
land there, I’d guess that they average at least one takeoff every three
minutes. But let’s guess that I’m wrong, and double the time between
takeoffs, assuming one every 6 minutes. That’s 10 flights per hour,
which adds up to about 140 flights per day. That’s definitely way
to small – but it adds up to well over 50,000 flights per day.

And that estimate turns out to be rank garbage. Looking up official
information, there were over 225,000 takeoffs at Newark in 2007 – which is a
significant reduction over previous years. So even biasing towards
the low side, Newark airport alone had well over 2 million flights
take off in the last decade.

According to the FAA, there were about 37,000 commercial flights per day
in 2008 – for a total of over 13 million commercial flights per year.
And that’s a dramatic decrease compared to a few years earlier, before
high fuel prices drove many airlines out of business, and forced others to reduce
their schedules.

All of us who live in metropolitan areas see ton of airplanes taking off and
landing every day. But we’ve got absolutely no intuitive understanding of just
what that actually means. I constantly rant about how piss-poor we are at
understanding scale, but I was seriously surprised by the number of flights per day
around NYC. Once we get beyond numbers that we can easily count to, our
intuitions about numbers are just a total disaster. We just don’t really
understand scale; we don’t understand how big numbers add up.

And on the topic of airport security: put the numbers into context, and you’ll realize
that all of the panic over terrorism on airplanes is really amazingly overblown. The chances of being hurt by someone who got past airport security, even without things
like the full-body scanners being deployed after this latest panic, are smaller than dying
in your dentist’s office from an anaesthesia error. And how often does anyone worry about that?

Comments

  1. #1 Matt
    January 4, 2010

    Heh. I’m sitting in a dentist chair right now, waiting for my novacaine.

  2. #2 foole
    January 4, 2010

    That’s 10 flights per hour, which adds up to about 140 flights per day. That’s definitely way to small – but it adds up to well over 50,000 flights per day.

    That should be 50,000 flights per year.

  3. #3 Ec5618
    January 4, 2010

    “.. about 140 flights per day. That’s definitely way to small – but it adds up to well over 50,000 flights per day”

    I think that last word should read ‘year’.

  4. #4 DrMathochist@gmail.com
    January 4, 2010

    On an unrelated note, please join my efforts to refer to this latest clown as the “knickerbomber”. :D

  5. #5 Thad
    January 4, 2010

    What irks me about air safety statistics is the comparison with driving, using passenger-miles, and not passenger-trips. I’m not likely to drive cross country without a very good reason. It’s worth the price of an airline ticket to not do so. An I can’t drive overseas.

    Wouldn’t the comparison of death by accident and/or malice be better when caomparing the number of trips taken?

  6. #6 Crosspolytope
    January 4, 2010

    Although if his guess of only 50 attempts is correct that means they are relatively successful when they do attempt. so we should be concerned about our previous administrations lack of diplomacy which led to an increase in anti-american sentiment around the world. It might lead to more attempts.

  7. #7 MPL
    January 4, 2010

    To be fair, pants-wetting could be an effective defense against crotch fires.

    I am glad to see that the debate over aviation security seems to be slightly more rational this time—a lot of people are demanding that the government start focusing on effective, not theatrical security measures, and announcing their willingness to rationally consider accepting small risks.

    After 9/11, many of us were worried that there might be a huge number of very dangerous terrorists attacking any day. The last 8 years have suggested that instead there are a small number of mostly incompetent terrorists out there who want to hurt us. Worth defending against, yes; going crazy, no.

  8. #8 Trevor
    January 4, 2010

    Fun infographic, “True odds of airborne terror”: http://cache.gawker.com/assets/images/4/2009/12/500x_odds-of-airborne-terror2.jpg

  9. #9 Jordi
    January 4, 2010

    I was thinking the same as Crosspolytope (thereby more or less completely sidestepping the central thesis): if 1 in 11 people who *do* want to attack us actually succeed, then that *is* something to be afraid about. It’s nice that it implies there are apparently few terrorists, but I don’t really like the idea that any random idiot has a 10% chance of succeeding. That would mean that security is extremely bad.

    Of course, these numbers were made up, but still, the fact that few people are attempting attacks doesn’t necessarily mean that we shouldn’t worry (or that airport security is any good).

  10. #10 B-Con
    January 4, 2010

    Where is your source for the FAA statistics?

    The best I could find in a few minutes of searching was this:
    http://www.natca.org/mediacenter/bythenumbers.msp#1

  11. #11 Eric Lund
    January 4, 2010

    Although if his guess of only 50 attempts is correct that means they are relatively successful when they do attempt.

    I think you are misreading the post here. The hypothetical is that there have been 50-ish total attempts since 9/11, of which only about five have been made public. The key point is that these are attempts, not necessarily successes. This guy, like the shoe bomber eight years ago, constitutes an attempt, not a success–his fellow passengers subdued him, and the aircraft made a controlled landing. As MPL@7 said: worth defending against, but not worth going crazy over. To which I’ll add that the best way to deal with any would-be plane bombers is to intercept them before they get to the airport.

    My own back-of-the-envelope calculation is that US carriers account for at least 5 million commercial passenger flights per year, even at the current (reduced) schedule levels. I figure seven major airlines (Delta, American, United, Southwest, Continental, US Airways, Alaska) with an average of 2000 daily flights each (probably an underestimate). The 13 million figure Mark quotes likely includes cargo flights (FedEx et al.).

  12. #12 Mark C. Chu-Carroll
    January 4, 2010

    @9:

    It’s not the case that 10% of all people who want to commit terrorist attacks on airplanes are successful. It’s the case that 10% of the people who want to commit terrorist attacks *and* who got onto an airplane were successful.

    Remember that there’s a lot of pre-screening going on. In many cases, people do get caught long before they go to the airport. They’re not getting onto the airplanes.

    To realistically assess the success rate of air travel security, we’d have to know how many potential attackers are discouraged from attempting attacks by their knowledge of security, and how many attackers are stopped either during the planning stages of the attack, or by security before boarding the aircraft. Knowing how many motivated people give up on attack plans is impossible; and knowing how many attacks have been blocked requires information that, at present, our government doesn’t share with us.

    To me, the best argument for the success of air travel security is to look at the number of non-airplane bombings. How many times have explosives been used by terrorists outside of airplanes? There’ve been dozens of successful bomb attacks – roadside explosives, car-bombs, backpack bombs, home-made rocket-bombs – there’s a successful bomb attack somewhere in the world several times every week. And yet, there hasn’t been a single successful airplane bombing in decades. It’s obviously not because terrorists can’t get ahold of working explosives. And it’s not for lack of motivation. But our security system is pretty good. Not perfect – but pretty good. It’s good enough that it’s just too damned *hard* to get a large-enough working explosive past security, carry it onto an airplane, and detonate it without getting noticed and stopped.

  13. #13 Bryan Lambert
    January 4, 2010

    Busted.

    “Tens of thousands” was a writerly thing, not a matherly thing. Guess low, use a vague term, and it leaves the available dodge that I didn’t say how many tens of thousands there were – turns out it’s an epic shitload of tens of thousands of them.

    And as long as I come in under the real figure, it doesn’t undermine my point to be wrong. Except for the bit at the end where I say I know math.

  14. #14 Mark C. Chu-Carroll
    January 4, 2010

    @11:

    Yes, the figures that I found include cargo planes. The travellers information site for Newark airport has figures on the number of takeoffs per day, but they don’t separate cargo from passenger. I couldn’t find any source that provided a separate count of passenger planes versus cargo. Newark isn’t a huge cargo hub – my best guess would be that a large majority of the flights are passenger, not cargo – but I could easily be wrong about that.

    In any case, I think it’s reasonable to use the total number of flights, because blowing up a cargo plane by someone sneaking an explosive package on-board and detonating it would certainly count as a successful attack, and would likely take quite a few lives if it were blown up above a populated area.

  15. #15 Douglas McClean
    January 4, 2010

    For the 12-month period ending December 31st, 2007, Newark Liberty International Airport averaged 1,089 operations/day.

    An “operation” in this context is industry/FAA-speak for a takeoff OR a landing, so make sure to keep that factor of 2 in the right place.

  16. #16 Mark C. Chu-Carroll
    January 4, 2010

    @13:

    My intention really wasn’t to mock you. LIke I said, you’re one of my favorite ranters, and your basic point was sound. But one of my personal interests is in how hard it is for people to really understand big numbers, and how they add up.

    I grew up with a dad who was a serious airplane buff. Back when you could actually get to the observation decks of airports without a boarding pass, he used to take my brother and I to the airport just to watch airplanes. So I knew how many planes take off in an hour at a big airport. But I was shocked when I first heard the figures for the total number of planes per year. Even being a math geek, and knowing the basic numbers, understanding just how they add up surprises me.

    The real fundamental truth of the matter is that we’re still just monkeys who are most comfortable when we can count on our fingers; when we get past that, things rapidly lose intuitive meaning. Our brains just aren’t wired for it.

  17. #17 Bryan Lambert
    January 4, 2010

    No, it’s fine. Believe me, given my subject matter (and subjects), it’s almost unheard of for me to get rational, well-thought-out, and correct criticism. I like it.

    And I was totally being lazy there.

  18. #18 Joel
    January 4, 2010

    @4 — My personal favorite so far has been Captain Crotchfire.

  19. #19 KeithB
    January 4, 2010

    Albuquerque Airport has an observation lounge on the non-sterile side of the security barrier, and they have take-offs of fighter-jets, Ospreys and other cool things.

  20. #20 Gilbert Bernstein
    January 4, 2010

    You made a typo:

    “That’s definitely way to small – but it adds up to well over 50,000 flights per day”

    You mean per year here?

  21. #21 Kaleberg
    January 4, 2010

    You can find some statistics at:

    http://www.bts.gov/programs/airline_information/air_carrier_traffic_statistics/

    That’s for revenue flights only, not ferry flights. They say about 11,000,000 departures per year but it’s down a bit.

    Back in the 60s, a busy airport was good for an operation a minute, or 60 takeoffs and landings an hour. By the 80s it was up to three operations a minute giving pilots and controllers a leisurely 20 seconds between takeoffs and landing for 180 an hour. That means that ORD, EWR or LGA might have 2,000 departures daily or 730,000 per year. There are at least ten major airports and a power spectrum of smaller ones so traffic on the order of 10,000,000 flights a year is quite reasonable.

  22. #22 Robert Thille
    January 5, 2010

    Nate Silver gets it right:
    http://www.fivethirtyeight.com/2009/12/odds-of-airborne-terror.html

    “one terrorist incident per 16,553,385 departures”
    and
    “the odds of being on given departure which is the subject of a terrorist incident have been 1 in 10,408,947 over the past decade”

  23. #23 Chuck
    January 6, 2010

    The real threat is not from passengers, but delivery people, who are not scanned at all. Nor, are the employees, etc… If you really wanted to do something, just get a job as a delivery person. The whole passenger screening thing is a sham, a theatre, and a giant waste of money, not to mention a serious hassle.

  24. #24 trrll
    January 6, 2010

    It has always amazed me how much fear there seems to be of terrorist attacks, considering how tiny the individual risk is. I can see being pissed off, but afraid? Why?

  25. #25 Jonathan Vos Post
    January 7, 2010

    Idiocy occurs on both sides of the bad guy/good guy divide. The news story below, which made national network TV nerws in the USA today, is so stupid that I don’t know where to begin.

    Surprise, there’s a bomb in your bag!

    A Slovakian electrician unknowingly carried plastic explosives on a flight from Bratislava to Dublin after a major snafu in a Slovak airport-security test, Irish officials told the Times of London yesterday.

    To make matters worse, the man unwittingly had the explosives in his possession for three days before Irish security officials were contacted.

    Slovak Interior Minister Robert Kalinak expressed “profound regret” to the Irish government for the oversight and the delay in alerting Irish authorities. Dublin security chiefs said it was foolish for the Slovaks to hide bomb parts in the luggage of unwitting passengers under any circumstances.

    Slovak authorities placed real bomb parts in nine passengers’ bags Saturday. Eight were detected. But the electrician, returning to Ireland where he has lived for four years, boarded his flight at Poprad-Tatry Airport in Slovakia unaware that he was in possession of 3 ounces of RDX plastic explosive – enough to blow up the plane mid-air, according to the Times of London.

    The man didn’t find out he was carrying a bomb until Irish police, acting on a Slovak tip, raided his flat Tuesday. Police said they were led to believe the man might be a terrorist, until Slovak authorities told them of the security screwup.

    A major Dublin intersection was closed and neighbors were evacuated while Irish Army experts inspected the explosive. The man was released without charge after several hours’ detention.

    Irish Army spokesman, Commandant Gavin Young, said the explosive posed no threat to passengers because it was stable and wasn’t connected to other bomb parts.

    Read more: http://www.nydailynews.com/travel/2010/01/06/2010-01-06_slovakian_man_unknowingly_carries_explosives_onto_flight_to_ireland_after_botche.html#ixzz0btxHWFZw

  26. #26 tsa69
    January 8, 2010

    Only when the Obama administration is willing to profile will the public be safe. So far the prez is a politically correct moron. All the terrorist attacks in the last 10 years have been Muslim fanatics, male, under 40 yrs. of age. If our dipshit president wants to pretend that if we just try to be nice to the folks that chop off heads, they will be nice to us. We are in trouble.

  27. #27 Mark C. Chu-Carroll
    January 8, 2010

    @26:

    Bullshit.

    Profiling is just about the *dumbest* thing you can do. All that it does is inform the people who want to attack us of exactly what we’re looking for, so that they can avoid it.

    Further: there’s been *one* successful attack in the US in the last 10 years, and two failed attacks. (And one of the two failed attacks was by a white guy.) Around the world there have been hundreds of attacks – including attacks by teenagers, by women, by old men.

    And you can’t even really profile for “muslims”. You can profile for *arabs*, which is an entirely different thing. (There are non-muslim arabs, and *most* muslims aren’t arab). There are black muslims, white muslims, asian muslims, african muslims, persian muslims. You can’t identify someone as “Muslim” by looking at them. Profile for Arab-looking muslims, and all that you do is tell the al Queda to use non-arab muslims.

    And the public seems pretty safe to me. Millions of plane flights per year, and not a single successful attack since 9/11. Not a single bombing or terrorist attack by Muslims on american soil.

    On the other hand, there *have* been a few succesful terrorist attacks in America. Bombings of abortion clinics, killing of a couple of doctors, etc.

    Finally, President Obama is *escalating* the war in Afghanistan. How exactly you can interpret sending a tens of thousands of additional troops into a Muslim country in order to get the people who want to attack us as “if we just be nice, they will be nice to us”?

    Idiot.

  28. #28 rpsms
    January 13, 2010

    @#26 Don’t forget Timothy McVeigh. We should be profiling conservatiove white catholic decorated army veterans.

  29. #29 Jim Thomerson
    January 14, 2010

    Osama tried a major attack on the USA, 9/11, which was a failure. The goal was to frighten the American people and disrupt the Western global economy. Americans got mad, and the economy hiccuped. The only attacks we will see will be by individual, mostly incompetent, wackos who hate us and do not understand the big picture.

    The shoe bomber actually had considerable success. We all have to take our shoes off to fly. If we have to take off our underwear to fly, we can think of the underwear bomber as having similar success.

    Comment on TV that about two million people fly each day, while three million use the New York subway system with no screening.

  30. #30 jack
    April 1, 2010

    Only when the Obama administration is willing to profile will the public be safe. So far the prez is a politically correct moron. All the terrorist attacks in the last 10 years have been Muslim fanatics, male, under 40 yrs. of age. If our dipshit president wants to pretend that if we just try to be nice to the folks that chop off heads, they will be nice to us. We are in trouble.

  31. #31 Jonathan Vos Post
    April 2, 2010

    “All the terrorist attacks in the last 10 years have been Muslim fanatics…”

    Well, it has been 15 years since our home-grown equivalent, not of the religion you cite.

    http://en.wikipedia.org/wiki/Oklahoma_City_bombing

    The Oklahoma City bombing occurred on April 19, 1995 when American militia movement sympathizer Timothy McVeigh, with the assistance of Terry Nichols, destroyed the Alfred P. Murrah Federal Building in downtown Oklahoma City. It was the most significant act of terrorism on American soil until the September 11 attacks in 2001, claiming the lives of 186 victims and injuring more than 680. The blast destroyed or damaged 324 buildings within a sixteen–block radius, destroyed or burned 86 cars, and shattered glass in 258 nearby buildings. The bomb was estimated to have caused at least $652 million worth of damage.