I am an inveterate and unapologetic listener of NPR. I love to feel like I am getting something useful out of being stuck in traffic and I find their reporting to generally be much better and more in-depth than that of the print media. However, this week, All Things Considered's reporting has really irritated me. Not just because they almost exclusively focus on the horse-race aspects of the presidential race, but because they've been doing a pretty poor job of reporting their statistics in a correct and understandable way.
First the confusing example, from yesterday's story about McCain and Obama rolling out negative ads:
According to the Advertising Project, one out of three McCain ads has been negative, criticizing Obama. Nine out of 10 Obama ads have been positive, stressing his own background and ideas.
Let's show that statement as a picture:
But you are listening to the radio, driving through traffic. Can you manage to quickly mentally convert those to percentages and reverse the phrasing so that the negatives or positives are always up front. Can you come up with this mental image? Without missing a turn? With a toddler in the backseat?
Here's the correct mental image:
Let's rewrite NPR's statement too, while we are at it: "According to the Advertising Project, one 33% of McCain ad have been negative, criticizing Obama. 10% of Obama ads have been negative. The rest are positive, stressing his own background and ideas."
After the jump, NPR conflates correlation with causation.
This gem came from Tuesday's All Things Considered in a story about the "don't ask, don't tell" policy on gays and lesbians in the military. Here's the substance of the story:
Illinois Sen. Barack Obama favors letting the estimated 65,000 gay service members serve openly rather than keep their sexual orientation a secret -- as they are required to do under "don't ask, don't tell." Arizona Sen. John McCain, who voted for the rule in 1993, has said the policy works and should be left in place.
And here's what they closed with:
The two candidates' stances on the issue are being watched particularly closely by at least one group of voters. A Harris poll this month found that among gay, lesbian, bisexual and transgendered Americans, 60 percent said they supported Obama compared with 14 percent who favored McCain.
Right, because GLBT voters couldn't possibly be basing their support on other issues - like, say, the war, the economy, or the environment. Anyone other than straight, white males must be a single issue voter, easily identified by their ethnicity, gender, and sexual orientation.
I expect better of you, NPR.
- Log in to post comments
Damn straight. As it were.
I'm a big NPR listener, but we absolutely must think critically ALL OF THE TIME. I've heard NPR do stuff that was seriously woo. And I've heard them editorialize under the guise of reporting. And I've heard them proudly declaiming to the masses while being clearly misinformed about their topic.
I no more wish to associate with uncritical NPR listeners than I do with listeners of Bill O'Reilly or Rush Limbaugh.
Always take everything with a grain of salt...
Great minds and all that - I heard the exact same thing and had the same thought. This wasn't a quote or anything so I was thinking - is the reporter that math deficient? Did they not know how to translate it into percentages? I mean that is kind of what percentages were invented to do ...
Here's the thing:
Statistics exist because humans are naturally sort of bad about thinking about numbers. But even with statistics and mathematics we're still pretty bad at it.
The McCain/Obama ad thing is a great example of the subtle bias in a lot of media reports. In fact, I'm guessing that whoever wrote the copy did it subconsciously without even realizing it.
Orac - I think it is just confirmation bias - people in the media have it just like everybody else. Knowledge of numbers is supposed to be a remedy for it, but as I observed above - most of us are still pretty bad at numbers.
I'm not 100% sure about this, but I vaguely remember reading somewhere that it is better to use statements like "one in three" of "almost one in four" instead of "32%" and "23%", because percentages are more abstract and a lot of people find it hard to make the mental images like the ones in the post from them.
Martijn: I suspect that you are correct. But if you are going to make a comparison of two numbers in that way, it seems like it would be really helpful to do so using the same denominator. 1 in 10 and just over 3 in 10.
I think it is a case of a little math knowledge being a dangerous thing. Maybe someone was docked too many points in a high school math class for not reducing their fractions.
I think the most significant problem was reversing the phrasing when going from McCain (one in three negative) to Obama (nine in ten positive). It would be too easy for someone not listening closely (like a driver in traffic) to hear the parallel phrase "nine in ten negative". It would have been better and just as easy to say one in three negative and one in ten negative. Surely even the most innumerate could figure that out.
Sciencewoman,
You are assuming that both camps are airing similar number of ads. If Obama is putting three times as many commercials on the radio, then the first figure is correct. If Obama is putting five times as many ads on, then a distracted listener is more likely to hear a negative Obama ad, compared to a negative McCain ad.
My favorite NPR-ism was a statement that each year 80% of men in Seattle aged 20-35 die of AIDS.
I told Spouse, who was 28 at the time "Man! You are NEVER going to Seattle!"
I don't think journalists typically chose their field because of their love of math.
I absolutely despise election news coverage, no matter what the source. I would honestly rather be ignorant of the nitty-gritty details than try to navigate all of the manipulated and misleading information out there. Media coverage in this country makes me want to run screaming all the way to Canada.
It might be relevant to point out that radio and television channels are going to air your ad based on what you pay: you get to pay to play an ad a certain number of times during certain time slots for varying costs (on TV, commercials for some shows cost much more than commercials aired during other shows, based on viewership and targeting and market value). Radio, it's mostly about time, but on both, you pay for the number of times your ad gets played. What we need to know is not how many different ads they have and how many are negative (which is somewhat relevant based on how many times you see the SAME positive or negative ad) but how much AIR TIME negative Obama or McCain ads are getting - and that data exists. That, I feel, is a point we could all look at and draw similar conclusions about.
What I want to know is, who are these 14% of GLBT people who are supporting McCain, and what can they possibly be thinking? Are they just insane?
Sorry for that digression. Nice dissection of the NPR reporting, SW.
it is interesting how many think the NPR report was biased towards McCain. I would say that the report was biased toward Obama. Here is the quote:
"According to the Advertising Project, one out of three McCain ads has been negative, criticizing Obama. Nine out of 10 Obama ads have been positive, stressing his own background and ideas."
A non biased report would have been something like:
"According to the Advertising Project, three out of ten McCain ads has been negative, criticizing Obama, while one out of ten Obama ads has been negative, criticizing McCain."
But no, the reporter has a bias that Obama is postive and McCain is negative.