Lott caught cherrypicking again?

Lott has an article which purports to show that Rush Limbaugh was right when he claimed that Donovan McNabb was overrated because the media wanted to see a black quarterback do well.

Lott looked at whether media coverage was more favourable to black quarterbacks than to white quarterbacks and found that stories about black quarterbacks were slightly more likely to be positive (67% to 61%). He then did a multivariate analysis controlling for factors like the whether the quarterback’s team won and finds that after doing this, black quarterbacks are 27 percentage points more likely to get favourable coverage.

My reaction to his analysis underlines the damage that Lott has done to himself with his conduct. Even if all the data is correct and his regressions have been correctly calculated his analysis is not in the slightest bit persuasive. The reason is that his behaviour in the coding errors case suggests that he just keeps trying different models and just cherry picks the one that gives the result he wants. Would Lott want to get a result that supports Limbaugh? Well, check out this Mary Rosh posting:

You have got to download this paper. Lott has done an amazing piece here. Fits in perfectly with Rush Limbaugh’s program today.

On his show Limbaugh apparently claimed that giving women the vote was a bad idea and the same day Mary Rosh was on freerepublic.com promoting a Lott paper supporting him. It seems that John Lott is a Rush Limbaugh fan.

Seb from “Sadly, No!” has examined Lott’s data and analysis and sure enough has found evidence suggesting that Lott is up to his cherry-picking tricks again. First of all, Lott only gets his initial 67% to 61% advantage to black quarterbacks because he only considers a subset of the stories. When you look at all the data, the result was reversed, with black quarterbacks getting 53% favourable stories and white ones 57% favourable. The difference seems too small to be meaningful, but it is the opposite of what Lott reported.

Second, Seb noticed that Lott included some variables in his regression for the extent of media coverage that made no sense. Why would having more stories make those stories be more or less likely to be positive? Seb reran the regression without those variables and found that Lott’s result went away—the quarterback’s race no longer had a significant effect. Now this doesn’t prove that Lott cherry-picked his model since we don’t know if he tried doing it Seb’s way, but given Lott’s past conduct it seems quite possible.

In any event, the data Lott collected indicates that there is no clear bias in the media either in favour of or against black quarterbacks.