Posted by In a paper claiming that safe-storage gun laws increase crime and do not decrease accidental deaths, Lott and Whitley:
The Cummings et al., supra note 15, research provides evidence of a
23 percent drop in juvenile accidental gun deaths after the passage of
safe-storage laws. Juvenile accidental gun deaths did decline after
the passage of the law, but what Cummings et al. miss is that these
accidental deaths declined even faster in the states without these
laws. While the Cummings et al. piece examined national data, it did
not use fixed year effects, which would have allowed them to test
whether the safe-storage states were experiencing a drop relative to
the rest of the country. The simple dummy variable that they use is
only picking up whether the average juvenile accidental death rate is
lower after the passage of safe-storage laws. One potential problem
with this approach is that any secular decline in accidental gun
deaths would produce a lower average rate after the law even if the
rate of decline was not affected by the law.
But Cummings et al did control for national trends and did use fixed year effects and they did test and find that the safe storage states were experiencing a drop relative to the rest of the country. They state that they controlled for national trends multiple times — it is mentioned in the abstract, in the result section, in the comment section and in the methods:
To control for national trends over time in firearm mortality rates, all states were included in the analysis, and 15 indicator variables were used to represent each calendar year. Categories of age, sex, and race were examined as potential confounders.
This isn’t just a minor mistake — Cummings et al was the only other published paper measuring the effects of safe storage laws. The peer review for Lott and Whitley was inadequate, since the reviewers were evidently unfamiliar with the literature and did not even read the abstract of Cummings et al. Incidentally, the peer review in that issue was the subject of a lawsuit when Steve Levitt wrote an email that stated:
It was not a peer refereed edition of the Journal. For $15,000 he was able to buy an issue and put in only work that supported him. My best friend was the editor and was outraged the press let Lott do this.
Lott sued Levitt over this email, eventually settling when Levitt wrote a letter correcting the statement that the issue was not peer reviewed.
When I pointed out this mistake, Lott responded with an extraordinary claim:
We had been unable to replicate their claimed results using fixed effects and the only way we could get something similar was without fixed effects. It really shouldn’t have been that difficult for us to confirm what they found since we were used their dates for the laws. Unfortunately, Cummings, Grossman, Rivara, and Koepsell were unwilling to give us their data when we asked for it. I asked for the data from Cummings and one other coauthor. Possibly we should have made a big deal of yet more academics who refused to share their data, but we decided that the more straightforward approach would be to simply say what we found. Alternatively, we could have simply stated that we were unable to confirm their results.
Yes, it would have been better to state something that was true instead of deliberately misleading their readers that Cummings et al did not use fixed effects when they clearly stated that they did. And according to Lott’s claims in his lawsuit against Levitt, in the first sentence Lott libels Cummings.
When challenged on this point, Lott claimed:
I called up Cummings and tried to figure out what was going on. The discussion of fixed effects was drawn from the conversation that I had with him, and the investigation would have been a lot more productive if he had been willing to share his data when he was asked.
I contacted Cummings to see if he had really told Lott that they had not used fixed effects despite their paper saying that they did. He replied:
My analysis was correctly described in the paper. On page 1085, we wrote “…and 15 indicator variables were used to represent each calendar year.” As you know, unintentional deaths due to firearms declined during 1979-1994. Failure to allow for this change would exaggerate any benefit of safe storage laws.
Negative Binomial Regression Number of obs = 816
Model chi2(66) = 876.20
Prob > chi2 = 0.0000
Log Likelihood = -1562.2441956 Pseudo R2 = 0.2190
------------------------------------------------------------------------------
count | IRR Std. Err. z P>|z| [95% Conf. Interval]
---------+--------------------------------------------------------------------
_lnmean |
caplaw | .7671992 .0796342 -2.553 0.011 .6259715 .9402898
50 lines showing state dummies deleted
Iiyr_1 | .8479772 .0648767 -2.155 0.031 .7298956 .9851618 Iiyr_2 | .7984375 .0620741 -2.895 0.004 .6855904 .9298591 Iiyr_3 | .7438544 .0589091 -3.736 0.000 .6369092 .8687572 Iiyr_4 | .6461118 .0533019 -5.295 0.000 .5496505 .7595016 Iiyr_5 | .76188 .0598726 -3.461 0.001 .6531224 .8887477 Iiyr_6 | .7358223 .0583419 -3.869 0.000 .6299164 .8595339 Iiyr_7 | .6202522 .0517576 -5.724 0.000 .5266704 .7304622 Iiyr_8 | .6522139 .0535568 -5.205 0.000 .5552561 .7661023 Iiyr_9 | .7246687 .0575369 -4.056 0.000 .6202349 .8466867 Iiyr_10 | .7056072 .0562532 -4.374 0.000 .6035352 .824942 Iiyr_11 | .6139535 .0511819 -5.852 0.000 .5214054 .7229285 Iiyr_12 | .5805103 .049 -6.443 0.000 .4919958 .6849494 Iiyr_13 | .5635373 .0492037 -6.569 0.000 .4749003 .6687179 Iiyr_14 | .5285345 .0469417 -7.180 0.000 .444093 .629032 Iiyr_15 | .4775816 .0442729 -7.972 0.000 .3982349 .5727378 ---------+--------------------------------------------------------------------
Fixed effects terms for year are clearly in the model above. They show a decline in the mortality rate ratio compared with the initial study year, 1979: the rate ratio for 1980 (“Iiyr_1″) was 0.85 and this declined to 0.48 by 1994 (“Iiyr_15″).
I do not believe I had a conversation with Dr. Lott. There is no way I can prove I did not have a telephone conversation with anyone, but I think I would recall such a discussion.
There is no reason why Dr. Lott should need my data. I used mortality data and population estimates which are freely available from the National Center for Health Statistics. Anyone can do their own study and publish their findings.
Webster and Starnes also studied safe storage laws: Webster DW, Starnes M. Reexamining the association between child access prevention gun laws and unintentional shooting deaths of children. Pediatrics 2000;106:1466-9. They obtained mortality data as I did. They used the period 1979 through 1997. They modified the way in which time was allocated to a safe storage law or not, but I suspect this change did not affect the results very much. The main difference between their analysis and mine is that they had 3 more years of data. Across all states, they estimated that the incidence rate ratio for unintentional firearm-related death among children 0-14 years was 17% less (95% confidence interval 3 to 29%) when a safe storage law was present, compared with what would have been expected without such a law. This is not very different from the estimate of 23% that I reported for the years 1979-1994. The authors wrote: “Using the same data and methods used by Cummings et al, we first replicated their findings.” Apparently they had no problem reproducing the estimates in my paper. I look upon this as confirmation that there was no major error in my analysis. They included fixed effects for time: “Year dummy variables were also used to control for temporal variation…” [To be clear, Webster and Starnes did not conclude that safe storage laws were useful, except in Florida; see their paper by details.]
February 12, 2008
February 12, 2008
February 14, 2008
February 15, 2008
February 15, 2008
February 15, 2008