Cherry picking above and beyond the call of duty

Crime statistics for the last 12 months in NSW have been released. Most crime categories have decreased significantly.

Indecent assault, act of indecency and other sexual offences Down by 11.9%
Robbery with a weapon not a firearm Down by 19.7%
Break and enter - dwelling Down by 9.4%
Break and enter - non-dwelling Down by 17.6%
Motor vehicle theft Down by8.7%
Steal from motor vehicle Down by 13.7%
Steal from retail store Down by 17.4%
Steal from dwelling Down by 5.1%
Steal from person Down by 17.8%
Fraud Down by 12.3%
Murder No significant trend
Assault No significant trend
Sexual assault No significant trend
Robberywithout a weapon No significant trend
Robbery with a firearm No significant trend
Malicious damage toproperty No significant trend

Now if, hypothetically, you were out to show that crime had increased here because of the 1996 gun laws you would be faced with a problem. Normally with sixteen crime categories you can find one that increased and you can then run with that. But there weren't any significant increases this time. What to do, what to do?

Fortunately the crime statistics are broken down into region and subregion. If you scroll down in the report you find a table giving the crime categories for each part of Sydney. Fourteen rows and sixteen columns means that there are 224 cells in the table. The table is a sea of negative numbers. Crime is down everywhere and in every category. But wait! There is one, just one, positive number in the whole table: Robbery with a firearm increased by 34% in inner Sydney, from 123 to 165. So what do we find on Lott's blog?

With a reported 34 percent increase in armed robberies in Sydney during just the last 12 months, some have been driven to try and stop the attacks. (I don't have the numbers handy at the moment, but armed robberies have been going up dramatically for the last six or so years in Australia.)

Actually it was firearm robberies in inner Sydney that went up 34%. Armed robberies in Sydney declined by 16%. And armed robberies in Australia have also been going down. To be fair, we must give the Sun-Herald an assist on this cherry pick since they pulled out the 34% figure:

Latest Bureau of Crime Statistics figures show all major crime rates as stable or in decline except for armed robbery, which has undergone a 34 per cent spike in inner Sydney in the past 12 months.

But leaving out the part about all major crime rates being stable or declining was all Lott's work. As was the false claim that armed robberies were increasing nationwide.

More like this

Interesting. So what are those nice reductions in those crime stats being attributed to?

I guess that I just don't see the Cherry picking? Lott quotes the only numbers discussed in the article. He also provides a link to it so he wasn't hiding anything. What is the big deal? In addition, his point was that people were forced to defend themselves in risky ways without weapons. I could see that you might disagree with that last point, but it is not what you are complaining about and the rest of you discussion seems irrelevant to that point. What am I missing?

The cherry picking is that Lott picked out the increase and did not report the decreases. There has been a dramatic decrease in crime in NSW, but Lott wants you tho think that it has gone up.

I don't see any cherry picking and I have no clue what you mean by " that Lott picked out the increase and did not report the decreases." There was only one number in the article. Lott quoted just that one number. He didn't pick one number over others that would have shown something else. His point in the post was that people were being forced to defend themselves in ways that didn't make sense and that the newspaper didn't note the problems that were being caused for people. If you bothered to quote the last sentence in Lott's post, it would be clear to everyone what the point of Lott's post was. I have to say that the newspaper stories sure seem to confirm that people were reduced to protecting themselves in some pretty silly ways and I suppose that was the point that the newspaper was trying to make. Don't you think that was the point that the newspaper was trying to make? Of course, the newspaper had different advice for people than what Lott was telling peopel to do. Now, again, if you disagree with Lott's advice on how people should behave, that would be a challenge to what he said.

After your claim that armed robberies have decreased in Australia, I sent Lott a note. He sent me an excel file and some charts of what he claims happened to robbery rates in Australia after the 1996 gun control laws. In response he has also put the figures up on his website at If these numbers are accurate, it sure looks to me that both robbery and armed robbery rates are higher now than they were in 1996 and also higher than any of the years immediately prior to the law. Even if you pick 1995 to be safely before the law as a comparison, the increases in robbery rates are even a little larger. The figures are actually rather impressive. It appears as if you have been playing fast an loose with your selective cherry picking claims. In addition, it still concerns me that you selectively quoted from Lott's earlier posting on Sidney and left off the last half of his posting. You never responded to that.

Armed robberies have not been "going up dramatically for the last six or so years in Australia" as Lott claims. There was an increase from 1996 to 1998 and since then the rate has returned to the 1996 level. We only have figures for half of 2004, but it looks like the rate is continuing to go down.

Could you please explain what you mean by "not" going up dramatically. Here are some numbers from the spreadsheet that one can download from Lott's website. Please tell me if these numbers are wrong. Year & Armed Robbery Rate per 100,000 Aussies.
1993 30
1994 28.3
1995 29.1
1996 34.1
1997 48.9
1998 57.9
1999 49.9
2000 49.5
2001 57.9
2002 39.7
I checked Lott's claim and it is right that the average armed robbery rate for the six years from 1997 to 2002 is 74 % higher than in 1995. For 1996 to the average for 1997 to 2002 it is 48% higher. You could look at the pre years average versus the post years average and you would get similar changes. The numbers are flat before the 1996 law. 1998's numbers show a 99 percent rise over 1995 and a 69 % rise over 1996. Even if you pick just the low year for 2002, the increase is still 36%. It seems that I read someplace on your website that you attributed the recent drops to increased policing. If so, you might want to be careful attributing any drop in crime 8 yrs after law to the law.

We are going to have to label you Tim "the cherry picker" Lambert both for your numbers and your selective quotes.

Bob, it might be that Lott pounced on teh only figure in the article, but he left out the part about

"Latest Bureau of Crime Statistics figures show all major crime rates as stable or in decline except for armed robbery"

Now, Lott has been arguing that the gun laws following the '96 shootings have lead to ever-increasing rise in crime, especially gun-related crime, so it would seem natural that he would mention that this hasn't been the case.

By Kristjan Wager (not verified) on 25 Sep 2004 #permalink

Lott claimed "armed robberies have been going up dramatically for the last six or so years in Australia." If you study the graph on the left you will see that his claim is not true. There was an increase from 1996 to 1998, but since then the rates have decreased to roughly the same level as in 1996.
Most armed robberies are not committed with guns here so I don't think there is anyone attributing the decrease to the 1996 law.

I) Armed robberies are still not back down to where they were prior to the law. You had a big rise that started just when the law changed. Again, the drops still don't take you down to the pre-law level. Recent drops some six or seven years after the law have been attributed by others, including Tim, to increased policing. II) When I have read Lott's discussions or looked the excel file that he has made available on Australia he has taken averages which seems the opposite of cherry picking and doesn't put the whole weight on one recent year like Tim only does. Tim, if you took the average armed robbery rate after the law and compared it to the rates before the law, Wouldn't any one think that was a big (or large or whatever giant) increase in crime. III) While Tim has cited one part of Lott's 9/12 post out of context, the point of the post was that Aussies are being forced to defend themselves in not very smart ways.

Bob, you are just repeating points that I refuted already. The rate for 2003 (36) is slightly more than for 1996 (34). We only have figures for the first half of 2004, but it looks like the rate will be lower again.

Bob H. - Thanks for your comments and for getting the numbers for us, but I think you're missing Tim's point. Cherry picking is precisely that, picking cherries - that is, rooting around in a bucket of varied fruit until one finds one or two pieces that are ripe or rotten enough to sell or discredit the entire bucket apart from the majority of the fruit it contains. It makes no difference whether we're selecting only a statistic that says armed robberies in Sydney had an annual increase of 34 percent and ignoring a sea of downward trends for the same period, or one that says it went up 99 percent from 1996 to 1999 as you did. Neither does a carefully chosen average over a few years of variable rates like 1997 to 2002 because this often masks the trends that are of interest, as doing so here does. We're still picking and choosing as we please.

By contrast, a proper statistical analysis would test a particular verifiable (or falsifiable) hypothesis against a complete set of data that included all relevant variables and datasets over a statistically significant period. Lott's hypothesis is best summarized in his book title - More guns -> less crime. If this is true, we would expect to see a negative correlation between firearm sales and/or possession trends for all firearm relevant crime trends over a period long enough to be statistically significant and after all other variables had been appropriately (note the emphasis on appropriately) corrected for. If Lott is right, after the implementation of the 1996 Australian law, there should have been a downward trend in gun ownership offset by some time constant associated with the period over which the law worked out its effects on actual possession, criminal and otherwise. There should also have been an increase in crime across all categories where a gun might be relevant either offensively or defensively (which covers most if not all crimes, particularly violent ones) from the instigation of the law (1996) to the present. Both should have been accompanied by a compelling explanation of why the countless other related variables impacting these things had not been involved.

By contrast, the very data you provided from Lott shows a rising rate of armed robbery beginning in 1994, two years prior to the gun law, and increasing in frequency. Accounting for temporal sampling error, it looks as though the rate of increase actually peaked in the year the gun law went into effect and did not change appreciably for at least two more years. It has been dropping steadily since 2001 even though the gun law is still in effect. This argues strongly that both the increase and the decrease were likely related to factors other than the gun law. In fact, if it were shown that the time constant for penetration of the gun law to impact actual overall possession rates were on the order of 3 to 4 years, it would even argue for the gun law as being effective on armed robberies. Furthermore, the New South Wales Bureau of Crime Statistics and Research where Tim's figures come from also has overall trend data for several larger categories for the longer period of 1999 to the present. For the most part, these categories show constant or decreasing trends also, the only exceptions being Fraud (which is unrelated to firearm ownership), Assault, and Sexual Assault (data was not available for years previous to 1999). According to Lott, none of this should be true.

And this cannot be refuted simply by picking and choosing a few comparisons here and there at the expense of the larger picture.

In fact, most of this is moot anyway, as the Lott article Tim referenced clearly quotes him as stating that,

"With a reported 34 percent increase in armed robberies in Sydney during just the last 12 months, some have been driven to try and stop the attacks. (I don't have the numbers handy at the moment, but armed robberies have been going up dramatically for the last six or so years in Australia.)"

(My emphasis)

In other words, the last 12 months are precisely what he was examining in this statement, and it is over this period that he ignored a virtual ocean of other downward trends clearly published in the very same source his numbers came from, exactly as Tim said. Now THIS is cherry picking!

It's also false. The data actually says that firearm robberies in inner Sydney, not Sydney armed robberies, went up 34 percent. Armed robberies in Sydney declined by 16 percent. Nor is it true that armed robberies have been going up, as Lott said, over the last six years in Australia. This can be plainly seen, once again, in the very data you provided from Lott. Apart from a short spike up in 2000, is has dropped dramatically. In reality, far more rigorous work than Lott's has shown that overall crime rates here in the United States and elsewhere do not significantly correlate either positively or negatively with general gun ownership in a clear manner - a fact which does agree with the contents of your data and the larger dataset from New South Wales that Tim and Lott were referring to (see for instance, the FBI's Uniform Crime report from pretty much any recent year, as well as a number of other publications from the U.S. Dept. of Justice). What does happen though, is that gun ownership, or more specifically handgun ownership levels have a strong proxy impact on murder rates simply because apart from handguns, most violent crime is via bludgeonings or stabbings, and these are far less likely to be fatal than gunshot wounds. The real point to laws like the 1996 Australian one, is mainly that they often reduce violent deaths as opposed to general crime (note that per the figures Tim quotes, the murder rate has undergone no significant change in the last year, and according to the longer period figures I linked, it is down by 7.3 percent when tracked as monthly rates of victimization). Lott, of course, does not get into this much.

Lott needs to demonstrate that long-term regional and national Australian crime trends over a broad range of crimes where firearms are either offensively or defensively relevant negatively correlate with gun ownership, after all other relevant social, economic, and cultural variables have been accounted for, and with something far more rigorous than a few hand-waved parameterizations and numerical adjustments here and there that do little more than cluster his data. He has NOT done this to anyone's satisfaction outside of the rabid gun lobby that worships him.

Tim's argument stands... unscathed.

One other quick comment in this regard. The armed robbery increase that was observed isn't relevant here because, as I pointed out, it began before the law was implemented and as since turned around, both changes in trend not appearing to be correlated with it.

First, the law was implemented in 1996. It is hard to see anything going on with armed robberies prior to 1996. Tim has been all over the place on this. Claiming that "Armed robberies have decreased in Australia and in NSW and in Sydney. Lott pulled out that number to make it look there had been an increase." Decreased? Does anyone see a decrease after 1995? Then Tim claims "There was an increase from 1996 to 1998 and since then the rate has returned to the 1996 level." He makes it sound that it briefly went up only in 1997 and 1998 and then went down. Can Tim read his own graphs? Up until 2001 it remained just as high as in 1998. Count 1996, 1997, 1998, 1999, 2000, 20001. That seems like a pretty substantial period.
1993 30
1994 28.3
1995 29.1
1996 34.1
1997 48.9
1998 57.9
1999 49.9
2000 49.5
2001 57.9
2002 39.7
2003 36 (from Tim) Note still much higher than 1995's 29.1

Second, as I read Lott he is actually claiming that the crime rates surely didn't fall as people were predicting. Doesn't that seem beyond debate?

Bob H. - Thanks once again for the feedback and thought. But again, I refer to your own numbers. You said, "First, the law was implemented in 1996. It is hard to see anything going on with armed robberies prior to 1996." On the contrary, your own numbers for Armed Robberies per 100,000 Australian citizens are as follows,

1994 28.3

1995 29.1

1996 34.1

That is an increase of 5.8, or 20 percent of the 1994 level, in two years prior to the law being implemented. It also shows a rapidly increasing rate of rise over the same period (within sampling error limitations) - 5 crimes per year in 1995, and a maximum rate of increase of 13.8 crimes per year during the very year the law was implemented. The key point here is not the size of these increases or their duration, but that they were well under way before the law took effect. This is actually quite clear from the numbers that you were diligent enough to seek out from Lott himself and then kindly provide for us here (thank you).

As for the claim that crime rates surely didn't fall as people were predicting being beyond dispute, they actually did fall... and they rose by an equal amount. Though you are right to say that 1996 to 2001 is a significant period, so is 1998 to the present, a period of almost the same length which exactly reverses the trend of 1996 to 2001. In fact, an examination of the graphical representation of your numbers in Tim's 9/26 post shows that the if the current trend continues to the end of this year (as now seems likely), the 8 years since the law was implemented is characterized by two successive periods, one rising and one falling, which exactly mirror each other and cancel out, neither of which correlates to either the implementation or removal of the 1996 law.

Once again, let me point out that this issue is not, and never has been, about picking short-term trends here and there. It is about correlation. Do "More Guns" correlate with "Less Crime", "More Crime"... or neither? Your numbers demonstrate quite clearly that at worst, the Australian Armed robbery rate and gun ownership are uncorrelated, and at best (if the time for the law to filter down to actual ownership were to be around 3 or 4 years) that the law worked. Personally, between the two, I'm more inclined to believe the former. Either way, this pretty soundly refutes Lott's thesis.

Also, as was pointed out before, laws like the 1996 law can have a proxy effect on the murder rate in that fewer guns usually means fewer fatalities in violent crimes. This actually can be seen in the 1999 to present murder rates from the New South Wales Bureau of Crime Statistics and Research, though I wasn't able to get them all the way back to before 1996. If similar trends hold true back to before 1996, this would demonstrate the real value of the law.

Talk about cherry picking. I notice that you drop off the 1993 data, and for good reason. The armed robbery rate in 1995 was even less than that in 1993. Prior to the law being implemented in 1996. The crime rate bounced up and down within a relatively narrow range. 1996 is the first year that there is a big increase. There is then an even bigger increase in 1997. If you think that there is a big increase in 1995, notice that the gap between 1996 and 2003 is still even bigger.

Bob H - You are right. I should have included the 1993 data too. My apologies for neglecting it. But that figure, like the 1994 through 1996 figures, is prior to the implementation of the law, and like virtually every other number in the Armed Robbery table and those for other crime categories, it has no correlation with it. Until a correlation is demonstrated, virtually all talk of trends is meaningless.

Think of it this way. Suppose I were to dump a tanker of water off a pier into the bay. If I did this at low tide, 10 hours later it wopuld look very much like I'd filled the bay, or at least contributed to its rising. Now if I truly wanted to test this theory, I'd dump other tanker loads of water off that pier at numerous different times over different days, enough to cover a broad, statistically significant sample of times with respect to the tides. Then, I would carefully study not whether the tide was rising or falling, or even whether it did so after I'd dumped the water once or twice, but whether there was a clear, long-term pattern between my dumping and the rising that did not line up with any other variables that might be affecting it. We could argue from now until the next ice age about the tide rising this much between noon and 6 PM on Monday, or 5 AM and 2 PM on Friday, but until we demonstrate a clear, repeatable, and statistically significant pattern between my dumping and the rising or falling of the tide after all other potentially influencing variables have been accounted for, we've proven exactly nothing. It will be far more likely that the tides are driven by some other factor/s, like say, the moon and sun for instance.

Until Lott demonstrates that those Armed Robbery rises and/or drops are directly correlated somehow with the 1996 law, and not with anything else, all of his, and everyone else's discussions of this rise here or that drop there are irrelevant.

BTW - One more addition to my last post. Bob H, you noted that 1996 is the first year that there is a big increase in the Armed Robbery rates. Since this was the year the law was implemented, there would not have been time for the impact of it to filter down to actual gun ownership, and gun use in crimes or defense, and therefore, the 1996 rise also does not correlate to the law. The rise in rate, as I noted in my 9/29 post, began prior to this and was actually under way and picking up speed in 1996. The fact that this was already in motion when the law was rolled out implies that the law did not cause it. Once again, we're back to the previous point - no correlation.