Deltoid

"Eugene Volokh" writes:

but I was wondering what you thought about the NCVS point I raised again a few days ago. To my knowledge, waiting for respondents to volunteer information is generally considered rather bad survey practice; and we saw that with the rape statistics shifting to a direct question changed the total by about a factor of 2.5 or 3, if I recall correctly.

I have even been told -- entirely outside the defensive gun use context -- that the trick is cuing as often as possible: Asking the question directly, several times, in subtly different ways, to trigger people's memories (and perhaps willingness to respond).

This was considered in the NCVS redesign -- it asks the screening questions in several different ways. This would seem to be better than just having one screening question, as Kleck's survey does.

How exactly is this done in the NCVS redesign? I see a good deal of extra cuing as to rape, but little as to defensive use.
There are indeed two questions introducing the main issue: "Did you do anything with the idea of protecting YOURSELF or your PROPERTY while the incident was going on?" and "Was there antything you did or tried to do about the incident while it was going on?"

I was refering to cueing to get the person to recall the crime incident, not the detail of using a gun for defence. Because it uses more cueing questions, you would expect the NCVS to be better at getting the person to recall the incident in which the DGU occured than Kleck's survey.

But if someone says "Oh, I shouted at the guy and he ran away," there's to my knowledge NO follow-up question "What did you shout?" or "Did you shout anything about a weapon?"

I think they just ask something like "Anything else?"

Now of course some people will say "I shouted `I've got the gun' and he ran away." But others won't be that specific, and not just because they forgot about the gun or are reluctant to talk about it.
Some people, in an interview like this, will go into gory detail with the mildest of prompting. Others, and this is often just a matter of temperament, will give a relatively short answer, especially when it's an answer to question #42 (I realize it might not actually be the 42nd question -- some might have been skipped -- but it's not the second, either) and no end obviously in sight. If you don't ask for more detail, you won't get it. This is the lack of cuing that strikes me as particularly problematic, though I agree that both forgetfulness and reluctance are troublesome, too.

I dunno. It seems to me that stating "I shouted at the guy and he ran away," when you in fact used a gun for defence is highly misleading and that most people are not so poor communicators that they would do this accidently. The only way to find out for sure is to experiment with different questions. I don't know if this is one of the things they tried out in the NCVS redesign.

Finally, though, I'm happy to hear that there seems to be some agreement that the NCVS probably undercounts, at least by a factor of two (though I wonder why it would be just a factor of two).

I didn't say at least, I said at most. Since Kleck turned up so many fabricated DGUs, it seems probable that some of the NCVS DGUs were also fabrications. If this overcounting exceeds the undercounting we have already discussed then the NCVS will overestimate the number of DGUs. Even if the two factors cancel out and the NCVS estimate is correct it means that you cannot use the NCVS to argue that guns are the most effective means of self-defence.

Why a factor of two? It seems to me that a majority of gun owners will not answer straightforward questions in a misleading way. (If exactly half did, then the NCVS would bu out by a factor of two.) Also, the indirect estimates I have been to make all seem to end up as the same order of magnitude as 80,000.

Do we agree, then, that saying "there are 80,000 defensive gun uses a year" is inaccurate (just as I personally agree that saying "there are 2.5 million defensive gun uses a year" is inaccurate --- I always try to qualify any such statements about total uses)?

I've certainly not made such a statement. I would say "The most reliable estimate of DGUs is the NCVS one of 80,000, but even this might be out by a factor of two."