Donohue and Levitt’s new measure of abortion

Steve Levitt has a post with a detailed response to Foote and Goetz’s paper. They construct a new, better, measure of abortions under which more abortions are associated with less crime. They conclude:

The results we show in this new table are consistent with the impact of abortion on crime that we find in our three other types of analyses we presented in the original paper using different sources of variation. These results are consistent with the unwantedness hypothesis.

In comments to Levitt’s post Steve Sailer raises objections that do not impress me in the slightest but Daniel Davies makes a good point here:

Finally and most importantly, this is about as far from a double blind trial as you can get. I’ve written in the past about the perils of data mining in econometrics, and to be honest, all that is lacking in the series of changes to the data and the model that the Freakonomics blog presents is a phalanx of dwarves singing “Hi Ho, Hi Ho, It’s Off To Data-Mine We Go”. What has happened here is that Levitt and his research assistant have sat down in the knowledge that a perturbation to their model doesn’t deliver their result, and decided to have a think about what kinds of alterations to the data ought to be made.

You don’t need to suggest any intentional dishonesty to say that it is somewhat unsurprising that the outcome of the brainstorming session on “What sort of changes ought one to make to this data, in an ideal world?” was a dataset and model in which the result that Levitt is famous for was present. Even if Levitt and Ethan Lieber had sat down at a table with no computer on it, starting with a blank sheet to discuss the changes to make and not touching the model until they had finished, I would still guess that it would be the easiest thing in the world for someone who was intimately familiar with the dataset to subconsciously put his thumb on the scales. And I don’t think this is what they did; colour me cynical but I would bet quids that lots and lots of iterations of different possible changes to the data were tried. I note once more that there is no accusation of intentionally cooking the books here; medical science certainly doesn’t insist on double blind trials to protect them from unscrupulous doctors.

I think that there’s a general issue here which is endemic to the territory that Levitt chooses to operate in. By their nature, political debates are debates. One side produces arguments, the other side produces counterarguments and so on, so iteratively. This is an environment which is absolutely poisonous to datasets. By the time you’ve been through two or three iterations of a “controversy” like this it’s more or less impossible to pick a model without failing even the most homeopathically weak version imaginable of a double blind criterion. This is why I now say that we’re simply never going to know the truth (by which I mean, even the simple statistical truth about the existence of a comovement, much less the truth about the underlying causal hypothesis) about abortion and crime in the period 1976-2000. Stick a fork in this dataset, it’s done.

I don’t think that the situation is as hopeless as that. Foote and Goetz have access to the same data and tools so we can see if they can come up with another measure of abortions
that makes the results go away. Another possibility is that Donohue and Levitt present the results for a whole slew of alternative formulations of the abortion measure so we can see if their results are sensitive to the particular way that it is defined.

Comments

#1 Eli Rabett
December 11, 2005

I love it, ensemble datasets to go with ensemble forcasts. Still the distribution of results from such an exercise should be normal, and given enough examples outliers will be obvious. It might be possible to do this with paleoclimate data.
#2 Jack Strocchi
December 11, 2005

The lengths some people will go to to keep their names in lights.
#3 Jack Strocchi
December 11, 2005

Levitt and Donohue have moved to rescue their theory from inconvenient econometric tests by introducing, at this late stage, an entirely new set of data that would match their theory. For some reason this reminds me of Bertold Brecht’s sardonic comment on Communist state’s attitude towards inconvenient election results:

Would it not be easier
In that case for the government
To dissolve the people
And elect another?
#4 guthrie
December 11, 2005

I’ve never heard of this controversy before. (I am in the UK though, I wonder if that helps) From reading the freakonomics page and posts, it looks to me like its related to the old eugenics idea, whereby you get rid of the poor and there will be less criminals? Is that something close to it?
And, even if the correlation abortion -> less crime is correct, (Going by mere anecdotal evidence over here, where we’ve had abortion for decades and crime has been on a rising trend for nearly 30 years) what does it matter?
#5 Brian S.
December 11, 2005

I agree with Tim that political controversy doesn’t completely ruin any chance to settle a scientific controversy.

For example, one of the best validations of theories is through predictions. Freakonomics uses the banning of birth control in Romania as a validation in the past. Their theory should also predict a drop in crime starting several years from now due the reinstitution of birth control and abortion in Romania in 1990. We’ll see what happens, and it will be hard for either side to argue their way around the result if it goes badly for their position.
#6 Ian Gould
December 12, 2005

While it may be “unsurprising” that researchers attempting to correct errors just happen to produce modifications to their model which ensure that the original results are confirmed, it is still disappointing and unprofessional.
#7 dsquared
December 12, 2005

Guthrie, if you’re in the UK, crime has not been on a rising trend for the last 30 years. It has been falling for the last ten.
#8 Harald Korneliussen
December 12, 2005

Brian S: I’d guess there were quite a lot of other things which changed in Romania around that time. How could you possibly separate the influences from one case only?

As far as I’ve been able to find out (it’s won’t work to just google for an answer on this, unfortunately) the first modern nation which introduced abortion on demand was the newly formed Soviet Union, in 1918. Stalin removed it again out of population growth concerns, it was then reintroduced yet again, and used as the principal means of family planning until quite recently. If you are to study just one country for the effects of abortion laws, the former soviet union should be it.

Whatever we would find wouldn’t say anything about whether abortion is morally acceptable. (I’m sure things like eugenics could be seen as being beneficial to society, and there are also things which may be detrimental to society, but are nevertheless imperative)

I’m more interested in the effect on fertility than on crime. Russia has a truly abysmal fertility rate, which I don’t believe can be explained by vodka alone. I would appreciate a link, if anyone has anything (I’ve already googled it to death).
#9 Steve Sailer
December 12, 2005

Levitt’s theory, which struck many influential people as plausible precisely because it is a version of an old eugenic concept, failed even to predict the past in the U.S. The teen murder rate tripled among the first generation born after legalization.

The interesting question at this point is less whether legalization of abortion had a slight positive or negative effect on serious violent crime, but why the effect was so small relative to the extraordinary rise and fall of crime that ripped through America between 1986 and 1999. There’s a lot that can be learned about American society and social policy precisely from the failure of Levitt’s seemingly commonsensical assumption to pan out historically in the U.S.
#10 Steve Sailer
December 12, 2005

A reader says: “I’m more interested in the effect on fertility than on crime.”

According to Levitt, legalizing abortion in the U.S. had surprisingly little effect on fertility even though 1.5 million abortions were soon performed per year. Fertility fell only 6% (4% among whites, 12% among blacks). Pregnancy rates went up almost 30%, however. (Sexually transmitted disease rates also went up substantially.)

Apparently, legalized abortion turned out to be not just a cure for, but also a cause of, unwanted pregnancy.
#11 Brian S.
December 12, 2005

Harold: you’re right, the side that dislikes the statistical result in Romania will make exactly that argument. If there’s a strong signal in the data though, they will be very unconvincing.

It would be helpful to get each side to make their predictions now, so the hemming and hawing of the loser in the future will have to be somewhat constrained.

And there’s even the extremely remote possibility that someone might admit they’re wrong.
#12 ben
December 12, 2005

Speaking of comparing the former Soviet Union, the UK, Romania and others with the USA, it seems that cultural differences preclude a straight forward comparison. Seems like a daunting task to me, to come up with valid relationships here. How would you handle that?

There are other, similar reasons to conjecture that legalized abortion would lead to lower crime rates besides the “unwanted child” hypothesis. You could also conjecture that a person who would have an abortion is more likely to be 1. single and 2. an unfit parent. So by 1 you could test the additional hypothesis that it is better to have two parents. 2 comments on the type of person who has abortions. I’m sure there are many other interesting and important differences, on average, between people who would have abortions and people who wouldn’t, except that it doesn’t make any difference if any of them are men, oddly enough.

I guess the point is, when you make a conjecture about the effects of legalizing abortion on crime, does it make a difference why you think the affect on crime will or will not occur? It seems reasonable to think that there would be an effect on crime, I’d be surprised if there wasn’t. Like everything else, it’s too bad we can’t simply run the experiment both ways. That would almost be definitive.
#13 Ian Gould
December 12, 2005

Just thinking abotu the possible mechanism by which legalised abortion COULD lead to lower crime rates.

The focus to date has been on the idea that aortion leads to fewer children at high risk of becoming criminals.

But what effect does legalised abortion have on the women who get abortions?

I can see this going in either direction.

Many women find abortion traumatic and an increase in abortion rates could lead to an increase in mental illness or drug problems for the women who’ve had abortions. This would tend to increase the crime rate.

Alternately, legalised aboriton might lead to fewer single mothers; less welfare dependancy and therefore tend to reduce crime rates.

I’m talking off the top of my head here and simply throwing ideas out, just trying to suggest that the effects of legalsied abortion are likely to be much more complex than “fewer unwanted kids”.
#14 guthrie
December 12, 2005

D squared- yup, I suppose your right. But we’ve had legalised abortion for IIRC upwards of 40 years, during which time crime has gone up and down. Which suggests to me that there is little link at all, although since I know no statistics, I shall leave you all to argue about it.

Or another random thought to continue ian goulds, what if the women having abortions were of the middle classes and above who are traditionally suppose to be good at bringing up children who dont turn into criminals. Whereas if there are few abortions amongst lower class jail fodder, then, assuming that the correlation between being lower class poor people = more criminal, then there wont be any difference.
#15 ben
December 12, 2005

Without exception, the women I know who have had an abortion (about five, I think) were shallow and self-centered (in the non good way of being simple pleasure seekers). That sort of person is traditionally viewed as a poorly suited to motherhood. Their standing as middle class and above would not have helped.
#16 Steve Sailer
December 12, 2005

It’s quite possible that legalizing abortion had different effects in the United States than in other countries. There are obvious differences between the American population and, say, the Romanian population that should make one cautious about generalizing from Romania to America, or vice-versa.

For all I know, changes in the abortion law in Romania really did have a substantial impact on crime rates there. I haven’t looked at the Romanian data (and I’m not sure how valuable data from the old psychotic regime in Romania would be). What I do know is that whatever the effect of legalizing abortion was on the serious violent crime rate in America, it was swamped by other changes.

To bolster his theory that abortion-cut-crime in America, Levitt cites studies from Europe claiming that legal abortion was used more by women who would have been incompetent mothers. He doesn’t mention American studies of the question, because most of them point in the opposite direction.

Economist Ted Joyce of CCNY pointed this out to Levitt in a critique of the Levitt-Donohue 2001 paper:

“Second, analysts, I being one, have tended to overestimate the selection effects associated with abortion. A careful examination of studies of pregnancy resolution reveals that women who abort are at lower risk of having children with criminal propensities than women of similar age, race and marital status who instead carried to term. For instance, in an early study of teens in Ventura County, California between 1972 and 1974, researchers demonstrated that pregnant teens with better grades, more completed schooling, and not on public assistance were much more likely to abort than their poorer, less academically oriented counterparts (Leibowitz, Eisen, and Chow 1986).

“Studies based on data from the National Health and Social Life Survey (NHSLS) and the National Longitudinal Survey of Youth (NLSY) make the same point (Michael 2000; Hotz, McElroy, and Sanders 1999). Indeed, Hotz, McElroy, and Sanders (1999) found that teens who abort are similar along observed characteristics to teens that were never pregnant, both of whom differ significantly from pregnant teens that spontaneously abort or carry to term.

“Nor is favorable selection limited to teens. Unmarried women that abort have more completed schooling and higher AFQT [the military's IQ test for applicants for enlistment] scores than their counterparts that carry the pregnancy to term (Powell-Griner and Trent 1987; Currie, Nixon, and Cole 1995).

“In sum, legalized abortion has improved the lives of many women by allowing them to avoid an unwanted birth. I found little evidence to suggest, however, that the legalization of abortion had an appreciable effect on the criminality of subsequent cohorts.”

But, for some reason, Levitt forgot to mention all the American studies shooting down his theory of American crime rates in “Freakonomics.”

A general bit of advice is that social scientists should not automatically assume that how people they know (e.g., other academics) think and act is also how people think in act in those segments of the population that produce most of violent criminals. Levitt’s model of how poor American women think about pregnancy and abortion is extremely naive.
#17 Eli Rabett
December 12, 2005

Whereas Ben is as deep as the mato grosso in summer, or something like that. I think Romania will be a particularly hard case because it got exceptionally poor at the end of the Ceausescu dictatorship and much poorer after that. It is growing at over 5% per year in the last few years, but it is easy to grow at a fast rate when you are coming back from nothing. IMHO econometric analysis might be capable of separating roughly equal effects, but it would be damn hard to dig a mosquito out from an elephants butt.
#18 Harald Korneliussen
December 13, 2005

It’s interesting to hear that women who are well off are more likely to have an abortion in the US than poorer women. I’d have thought that with the social safety in the US being what it is, poor women would have been forced to take an abortion. Apparently, they don’t take such pragmatic concerns as often as wealthier women?

Perhaps it’s like here in Norway, poorer people are more likely to be active Christians, and active Christians are (irrespective of denomination) strongly opposed to abortion.

As I’m already throwing out questions, I’ve heard the assertion made that eugenics advocates supported the legalisation of abortion, because they though that it would lead to the poor and stupid getting fewer children. Does anyone know of any examples of this? Or of the opposite? (eugenicists opposed to abortion?)
#19 dsquared
December 13, 2005

Romania could have been very different because abortion used much more as a means of birth control in a lot of Eastern European states, and the banning of abortion was explicitly a population-growth measure, so there are plenty of size-of-cohort effects to deal with.
#20 Brian S.
December 13, 2005

Sounds like Ben and Steve expect their side to lose in Romania.

Eli’s got a good point, but I’d expect the effect of a constantly-growing economy to be constant, while the abortion effect, if any, should kick in suddenly.

I don’t quite understand dsquared’s point on how the size of cohort makes a difference. As for abortion as birth control, it seems to me that all birth control serves a similar function of preventing unwanted children. Abortion also adds “one last chance” that’s especially useful for the less-sophisticated teenage mother. As a quantitative comparison, I can see d’s point, but as a qualitative assessment, I think Romania is relevant.

I’d agree that Romania will be one more data point, and not be the entire argument.
#21 z
December 13, 2005

“women who are well off are more likely to have an abortion in the US than poorer women”

dubious; i know a ob/gyn in a large southern us city, and a large portion of his practice consists of abortions for inner city (i.e. poor black) teenagers who are not competent to take a pill every day or use a condom, and really do rely on multiple abortions for birth control.
#22 ben
December 13, 2005

Whereas Ben is as deep as the mato grosso in summer, or something like that.

Maybe I’m not deep, but my kids are my first priority. For these women, partying and the like was their first priority and that’s why they had abortions; kids would have cramped their lifestyle. Except for one, her first two were for that reason, and the third was due to severe physiological defects, she wanted to keep it. And no, none of them were women I knew.
#23 z
December 13, 2005

of course, anything that decreases the number of teenish males in society will reduce crime, generally speaking, over all social and racial categories. Unless of course you believe violent crime is strongly correlated with the number of middle aged black women.
#24 Jack Strocchi
December 19, 2005

Brian S. Says: December 14th, 2005 at 1:04 am

Sounds like Ben and Steve expect their side to lose in Romania.

Romania is the basket case of Europe-in-transition and not very comparable to normal civil societies. The Levitt-apologists might want to look at the Russian crime rate. My impression is that Russian crime rates started to climb in the seventies (it was called “hooliganism” then) about a generation after Kruschev legalised abortion.

Obviously the causes of Russian crime rate changes are complex. There are other factors at work here, the repeated totalitarian assaults and subsequent mafiya driven break down in Russian civil society have not helped. But this only shows how weak changes in the abortion rate are at predicting changes in the crime rate.

The debate about abortion-cuts-crime smells fishy to me. It is being driven by ideological axe grinders. On the Right eugencists think that pre-natal culling of minority groups will reduce social pathology. On the Left the feminists think that abortion is a sacred right so any exercise of it must likewise be wreathed in sanctimony.

Most sensible people, eg your mother, know that promiscuous unsafe sex, abortion, crime and drug-usage are all positively correlated and symptoms of moral collapse. They might also be aspects of fun-loving libertarianism. But it does not impress me that Levitt-apologists are so concerned to justify abortion that they are prepared to turn a blind eye to more ppwerful causes of crime.

I think that they just enjoy being contrarian because it is so…transgressive.
#25 Jack Strocchi
December 19, 2005

z Says: December 14th, 2005 at 8:37 am

of course, anything that decreases the number of teenish males in society will reduce crime, generally speaking, over all social and racial categories.

Not necessarily. East Asian Americans, and their children, commit crime at much less than half the average US rate. So reducing the ratio of East Asian teens in society, by abortion say, will actually increase the actual crime rate above its projected (East Asian-inclusive) rate.

The effect on the national crime rate per capita, however, depends on whether the per capita crime rate for a group is above or below the national average. For example, getting ride of all Asian Americans would raise the national per capita crime rate because Asians are imprisoned a rate barely than 1/5th that of non-Hispanic whites and 1/33rd that of African-Americans.)

I am not sure what the crime rate for Asian teens is. But my educated hunch is that it is still below the rate for non-Asian non-teens in the US. Does anyone have the data on this. They seem to awfully dedicated swots, with not much time left over for hanky-panky.

Unless of course you believe violent crime is strongly correlated with the number of middle aged black women.

Don’t you mean “middle aged black [mothers]“?
#26 dsquared
December 19, 2005

I think that z was making the comment about teenage males in the context of what we should expect from Romania. I have not actually checked the numbers but I suspect that we can probably ignore any effect on the data coming from a change in the proportion of East Asians there.
#27 z
December 19, 2005

In general in any group it will be the adolescent (including early 20s) males who are involved in crime and crime-like behaviors, which is not unrelated to why there are a lot of adolescent American males in Iraq fighting with a bunch of adolescent Iraqi males.

As for East Asians, there are now a lot of singularly nasty Asian gangs in the US.