How accurate is Wikipedia?

Orin Kerr writes writes about Wikipedia

My very tentative conclusion, based on a just few sample queries, is that I hope no one relies on Wikipedia for anything very important. Its entries seem to be a strange mix of accurate statements and egregious errors.

My own experience is that Wikipedia is quite accurate and errors get corrected. An erroneous description of the Patriot Act that Kerr pointed to was quickly corrected. It seems counterintuitive that letting anyone edit any page would result in quality information, but that seems to be what has happened.

Kerr argues:

If I understand accurately how Wikipedia works—a big “if,” I should point out—my views of what is in the Patriot Act are no more and no less valued by Wikipedia than the views of any other Internet user. Given the widespread misperceptions about what is in the Patriot Act, some one else is likely to come across my corrected entry and think, “What idiot wrote this? This is totally wrong!” They will then erase my entry and re-enter all the mistakes that I corrected. The “genius” of Wikipedia is that no one is there to resolve the disagreement: the loudest voice eventually wins.

Well, it doesn’t seem to work out that way. For example, someone with IP address 38.118.12.78 has been changing the Wikipedia entry on John Lott to remove any criticism of him. (The scrubbed version is here.) Mr 38.118.12.78 is very persistent—so far he has replaced the page eight times. However, since his changes are obviously unreasonable, each time they have been undone. The loudest voice has not won.

Oh, and IP 38.118.12.78 resolves to americanenterpriseinstitute.org.

Comments

#1 Jonathan
October 22, 2004

So this isn’t a very original question, I know, but what exactly do you have to do to get fired from AEI? If I spent all day writing fake reviews on Amazon or selectively editing Wikipedia entries, I’d have to look for a new job.
#2 Kristjan Wager
October 22, 2004

But then, you were not hired to do that kind of stuff.
#3 Sharon
October 22, 2004

But here’s my worry. How do I know that I’m looking at the entry after it’s been corrected rather than before? And I wouldn’t want to direct others to it as a reference because I don’t know what it might say tomorrow… or the day after. So this is one type of online resource I tend to avoid, I have to admit.
Of course, the golden rule applies here as anywhere: never rely on just one source of information (unless you really, really have no choice)…
#4 dsquared
October 22, 2004

No need to be so coy about “Mr Number”, Tim; Lott has coughed to this one in Wikipedia user talk.
#5 Jeff Harvey
October 22, 2004

I think that WIKIPEDIA comments on controversial people is often highly biased, depending upon the perspective of the reviewers. A case in point is the controversial anti-environmental writer Bjorn Lomborg. I have co-reviewed his book (which I still consider to be an atrocity) but the Wikipedia entry seemed to be stacked in his defense. For instance, in the introduction it had said (more-or-less) that Lomborg had upset scientists whose livlihoods depended on scaring the public to secure funding (seems more appropriate for the likes of Bush, Blair etc. who perpetually scare the public with phantom threats of terror to legitimize their economic wars). But seriously, this kind of smear is abominable. My research has nothing whatsoever to do with scaring the public (I study communities and ecosystems). Moreover, one can turn the tables quite easily and see that Lomborg has found a useful niche because his claim. that `everything is getting better` warms the hearts of those in the establishment who are eager to bolster the status quo.
Furthermore, the Wikipedia entry makes no mention of the serious allegations against Lomborg: 1. That he misunderstands basic concepts, 2. That he cherry picks studies that support his rosy views and dismisses studies that do not; 3. That he misquotes scientists, distorting the original meaning of their words; 4. That he attacks the integrity of many senior scientists with smears and outright distortions to legitimize his own position; 5. That he bases his conclusions not on the empirical data but on wishes or expectations; 6. That he refused to amend the more serious errors that plagued the first Danish edition.

Furthermore, I don`t think they`ll allow a passage to be added that I think is highly relevant: John Quiggin rightfully stated that the number of peer-reviewed papers by Lomborg on any of the areas covered in his book is ZERO (in fact, he has only one peer-reviewed paper in his 10 year academic career, and that is on games theory). For this reason, and because I have invested so much effort to debunk his nonsense, I prefer the DISINFOPEDIA pages. At least they peel away the veneer!
#6 Carl Jarrett
October 22, 2004

My bet is that Lott went after Wikipedia because that Wikipedia page gets posted on the usenet quite frequently. Lott still obsessively monitors the usenet and still appears under the name of his son every so often. If Lott gets so easily upset by the various criticisms, he must have gone ballistic when he received peer reviews of his journal articles. An academic who can’t handle criticism is an academic that will never get tenure…
#7 mark
October 23, 2004

Sharon, every Wikipedia page has a “history” link on it where you can see (and compare) every revision ever made. However, it’s a bit OTT for readers to have to check “history” every time they want to cite something, just in case there’s been some petty scuffling (as in John Lott vs everyone else on his page) about what The Truth should be.

I wouldn’t trust Wikipedia for controversial issues, as not everyone respects the Neutral Point of View ideal and sometimes we see a page before correction. Also, I wouldn’t cite it in academic work, if only because it lacks prestige (“ooh, an online encyclopaedia, eh? And what did a real source say about it?”). But for the casual reader, looking for extra information on a variety of topics, it’s fascinating and — usually — quite credible. I sometimes get lost for ages, following links all over the encylopaedia…
#8 Kristjan Wager
October 23, 2004

Jeff, it is hardly surprising that Lomborg only has zero peer reviewed articles about the subjects in his book, since it is hardly his area of expertice (he has a degree in social/political science, and not even in statistiks as many people think).
However, it is surprising that people believe he is qualified to write a book on the subjects.
#9 Martin Wisse
October 23, 2004

Where wikipedia breaks down is in very specialised subjects, where you have only a handful of experts and much of the common wisdom on the subject is wrong.

Controversial subjects, e.g. anything to do witht he US elections can and do break down into editing wars, but fortunately Wikipedia has a class of dedicated, as neutral as possible editors who can and do interfere when things get out of hand.

Should you blindly trust Wikipedia? Of course not, no more than any other source of information. But there is no reason to trust wikipedia less than any other source of online info.
#10 Kristjan Wager
October 24, 2004

Martin, you are right of course – the problem is that I have come across a few cases where people refer to Wikipedia as an authority on an issue. It’s a nice tool, but not an authority by any means.
#11 Louis Hissink
October 25, 2004

Scientifically?
#12 William Connolley
October 27, 2004

I don’t understand JH’s criticisms of the wiki article on Lomborg.

Its http://en.wikipedia.org/wiki/Bjorn_Lomborg if you want to read it.

I’ve edited it. JH could too if he wanted to. The intro says the book is controversial (true) and attacks enviros (true). Then the very first section (after the bio) is “accusations of scientific dishonesty” and it goes on to quote from the DCSD ruling aginst him. You may want the article to say his book is a pile of tosh, but that would be POV so would be quickly removed.
#13 Jeff Harvey
October 27, 2004

William,

It was me who changed the ridiculous assertion that the book was attacked by scientists whose livlyhoods [sic] depend on arguments scaring the public in order to secure funding. But I wonder how such an unsubstantiated point – which amounts to an ad hominem attack on the integrity of the scientific community at large – could have passed the scrutiny of those who maintain the site. Of course, Lomborg uses a similar argument in his book (realizing that most of his gobbledegook arguments lack an empirical basis, so it’s a clever ploy to defuse any or all of the criticism that he knew was headed in his direction), but I thought Wikipedia would know better than to rehash it. I don’t want the site to, as you colloquially say, state that his book is a “pile of tosh” (although it is), but that single statement in the intro was inexcusable.

And BTW, I work for the Netherlands Institute of Ecology, and not Technology. I couldn’t be bothered to change that one.
#14 Tim Lambert
October 27, 2004

Jeff, the ad hominem did not pass scrutiny — you removed it. Anyone can put rubbish into a Wikipedia entry but it usually doesn’t seem to last that long. It’s nowhere near perfect, but I find most Wikipedia entries to be good quality.

Do you think there is something wrong with the current version of the Lomberg page?
#15 William Connolley
October 27, 2004

Jeff – I’m curious. When did you make the change (an as who?). I can’t find it from the edit history.

As to “I thought Wikipedia would know better than to rehash it”… well, wikipedia doesn’t think, only the contributors do, and there are (of course) opposing views.

The reason I ask when you edit was, is to see how long the “wrong” text persisted for.

(ps: any chance for a bit of wiki syntax here? Ie, blank lines = new para; URLs auto-link…)
#16 Jeff Harvey
October 28, 2004

William,

I made the change last week. Normally I have much better things to do than to change Wikipedia entries but as I am on a sabbatical in Japan, I thought I would make an exception. I think that ad-hom comment had been in there for at leadt three or four months, judging by the history of the page. I was seriously tempted to counter it by suggesting how much money Bjorn has made from his book (and from his tedious lectures, where apparently some industries and think tanks are coughing up big bucks to hear his usual spiel). I also added John Quiggin’s comment, which puts Lomborg’s relevant ‘expertise’ to serious question. On Lomborg’s Wkipedia page there is a great deal of information on his professional interests, all of which have resulted in a single paper in a peer-reviewed journal (in 1996, on games theory). The Greenpeace claim also appears to be bogus, because Greenpeace has denied it and when challenged on it Lomborg has stated that he is a “suburban kind of member”. There is no such thing – he thus seems to be stating that he supported the organisation without any evidence that this was so.

Tim, I agree that the page could be worse. Since I gave him a battering in a debate we had on his atrocious biodiversity chapter in Holland two years ago, the guy steers well clear of me (several offers to attend my lectures or debate me have been refused). Having been involved in debunking his book almost from day one, I am a thorn in his side and he knows it. I do not hesitate at all to state that he’s a supreme con-man and deceiver, and that the media-love in with his polemic has more to do with his saying things that the establishment desperately want to hear and promote, rather than containing much in the way of empirical truth.
#17 William Connolley
October 28, 2004

Following a lot of confusion, I have finally made sense of what JH has been saying: he is talking about the

Skep Env’ist

page on wiki, *not* the BL page…
#18 Mark A. York
November 1, 2004

I don’t like the open committee concept of this. real definitions are corrupted by misinformationists. I vote no.
#19 Al Lowe
September 20, 2005

“…If I spent all day writing fake reviews on Amazon or selectively editing Wikipedia entries, I’d have to look for a new job.”

I have asked this very same question, numerous times. I’ve yet to receive an adequate (or ANY) answer.

With regard to Wikipedia, I’m afraid Mr. Kerr is more likely correct. It does appear that the loudest, or rather the most persistant voice, is the one that remains on Wikipedia. At least with regards to any item that is what we might call controversial.
#20 Jack Strocchi
September 26, 2005

Wikipedia is the application of evolutionary selection principles to collaborative intellectual effort.