I had wanted to proceed to part 5 of my Medicine and Evolution series, but, frankly, I wasn't much in the mood for anything serious over the weekend, and, let's face it, that case of the blog blahs continued even into yesterday. Otherwise I would have done my blog buds Abel Pharmboy and Bora more of a solid and tackled this a bit earlier, rather than posting about Jack Chick parodies and fluff about a possible new Star Trek movie.
Spurred on by Bora over at Science and Politics, Abel Pharmboy at Terra Sigillata has been discussing scienceblogging, whether it would be a good idea for science bloggers to publish hypotheses and data on their blogs. More relevant to me, he's also brought up the question of whether starting up a medical Wikipedia would be a good idea, while linking to this article about the first five years of Medscape and this editorial by Peter Frischauf, who strongly argues for a medical Wikipedia (free registration required).
I'm not going to comment extensively about whether publishing data and hypotheses on a blog is a good thing, whether it can be a new way to disseminated scientific information, but I will comment briefly. For science, certainly getting ideas out there is a good thing, but from a career standpoint, I have very grave doubts over whether publishing such information is advisable, particularly for young scientists who could easily be scooped by larger, more established labs. I would never recommend that any young scientist publish a hypothesis that way, unless it is a hypothesis that he doesn't intend to test (whether because of lack of resources or whatever) but is hoping to get another group interested in.
While I also agree with Bill Hooker that blatant scooping is despicable and that scientists who practice it as blatantly as the example he cites should be shunned, I'm not as sanguine as him about actually making such a culture change come about any time soon. Besides, most cases of scooping aren't nearly as blatant as the one he describes. Most are a lot more subtle, and the vast majority don't involve any chicanery at all. Indeed, in my experience, most cases involve multiple labs working on the same question. In such cases, one of these groups will inevitably succeed at publishing their results first, and the rest will be "scooped," no dishonesty or using ideas or experimental protocols without appropriate attribution necessary. Important questions in science often attract multiple labs to solve them. If the principle investigator of a lab in a hot area of research publishes ideas and data that give his competitors, who were already working on the same question, a leg up, either by informing them of a blind alley that they shouldn't waste time pursuing or of a promising experiment to try that is highly related to what the other lab was doing anyway, then the PI prematurely revealing his ideas has no one to blame but himself for getting "scooped." (In fact, I wouldn't even call it getting "scooped.") Besides, in the case of a junior scientist accusing a big name in his field of scooping him, who's most likely to be believed? True, it's not fair, but that's the way the world works, and not just in science. In an ideal world, Bill Hooker's concept would be the way things should work and any hint that labs might be scooping each other would result in offers of collaboration, but that isn't always how things actually work. Besides, competition is not always a bad thing in science; it spurs investigators on in a way that even a scientists' love of discovering something new can't always do.
Maybe I'm just turning into a cynic in my middle age.
As for a medical wikipedia, although the idea sounds superficially tempting, I reluctantly remain skeptical of the idea. The whole concept behind a Wikipedia is that anybody can contribute articles or edit currently existing articles. Indeed, Wikipedia's slogan is "the free encyclopedia that anyone can edit" regardless of qualifications. However, I have grave doubts about whether this could work so well in medicine. For one thing, allowing anybody to contribute would open the door to any number of quacks, alties, and activists to alter articles to push their favorite forms of pseudoscience and quackery. Even though there are safeguards in place at Wikipedia to try to prevent this sort of chicanery, it doesn't take much imagination to predict that Generation Rescue or SafeMinds would hijack any articles on autism, vaccines, or mercury to fit their agenda that claims that mercury from vaccines is the primary cause of autism in children, despite all the evidence to the contrary, or that organizations made up of HIV/AIDS "skeptics" would try to alter articles on AIDS, HIV, antiretroviral drugs, and related topics to enshrine their bad science in what would certainly become a major resource for medical information targeted at the lay public. Similarly, it's not hard to imagine individual dubious practitioners like Dr. Rashid Buttar trying to edit articles on chelation therapy or various alternative medicine practitioners trying articles on homeopathy to make these treatments sound effective. Indeed, having been involved with the refutation of Holocaust denial, I know that some of my fellows in the fight against Holocaust denial have become active in Wikipedia because Holocaust deniers frequently try to alter articles on the Holocaust and various death camps to minimize or deny the magnitude of the atrocity. Any dedicated medical wikipedia would require a large number of expert volunteers to do the same. Where will these experts come from?
The very concept behind Wikipedia is that "exposing an article to many users will result in accuracy," and that is presumably what would keep the dire predictions I made above from happening. Maybe so. However, here's the problem with that concept in medicine. Activists and alties tend to have lots more time and, more importantly, lots more motivation than scientists and physicians who are experts in the field, and they would use them to distort the information on such a medical wikipedia. It would be a constant battle. Volunteers with real credentials would be hard pressed to keep up with all the attempts at editing controversial articles, unless there were some sort of mechanism in place to limit who could edit articles. Indeed, nearly everyone discussing a medical wikipedia seems to recognize implicitly that a true open wikipedia would probably not work in medicine, because nearly everyone I've seen who brings up the topic seems to qualify it by saying that contributors would have to register and have their qualifications checked. But who would determine what qualifications for which topic are sufficient or appropriate? (Remember, a fair number of purveyors of dubious treatments are doctors.) How many people would it take to evaluate potential new contributors? And, if there is an "entrance" test for contributors, then would this medical wiki really be a wikipedia at all? Would it not be, instead, simply a more rapidly updated real time version of medical sites like WebMD or Medscape rather than a true wiki? (Not that such a rapidly updated resource wouldn't have the potential to be a very good thing indeed for patients. It would.)
Finally, I also have to disagree with Peter Frischauf's questioning of whether the traditional model of peer-reviewed articles is obsolete. His complaints, that publishing is too slow and sometimes results are out of date by the time they are published, are valid. However, the solution is not to abandon the model that has served us so well for so long, but rather to develop ways of more rapidly publishing results and widely disseminating them. For example, many medical journals it can take a year or more from the date of first submission before an article sees print (it's usually half that in basic science journals), even though the time for peer review for the two types of journals does not differ all that much. There is no reason that couldn't be sped up considerably if we are wiling to invest the resources. Also, distributing the results widely and for free, as the NIH is now doing by requiring that all accepted manuscripts be registered and submitted to a database.
Finally, he states:
For readers, Wikipedia is a win. In traditional publishing, readers must wade through many articles on a subject, each written by a few experts, published at 1 moment in time. In Wikipedia you read 1 living article written by many, continually updated by many. Who needs 50 articles on avian flu when 1 will do? And Wikipedia content is often the best on the Web, which means the best anywhere.
This is so overly optimistic that I have to wonder if Frischauf has actually evaluated a large number of Wikipedia articles. In my experience, at least, Wikipedia content can be maddenly uneven in its quality and completeness. Some of it is excellent; most of it is merely adequate; some of it is useless. It's way too hit-or-miss to be used as a primary resource for information, particularly specialized information, without evaluation of additional sources.There is no reason to think that a medical wikipedia would be any different. Wikipedia is a useful tool as a first pass at obtaining information on a variety of topics and getting ideas about where to look for more information, but for the vast majority of topics it doesn't usually provide much in-depth analysis. (It is, however, very useful to bloggers as a source to link to when they don't want to explain a term that they are using.) Frischauf vastly overrates the probable utility of a medical wikipedia when he uses an example of one article on avian flu being likely to replace 50 articles, and, by extension, implying that most of the articles on a medical Wikipedia would ever allow one article to replace many. Wikipedias, by their very nature, tend to be far too generalist for that.
Now that I've played the role of a the curmudgeon and given all sorts of reasons why I think a medical wikipedia is a problematic idea, let me just say that I'm quite aware that a lot of the same criticisms were leveled at the original Wikipedia, and it seems to be doing pretty well. Indeed, there are studies that show it having a similar error rate to more established sources, like encyclopedias. Although I remain skeptical that the wikipedia concept can be so easily translated to the medical realm in a manner that is accurate and useful for patients, physicians, and medical scientists, I for one would certainly be willing to give it a shot, if only because it's an idea whose implementation is probably inevitable now that the concept has started to be discussed in the medical blogosphere. Someone is going to set one up, regardless of what one slightly popular medical/scientific blogger says.
Of course, if I ever signed on to be a medical Wikipedian, I'd probably have to give up blogging to do it, and I certainly don't want to do that any time in the foreseeable future.
More like this
I am a somewhat active wikipedia participant, both in the English and the Danish version, and I must admit that I feel that people have a glorified view of how good wikipedias really are. The problem with them, is that an unqualified participant counts as much as a qualified participant, as long as the other participants can't see the difference. Quarks and alties can cite huge amounts of articles which they claim support their opinions, and to the layman it might look like their claims are accurate.
The fewer active users the bigger the problem - this I can see by comparing the Danish and the English wikipedia - it seems to me, that I spend most of my time complaining about the quality of the articles at the Danish wikipedia, since they are quite frankly often quite bad (even when it comes to fairly well known subjects like William Shakespeare). I do try to contribute positively as well, but there is just too much work that needs to be done.
I would think the same would be the case of a medical wikipedia.
I don't think a medical Wiki would be good because there are a lot of people who would use it to promote their own agendas. Like Orek says it would be pretty much impossible to keep up on it and people who sell herbs and other Alt therapies are very persistant. Since the internet more and more people already try to diagnose themselves and with all the advertisements actually tell doctors how to treat them. This is a problem because drugs are not a panacea to all that ails us. So I do not believe a freely edited Wiki would be a good idea. Plus I do not think Wiki's are really respected sources. As a student I know my professors would not accept a Wiki as a reputable source. It could be used in addition to other peer reviewed sources in respectable journals but not by itself.
As a patient who monitors cancer support boards, I would agree that combating the alties and dubious practicioners is a constant battle-- they are like mosquitos in the rainforest!! The same stuff comes up repeatedly (requiring constant swats), sugar feeds cancer, take x y or z because it worked in vitro-- take low dose naltrexone because this practicioner recommends it ( http://www.lowdosenaltrexone.org/ ) etc etc.... My brother's an oncologist and my husbands an oncology researcher-- neither of them have the time or inclination to help me much when I swat at them. With the wickopedia concept-- the alties do find dubious practicioners to go to, and, as you say-- how would they be kept off-- seriously-- these guys dont stop and most lay folks arent prone to critical thinking!! But, even without a medical wickopedia, there are endless places to find the nonsense and misinterpretations of abstracts from sources like pub med-- I do believe these guys would infiltrate a medical wickopedia and controls would be difficult.
Interestingly enough, on Wikipedia under a chemical my professor co-discovered for a certain possible treatment, his name was not listed. Instead, one of his former colleagues (who had previously lost a suit for claiming as his own my professor's ideas and research on this subject) was referenced. I added the reference to my professor's research, citing it as "earlier." But who knows how long that big oversight would have existed had I not done a search on a whim. I am skeptical of a medical wiki, too.
I am speculating that the final solution will be someting in-between: the format, software and speed of a Wiki, but edited and reviewed by (semi)professional staff, almost like an open-source online journal, except it is not all new papers all the time, but review articles being updates continuously And there would notbe just one MedWiki, but a whole set, with a different 'editorial board' for each subspecialty.
I've been toying around with the idea of making a much more narrowly focused Wikipedia, just for Chronobiology and to invite only members of the SRBR to participate in editing, while everyone else would be welcome on an associated forum or blog.
Might there be something to keeping the "fight" against alties in one central location?
One of the big problems with pseudoscience is that it's like a big game of whack-a-mole; the same fallacy pops up in a new location with a fresh audience and an uncertain chance of rebuttal. But because it is the same nonsense repeated ad infinitum, might there be value to a common site that list those "criticisms" and the counterpoints to them?
It would take a very dedicated squad of administrators to keep a Mediwiki vandalism-free. Either that or extremely strict restrictions on who can create or edit articles.
The problem with Wikipedia "culture" as such is that expert participation is strongly discouraged by what passes for their community. It's direct democracy taken to the extreme; anything that smacks of elitism has to be taken down a few pegs.
I think that a successful medical wikipedia would be tightly linked to data, either directly (as images are linked in wikipdia now) or through something like pubmed/medline. Medical information is often in the form of assertions of risks and benefits, and a clearinghouse of statistical data in the form of a medical wikipedia would be of great benefit.
How often does a review of studies in the literature get rubbished because of methodology, data set size, etc? But to truly refute it requires a lot of work to gather the same and addiiontional information. A medical wikipedia can bring it all together. As for promoting one side over another, it would be very bad form to delete the data; people could just add their own analysis and spin.
Wikis are frequently used as documentation for software components, requiring registration for editing rights. However, this model requires someone with an vested interest in keeping the quality high (in this case the manufacturer of the component), who is willing to boot less qualified editors.
i'm wary of it for many of the same reasons you pointed out in the medicine and evolution series. I feel like we'd see a lot of horrible interpretation of excellent scientific results.
And I shudder to think of the pure crap we'll see in the psych section.
Yes, the problem with Wikipedia is that it only works in practice, and not in theory.
I, for one, would hate to see the Internet turn into a source of questionable scientific information.
Cute.
Actually, that's what we're talking about. We're arguing about whether it actually works all that well in practice, specifically whether it works well enough to "replace" all those articles that Peter Frischauf refers to as a primary source of information. My point is that I've been underwhelmed by at least half the articles that I've looked up on Wikipedia as it exists, and I'm aware of, for example, Holocaust deniers trying to hijack the entries on the Holocaust and World War II. So what makes us think a medical wikipedia would be any different or do any better than Wikipedia if anybody can edit entries? My main point is that it is being vastly oversold compared to what it would be likely to actually deliver.
Very cute.
You missed the point. Sure there's lots of dubious medical information out there. We don't need to add a new site that, because it's called a wikipedia, will likely be viewed by many Internet users as having more reliability and authority than those sites peddling dubious all that dubious scientific information but that will be prone to being hijacked by the very pseudoscientists and quacks who make parts of the Internet such a cesspool of misinformation about medicine.
Personally, I hope I'm wrong with my misgivings, because a medical wikipedia is probably inevitable.
Left Wing Fox, such a central store of de-bunking does exist for one branch of pseudoscience.
http://www.talkorigins.org/
It's pretty useful to have all the counters to creationists' tired old garbage collected in one place, as one can simply reference talk origins and say "please do your homework and stop repeating noise from 30 years ago".
I don't know how much effect it's had on the prevalence of creationism. Probably not much.
I think I agree with Orac: a medical Wiki is probably inevitable, once it exists it will almost certainly come to be relied upon by many people as a medical authority, but the real quality of articles and information available on the medical Wiki will be highly variable unless somebody comes up with some way of eliminating the trash.
Why would one need a seperate wiki for medicine, when there is already a pretty decent repository of information for consumers at:
A better resource than Google!
argh... the link got mucked up:
Medlineplus
Funny,
I always looked at Wiki as a long term dephi experiment.
You may remember in John Brunner's 'Shockwave Rider' the concept of Delphi boards. It was not new to Brunner. I was intrigued with the idea and looked into it.
Apparently the hypothesis was that whereas an individual can only know a fraction of the details of a subject, as a whole the population knows a great deal about a subject. So, when making predictions about future market trends, the best solution is to ask a huge number of experts, compile their answers, and you would get a pretty good prediction. Thus delphi from the delphic oracle (not the automotive parts supplier). In practise, the results have not reached the level of accuracy the theory predicted.
Wikipedia was the same idea. Allowing everyone to include their knowledge should result in a collection of knowledge far greater than any individual's knowledge. Great in theory. Not quite so good in practise where people actively argue about what consititutes accurate knowledge.
However, as an experiment in human social behavior, there should be some very interesting data emerging from tracking which articles which are changed and in what way. (The identity of the people making the changes is probably irrelevant.) Make me wish I had the time to follow up on this notion.
I like the idea of a repository of notions, hypotheticals and tests for the same. Not just for medicine but for all science. This would not be a Wiki, but instead a tool to inspire researchers and improve testing protocols. I wouldn't expect a database of this sort to include results, just concepts. For, as the benefits of first publication are more than just bragging rights, the competition to be first publisher will continue to result in secrecy. Of course, as I am only on the fringes of science, this may exist already.
Cheers,
-Flex
Trackback (the usual mechanism failed).
It is true that noone knows everything so the idea that having a bunch contribute seems plausible. However, it doesn't seem to work because everyone has different interperetations. The problem I forsee with a Medical Wiki is that it would be a more respected forum (than the current ads masquerading as medical information) for the quacks to promote their ideas. I would prefer to leave it at the more reputable sources such as The Merck Manual and medline.
Flex's second idea to inspire reseach is also intersting but unlikely to happen. Noone seems to discuss current research on blogs now and I don't think a Wiki would be any different. Research is just to competitive. Why give the competition easy access to your ideas.
Seen this?
http://ganfyd.org/index.php?title=Main_Page
Nice to see you found us: at The free medical knowledge base that anyone can read and any registered medical practitioner may edit . A Medicine orientated encyclopaedia in wikimedia style that only registered doctors can edit. No real hassles in almost a years existence.
Hi all,
ganfyd is in its infancy (it will be one year old on the 15th November 2006), and as such is still very much like an encyclopedia with most of its pages missing. However, as an experiment it is working very well. Only time will tell whether it lives up to its potential, but I and many others have high hopes.
ganfyd manages to maintain credibility because only doctors can edit pages. Currently the headache of validating applicants means that the registration system can really only cope with docs from the UK, Australia and New Zealand, though members from other countries such as ROI, Canada and the USA can be added on an ad hoc basis. Once we can afford to look into this in greater detail we can open the gates properly to docs from other countries.
The aim of ganfyd was to provide a knowledge base that was free to all to read and free from commercial interests while being written only by doctors, hence maintaining credibility amongst peers. Of course everyone can make errors, but the aim is that edits and re-edits by subsequent contributors will act as a peer review and iron out anomalies and errors. This will of course take time.
Much work (and hard-earned cash!) has gone into building the foundations of ganfyd, and while there will be critics with perfectly reasonable arguments against, I think the project looks like it has a wonderful future, free from commercial interests.
Like I say - early days yet, but from little acorns...