Nature Medicine on Wikis (edited)

Update: Don’t miss this update which shows AskDrWiki taking seriously critiques from libraryfolk and using them to improve AskDrWiki’s authority.

(Also be sure to check out the List of Medical Wikis)
________________________________

WikiMedia
Brandon Keim
SUMMARY: Uneasy with information websites policed by people with little expertise, scientists are creating their own online encyclopedias. But are they accessible enough to supplant Wikipedia,
News@Nature 13, 231 – 233 (01 Mar 2007) News

Full Text | PDF
Subscription required

Best part of the article: Great comment from Dean Giustini.

“The Wikipedia model is simplistic for medicine,” says Dean Giustini, a University of British Columbia medical librarian. “There’s something to be said for the wisdom of the crowd, but that doesn’t supplant the wisdom of the expert.”

Worst part of the article: Lousy caption for a graphic.

Second opinions: Websites run by medical experts, such as Ask Dr Wiki, may be accurate, but are often written in language inaccessible to the average user.

First, Not all Wikis are intended for broad, populist contribution and consumption. That’s Wikipedia’s model. Specialized disciplines use specialized language. The caption seems to imply that the use of specialized language is a flaw, when in fact it is a feature. This is especially weird because the author seems to understand this in the rest of the article.

Second, Ask Dr. Wiki may be accurate, but given that they don’t have the necessary controls in place to make sure that only accurate information is posted, it just as well may NOT be accurate. The caption says Ask Dr. Wiki is “run by medical experts,” which implies that the content is expert. The text of the article calls Ask Dr. Wiki “a doctors-only wiki similar to Ganfyd.” Here’s the thing: it isn’t. Unlike Ganfyd, they don’t even try to restrict access to physicians-only.

It took me all of two minutes to register an account and add the word “poop” to the Gastroenterology page.
Poop on Ask Dr. Wiki

Juvenile? Yeah, but it makes the point. Medical information is not like other kinds of information. If you get Stalin’s birthday wrong from a bad edit in Wikipedia, no-one gets hurt. Wrong information in a health information resource, whether for health professionals or consumers, can do great harm.

Because Ask Dr. Wiki tries to follow the Wikipedia model, it was a failure the moment it started. It can be at least partially redeemed, but only if it’s administrators actually start restricting contributions to qualified clinicians and have some sort of editorial oversight and/or peer review.

Near as I can tell, PubDrug is doing it right, and Ganfyd is at least trying. The rest of the medical Wikis are not to be trusted. Edit: It is clearer and more accurate to say that a Wiki for medical information that does not have these controls should not be trusted by any responsible health professional or healthcare consumer. I can’t claim to have critically examined (or even identified) all medical wikis.

As Dean has said, there’s nothing wrong with the Wiki technology for medical information, but the model we’re accustomed to from Wikipedia will not suffice.

Other posts on Wikis for health information

First PubDrug Monographs Available

PubDrug Updates

OVER!MY!MED!BODY! on Medical Wikis

Lots More Medical Wikis

Medical Wikis

PubDrug

Nursing Wiki

Alternately, you might like to see the list of ALL posts about Wikis

(Thanks to Dean Giustini for pointing this article out to me).

12 thoughts on “Nature Medicine on Wikis (edited)

  1. David,
    As you mention in your article Wikis can be vandalized but since we have started we have only had 2 vandals. Both were bloggers who posted blogs about their acts within hours of vandalizing the site. Ask Dr. Wiki uses RSS feeds to alert the editors when pages are changed and all edits are approved or rejected within 24 hours and usually less. Your blog was posted at 2:21 am and your entry was erased the same morning and since you were considered a vandal your user name was blocked. If needed I can block your IP address. I assure you that if anyone post anything that we consider inaccurate information it will be deleted within 24 hours. We have been happy with the way we reviewed information on our wiki. If vandalism ever becomes a major issue we would consider requiring users to send us a copy of their medical license or proof of medical training, but to be honest with you it has not been a problem. We have had great success and if the wiki continues to grow we will have created a free up to date source of medical information that can ultimately help improve the care of patients throughout the World.

  2. Hi Ken- thanks for the comment.

    1. It took nearly 10 hours for my note to be erased. Imagine what could happen if the wrong information was posted for 10 hours and someone acted based on it in those 10 hours. It doesn’t matter that it was 2 AM in my hemisphere- someone could be using the Wiki on the other side of the planet and applying bad information to patient care.

    2. I didn’t say ANYTHING about vandalism. Vandalism isn’t the problem. The problem is that you don’t vette users before allowing them to post or content before you allow it to be posted.

    In order to have any semblance of authority, Ask Dr. Wiki absolutely must do at least what Ganfyd does (vette users) or, even better, what PubDrug does (vette content, too) to help ensure bogus/inaccurate information doesn’t get posted in the first place.

    Until then, I believe Ask Dr Wiki could potentially be a greater threat to health than it is an asset.

    I thought Graham expressed it well:
    http://www.grahamazon.com/2007/01/i-will-never-trust-dr-wiki/

    A Wiki for physicians is a great idea, but the model for medical Wikis needs to be more like Citizendium than Wikipedia.

  3. Ken,

    What if, on the anaphylaxis page, someone changes the dosage of epinephrine from micrograms to miligrams? Or just changes the decimal point?

    If/when the site gets popular, will editors have the time/resources/attention span to catch small but deadly details like that?

    I’m all for a consumer site that explains medical information in better detail–most patients are woefully ignorant about medicine–but presenting information as accurate, when it could have been changed, is scary. I agree with David–there need to be better controls, and information should really be reviewed before it’s presented publicly.

  4. Compelling article. Your argument and illustrating-it-with-poop (funny) graphic are persuasive, as is Dean’s excellent quote. Wiki is a great platform for decentralized online collaboration, but wide open, no holds barred, decentralized collaboration isn’t the right platform for all online information nor, as noted, is wiki without its mechanisms of control–it doesn’t have to be wide open. Expert medical information is not had by dialing a random number, and no matter how much I love submarines, flying one to the moon is a fantastic but dumb idea.

  5. David, Graham:

    A few thoughts. Forgive the length, but just as you are worried about the potential for a medical wiki, I am just as excited!

    I think it’s easy to criticize the technology when you may not see the bigger picture. Any time new technology is developed, there are always critics. Whether you choose to contribute, advance the technology or never use it is your decision.

    Graham, once you graduate from medical school and venture out into the real world, you will see that many of the ‘abundant’ medical resources we take for granted are no longer there. David, as a librarian, I’m sure that it seems that these resources are all around you. However, as a practitioner outside of the hallowed academic halls, I can safely tell you – they are not. For example, with the financial constraints affecting one of the hospitals I practice in I’ve seen the subscription for Up to Date and several online resources eliminated.

    The idea of a medical wiki is not designed to supplant 13 years of my medical education. Instead I hope it to be a resource to improve upon it. Hopefully it can grow into a community of like minded individuals creating a basic resource in medical education available to all levels of people in the medical field. As its contributors multiply, it will develop in scope gaining an ability to self police and misinformation or net vandalism is dealt with even faster than in David’s instance this morning. In fact, in direct response to Graham’s previous malicious posting, several safeguards were instituted in an effort to further prevent future instances.

    I also personally believe that some restriction of contributors is necessary, but where do you draw the line? As I hope you both realize by now, in the medical field there are many non-physicians who make significant contribution to expanding medical information and improving patient care. Whether it’s the nurse with observations on the bedside care of the patient, the pharmacist that can offer tips on compounding, or the medical student with enough time on their hands to research a topic in depth or offer tips to improve online medical technology. Should these people be restricted from contributing? What about the PhD in genetics or physiology? Their contributions I know could be more comprehensive than mine in some fields – so why restrict them from the onset? It’s easy to tighten the ropes once something is seen as a significant problem. It’s much harder to loosen them if potential important contributors are scared off by too strict a policy or intimidation by the technology.

    As someone who has published and written many medical chapters, reviews and articles I can safely tell you that there is just as much misinformation out there in ‘peer reviewed’ medical journals and text books. I would refer you to the case of “Dr. Jon Sudbo” and his articles published in The Lancet and New England Journal of Medicine. Luckily no one wrote a blog topic entitled “I will never trust The Lancet” In fact, our hope is that a constant review in real time by a medical community with vested interest in the website will allow for a whole new level of peer review. As pointed out in an separate Nature editorial, as the wiki grows – so does the accuracy (see http://www.nature.com/news/2007/070226/full/070226-6.html)

    I hope this fuels the discussion. I hope that you give second thought to the use of technology and embrace it in your education and in the care of your patients. It isn’t going away and will only improve in availability and scope by the time you are practicing.

    Best Wishes,

    Brian Jefferson

  6. Hi Brian- thanks for sharing these thoughtful notes. I’ll try to address each point you’ve made.

    Neither Graham nor I have criticized the technology of Wikis. Anyone who reads this blog knows I’m a big fan of Wikis. I’m pretty solidly on record as a fan of PubDrug and a cautious admirer of Ganfyd.

    What I’m opposed to is a medical Wiki intended for clinical reference (informing patient care) that has no quality control. I’m opposed to it because it is dangerous.

    Brian, I’m outside the hallowed halls of academia, too. I work in a small, purely clinical hospital library with limited resources. Trust me when I tell you that whatever pains you have felt as a result of these expenses are felt quite keenly in my library.

    Like Ken, you’ve incorrectly described my criticism as highlighting vandalism as a cause for concern. Again, malicious vandalism isn’t the problem. The problem is that contributors to AskDrWiki aren’t vetted, and the information added to AskDrWiki isn’t vetted.

    Sure, peer-reviewed, edited resources sometimes contain bad information, but at least this is minimized by the existence of an editorial or review process.

    I’m glad you agree that “some restriction of contributors is necessary,” and absolutely agree that the question of how to define and manage restrictions is a large and important one. Contrary to what you have implied, I have consistently advocated for contributors to be restricted to qualified medical professionals- I have never suggested limiting participation only to MDs.

    Graham and I have both expressed a concern that there is too little review and editorial oversight in AskDrWiki. In response, you have asserted that even WITH peer review and editorial oversight, bad information gets published. So if bad information gets published even WITH quality control measures in place, how much WORSE would the published information be without them?

    My worry is that we’ll find out if a medical Wiki is run on Wikipedia-type model rather than on a Citizendium-type model.

    Again, thanks so much for sharing your thoughts. I’m really delighted to be participating in a conversation with such engaged, articulate participants.

    Best,

    -David

  7. Pingback: woodyevans.com » wiki surgery update

  8. Pingback: davidrothman.net » Blog Archive » American Medical News on Medical Wikis

  9. The licence we wrote for Ganfyd may be worth your looking at. It is a model for professional information collaborative editing systems – wikis etc – deliberately written to solve some of the problems above.

  10. Hi Adrian-

    Do you have a link where I can see this license?

    Best,

    -David