Why I don’t trust NewsTrust for health news

Darren Chase has an interesting post at medlibrarian.net about NewsTrust that I wanted to leave him a comment about, but the comments on his blog are turned off, so this post will have to suffice. Here’s what Darren posted:

Darren's post about NewsTrust

The part of Darren’s post that really caught my attention was this:

Overall, it succeeds in its goal to “identifying trusted news sources hidden in the deluge of information available online.”

If the site works like Digg (and it seems to), a high user score for an item indicates that a high number of users scored the item as good. So it would seem to measure popularity of an item among NewsTrust users. How does this indicate anything about authority?

From the site itself:

How do you pick which stories to feature on the NewsTrust site?
The content on the NewsTrust homepage is continually being update by story reviews and new submissions. Generally speaking, stories displaying a high level of journalistic quality as judged by our citizen reviewers, are more likely to appear on front page of the site.

So it would seem that health news stories are submitted to the site by “citizen reviewers” (with no apparent credentials or training in the health sciences) on the basis of “journalistic quality.” Then they are scored by popular vote by people who are also not health or information professionals.

Here are a couple stories I found in the Health News section, submitted to NewsTrust by these citizen reviewers “on the basis of journalistic quality:”

Could a medical librarian really recommend these as authoritative or trustworthy sources for health news? Could any librarian responsibly recommend NewsTrust as a useful tool for identifying trustworthy sources of information? What the heck is “journalistic quality”? The site doesn’t even list its criteria for evaluating a source, much less meet the standards of evaluation that should satisfy an information professional.

I think there are times when social models are not the best way of separating the the wheat from the chaff. I appreciate the “wisdom of crowds” as a concept and as a useful model for some applications, but the “wisdom” of a crowd of laypeople cannot reliably be used to identify trustworthy or authoritative sources of health information. That can only be done effectively by health professionals- most ideally by health professionals that are specially trained to be expert in finding and evaluating health information: Medical Librarians.

The World Wide Web is glutted with health information that is of poor quality, outdated, inaccurate, confusing, or just fraudulent. The need to help consumers navigate and find quality health information has never been greater. I’m concerned that a casual endorsement like this one at medlibrarian.net could seriously mislead a consumer who stumbled across it and misrepresent the value of medical librarians.

What do you think? Am I off-base in giving Darren a hard time about this? Please leave a comment and share your thoughts.

14 thoughts on “Why I don’t trust NewsTrust for health news

  1. I agree with you that social models (like NewsTrust) are not the best way of separating the wheat from the chaff. NewsTrust (like other social networking applications and services) is a tool–in the skilled hands of a professional it can be useful in identifying relevent, breaking healthcare news. Of course it goes without saying that NewsTrust, other social models, are not the only tools a medical librarian should use, nor that these take the place of our skills and expertise in identifying, providing quality health information.

    Medical librarians and other healthcare professionals have an important role to play in social models like NewsTrust, wikis, etc.–our participation that helps build collections of enriching, vetted news or content.

    Social models demand a discerning eye as well as participation. The 2 NewsTrust examples you provided in your post are not characteristic of the entire selection of Health news, rather these are exceptional examples you’ve chosen to illustrate the hazards of social models.

    The hazards of low-quality content and misinformation do not completely invalidate NewsTrust and other social networking sites and applications. Countering the “chaff” requires active participation from experts and professionals.

    I stand by my brief, casual endoresment of NewsTrust. At the same time I would not direct consumers to NewsTrust alone as a source of comprehensive health news. As medical librarians, we use the Web. NewsTrust is another site, among the vast array of sites, that could be useful to us–especially with our participation.

    Your post has really gotten me thinking critically about misrepresenting the value of medical librarians. Thanks, and by the way, I am not sure why commenting at medlibrarian.net wasn’t working–it should be now.

  2. Hey Darren, good to hear from you.

    First, I feel as though I must have failed to make this point:

    Popular does not equal authoritative. Popularity and Authority might sometimes coincide, but it is a massive mistake to confuse the two.

    I would never, therefore, recommend NewsTrust as a source of health news to a consumer for the exact same reasons I wouldn’t recommend Wikipedia as a source of healthcare information to a consumer. The crowd doesn’t have the knowledge to determine what is good/important/acurrate and what isn’t… and the stakes are too high when it comes to healthcare information to take the advice of laypeople.

    Your post didn’t recommend NewsTrust to Medical Librarians for use in the context of expertise and other tools, it just endorsed NewsTrust without context or conditions, and that’s part of the problem I have with it.

    Lastly, you wrote:

    “Medical librarians and other healthcare professionals have an important role to play in social models like NewsTrust, wikis, etc.–our participation that helps build collections of enriching, vetted news or content….Countering the “chaff” requires active participation from experts and professionals.”

    If the participation of a large number of experts (medical librarians) is required for the model to work, the model of the “wisdom of crowds” for health information is clearly flawed.

    Besides, even if NewsTrust could claim that 20% of its participants for health news stories were medical librarians, it STILL wouldn’t be a trustworthy, authoritative source- so I have to reject this argument. It seems to me that this is cheering for the social model without considering the practical and ethical questions.

    I would have left this comment at medlibrarian.net, but the comments still aren’t working.

    If you’d like to continue the conversation here, I’d enjoy that!

    Best,

    -David

  3. Hi David, comments at medlibrarian.net should be working now–sorry for the snafu.

    I regret that I didn’t qualify my support of NewsTrust in my original post; however, neither did I promote it as a source for authoritative consumer health information.

    Popular does not equal authoritative, and I do not make the mistake in my original post or response to your post of indicating that NewsTrust is a resource of authoritative health information. It’s a resource for news. It would be impossible for me to measure the amount of trust I have for any given thing. That’s not to say that my trust is completely unquantified, but rather that the nature of trust is imprecise, immaterial.

    I don’t accept any news at face value, but I am still interested in reading some of it, considering it. By opening reportage up to comment and evaluation, NewsTrust adds a component of accountability to Web journalism. Social networking provides the platform for dialog–I find dialog useful, though I accept that not all dialog/participation will be thoughtful. I simply don’t expect airtight perfection with social networking, but I do expect opportunities for exchange, sharing, feedback.

    The evaluation criteria on NewsTrust are not professional standards, but they are not purposeless.

    I have well considered the practical implications of the social model–the evidence of its dynamism includes our dialog on this blog.

    A medical librarian would–without difficulty–separate the wheat from the chaff on NewsTrust, and I recognize it to be a useful tool for health news/headlines.

    I’m really enjoying our dialog, thanks!

  4. Hi Darren-

    Thanks, I’m enjoying it too!

    You wrote “I do not make the mistake in my original post or response to your post of indicating that NewsTrust is a resource of authoritative health information. “

    Except that you did:

    Overall, it succeeds in its goal to “identifying trusted news sources hidden in the deluge of information available online.”

    You wrote: “neither did I promote it as a source for authoritative consumer health information.”

    Again, in the quote above, you said it successfully identified trustworthy bits. If you don’t qualify further, you ARE recommending it to any reader, whether medical librarian or Joe Googler.

    You wrote: “By opening reportage up to comment and evaluation, NewsTrust adds a component of accountability to Web journalism.”

    Darren, I have to admit that I’m completely confused by this. Users rate videos on YouTube. Does this add accountability to amateur video production? Does Digg add accountability to the stories it covers? I’d argue that the answer is “no” in both cases and that social evaluation has never claimed to add accountability. Peer review and editors do that. Besides, most online news sources have their own comment sections, email addresses, and forums.

    You wrote “The evaluation criteria on NewsTrust are not professional standards, but they are not purposeless.”

    How do you know? NewsTrust doesn’t DISPLAY its evaluation criteria for journalistic quality OR for those who are chosen to submit stories! If they’re not known, the only purpose they have is to feign authority…while actually having absolutely none.

    Nothing less than professional standards (journalistic, medical, or librarian) should be acceptable to a medical librarian if the material is to be used for professional purposes. Hey, if it’s just for kicks and giggles, any standards (or lack thereof) are fine. If, however, you recommended the service on a blog about medical librarianship, that makes it a whole ‘nother thing. Knowutimean?

    You wrote: “I have well considered the practical implications of the social model–the evidence of its dynamism includes our dialog on this blog.”

    Except I’m not questioning the “dynamism” of the social model. I’m questioning the ethics and practicality of applying it (or recommending its application) to the evaluation of health information!

    I like social applications. I think they can be fun and useful. But my point (which I apologize for repeating endlessly) is that recommending information evaluated by popularity is irresponsible for any librarian, but particularly irresponsible (and antithetical to how I understand the profession) when it comes to health information.

    Your turn! (We should take this show on the road!)

    Best,

    -David

  5. Oi! David!

    I expect that we agree that there is a difference between trustworthy news and authoritative health information, and that I am using NewsTrust for one (news) not the other (authoritative consumer health information). PubMed is one thing, NewsTrust is another. This doesn’t mean that PubMed is the best and only tool, or that NewsTrust is useless to the medical librarian.

    Accountability: social networking is the platform that supports our exchange for one–wherein you, I, and other participants are accountable for our content. NewsTrust is a neutral space, a the NYT social network comments and stuff are under their benevolent corporate control.

    You write: “Nothing less than professional standards (journalistic, medical, or librarian) should be acceptable to a medical librarian if the material is to be used for professional purposes. Hey, if it’s just for kicks and giggles, any standards (or lack thereof) are fine. If, however, you recommended the service on a blog about medical librarianship, that makes it a whole ‘nother thing. Knowutimean?”

    I know what you mean, and I disagree. In addition to standards, a professional’s actions, decisions, and other purposeful activity are informed by their professional expertise, judgment, training, skills.

    The evaluation criteria I am referring to are displayed on the NewsTrust article page.

    Hey, I am all for taking this on the road–energized discussion! Best regards.

  6. Oy! Darren! 🙂

    You wrote: “I expect that we agree that there is a difference between trustworthy news and authoritative health information, and that I am using NewsTrust for one (news) not the other (authoritative consumer health information). PubMed is one thing, NewsTrust is another.”

    Not really. We need to care about authority in health information, whether that information is destined for a consumer or a physician, and whether that information comes from JAMA or MedlinePlus. Health news is consumer health information!

    You wrote: “Accountability: social networking is the platform that supports our exchange for one–wherein you, I, and other participants are accountable for our content. NewsTrust is a neutral space, a the NYT social network comments and stuff are under their benevolent corporate control.”

    And I’ll answer again that accountability just doesn’t figure into it. Digg doesn’t add accountability to the stories or sources it covers. Besides, by that argument, this blog isn’t a neutral space. I could (though of course I won’t) delete or alter any of your comments on my own blog.

    You wrote: “In addition to standards, a professional’s actions, decisions, and other purposeful activity are informed by their professional expertise, judgment, training, skills.”

    That seems to me a bit like an evasion. The medical librarian uses the expertise, judgment, training and skills to find and evaluate health information. If the information has no authority (read: doesn’t meet any professional standards), it isn’t passed on to the patron. Period.

    Medical librarians can evaluate information for authority. “Crowds” cannot.

    You wrote: “The evaluation criteria I am referring to are displayed on the NewsTrust article page.”

    But…they’re not. Seriously, Darren- they don’t exist. Prove me wrong, though. Go to NewsTrust, copy the standards for selection of “citizen reviewers” and stories and paste them here.

    Bottom line: We need to evaluate sources for authority. If sources don’t have authority and we still pass them on to our patrons (consumers OR health professionals), I don’t think we’re doing a good job.

  7. Hi David,

    I note that we agree that it’s the medical librarian’s professional responsibility to evaluate information sources for authority. NewsTrust is a source of sources, and I would be foolish to discount the value of a news item (like this one: Health insurers deny policies in some jobs -Los Angeles Times – By Lisa Girion) simply because it is featured in NewsTrust and the authority of the persons who recommended it and evaluated it can’t be verified using professional standards. Medical librarians don’t need professional standards to read news.

    The information professional understands the differences between news and scholarly articles, peer-reviewed journals, evidence-based care sheets, blog posts, etc.

    You make a good point about your blog not being a neutral space, but it is a space that supports exchange. NewsTrust supports exchange and is neutral. It is an open, unbiased (though participants may share their biases) platform for critically evaluating and commenting on news, free of control and censorship. As an information professional, I recognize the value in that model.

    I have consistently found good health news at NewsTrust. Yes, we should always evaluate sources for authority–do you assume that I am unaware of that, or that by endorsing NewsTrust for good health news that I don’t know the difference between news and peer-reviewed scholarship?

    You write: “The medical librarian uses the expertise, judgment, training and skills to find and evaluate health information. If the information has no authority (read: doesn’t meet any professional standards), it isn’t passed on to the patron. Period.”

    It is hard to imagine that you can’t appreciate the difference between kinds of information–not solely their authority–and the role that they might play in serving the patron. Explain what damage the patron suffers if I share the news article cited above from the LA Times about health insurers denying policies to some jobs? It’s news, insightful, well-reported health care news–not an EBM fact sheet, not a scholarly article.

    We should care about authority, quite right. No medical librarian expects a cat to be a hippopotamus, or a news article to serve the same information need or meet the same rigorous standards as a peer-reviewed journal article.

    These are the criteria via NewsTrust I referred to:

    Balance
    Context
    Evidence
    Fairness
    Importance
    Information
    Sources
    Style

  8. This will be my last comment on the matter, as I think we’re starting to go in circles, and perhaps talk past each other.

    Darren wrote: “I would be foolish to discount the value of a news item…because it is featured in NewsTrust”

    This is a straw man. I didn’t say stories should be discounted because they appeared in NewsTrust, but that appearance in NewTrust did not indicate trustworthiness/authority. My point of contention is that Darren’s post indicated inclusion in NewsTrust did indicate trustworthiness.

    Darren wrote: “The information professional understands the differences between news and scholarly articles, peer-reviewed journals, evidence-based care sheets, blog posts, etc….It is hard to imagine that you can’t appreciate the difference between kinds of information…No medical librarian expects a cat to be a hippopotamus, or a news article to serve the same information need or meet the same rigorous standards as a peer-reviewed journal article.”

    Another straw man, and this one is sort of insulting. Here, Darren implies that I don’t appreciate the difference between various types of sources. Of course I do. Where we appear to differ is that I believe that professional scrutiny needs to be applied to any and all sources. Darren seems to insist that the wisdom of crowds can do that for us.

    Darren wrote: “Explain what damage the patron suffers if I share the news article cited above from the LA Times about health insurers denying policies to some jobs? It’s news, insightful, well-reported health care news–not an EBM fact sheet, not a scholarly article.”

    A third straw man. I have never said that a patron suffers for having received an article from a reputable mainstream news source, and have never indicated that anything that wasn’t scholarly wasn’t authoritative.

    The patron suffers when he/she is ill-advised to accept an article as trustworthy/authoritative because NewsTrust rates it as such. It is a disservice similar to recommending Wikipedia for healthcare information. It is unprofessional and irresponsible.

    Lastly, I have scoured the NewsTrust site for these and how they are defined…and still can’t find them. Without definitions, they aren’t standards, they’re relative scales on which to score.

    It has been an interesting debate, but I’m calling it quits. I remain disappointed that Darren hasn’t corrected his original post (which he himself admitted lacks necessary qualification).

  9. David,
    This is such a can of worms, and I agree with what I think your intentions are. Journalism in health is often iffy at best, with completely incorrect or overhyped interpretations of research findings. For NewsTrust to include a health section, and claim that the listed articles represent quality journalism, is likely misleading. So people think the article is interesting – that doesn’t mean the reporter got it right. I think the difference is in how it’s framed – is this one of a million sites where I could pick up news feeds for information that may or may not be 100% correct, or does it make claims on the quality of the information presented? If the latter, I’m skeptical of the validity of the claim and would not recommend the resource on principle.

  10. I think that a lot of assumptions are being made about users in this discussion. In his first comment Darren describes NewsTrust by writing:

    “NewsTrust (like other social networking applications and services) is a tool–in the skilled hands of a professional it can be useful in identifying relevent, breaking healthcare news.”

    I think the main problem is that when a medical librarian casually endorses NewsTrust they are, in a sense, giving the green light to a tool that will most likely not be used by the hands of a skilled professional, but rather by the general public searching for health information.

    Having taught Mini Med School for several years, I have been in a room where the general public comprised mostly of senior citizens and middle-aged adults have point blank told me that when looking for health information they use Google and trust the results they get. A fairly recent Pew Study on Internet Use statistically proved the same point. Most of the people who look for health information probably fit into one of several categories: students, those who are sick, those who are caring for sick loved ones, senior citizens, those who take medication(s), and health care professionals. I am sure I am leaving some people out, but speaking anecdotally this seems to be the breakdown.

    I have taught first year PT, nursing and medical students – many of whom were in their mid 20s and knew little to nothing about blogging, RSS, news readers, and other 2.0 tools. They may have heard the terms or read blogs themselves, but most did not have newsreader accounts or subscriptions (I know this since one of my last lessons was to have them set up Bloglines accounts to monitor PubMed search results) and none were really blogging or had a web presence beyond a MySpace account.

    I think we incorrectly assume that our patrons know how to evaluate the health information they find on the web. Almost all of my mini med school classes had no idea that information returned in a results list can be influenced by sites that pay to have their sites listed before others. In this day and age of “Hot or Not,” “Rate My Kitty,” and adding people to their friends list on MySpace, people have a hard time distinguishng between “popular” and “authoritative” and that one does not equal the other.

    I like to think of myself as someone who embraces technology and change. My feelings about NewsTrust are not born out of FUD (fear, uncertainty and doubt – defined for David’s benefit – don’t know if you’ve read that article). Well actually, maybe they are: I Fear that we are sometimes blinded by the shininess of some of these 2.0 tools; I am Uncertain of the educational and informational value of a site like NewsTrust; and I highly Doubt that I would ever mention it in an info literacy class other than as an example of a site where one needs to use their evaluative skills while reading.

    NewsTrust seems to be a good place to get an idea of popular news articles. As far as getting health news from it, well there are already other sites that provide this service in a more peer-reviewed and edited framework. The MLA endorses a “Top Ten” Most Useful Websites list – all of which provide consumer health information and current health news.

    I have looked at the site and can’t find any of their criteria for reviewing or rating stories. Perhaps it is in the member area of the site and I did not create and account so I wouldn’t know. The site does state:

    “Our research studies suggest that citizen reviewers using our review tools can evaluate news quality reliably – and as effectively as experienced journalists.”

    In the case of health information and news, I find that hard to believe. Unless you have a health sciences background, know a thing or two about statistics and qualitative versus quantitative studies, or have an understanding of how to evaluate the best evidence, I don’t think that citizen journalists can evaluate the quality of the information presented. As Rachel stated above, people think the article is interesting and that doesn’t mean the reporter got the story straight.

    I agree with Rachel’s last statement: if the site is making claims on the quality of the information presented then I wouldn’t recommend this resource. If it is just serving as a place where citizen journalists can review and rate news they feel is important, then I would just add it to my news reader.

    But what do I know, I’m not a medical librarian anymore 😛

  11. I am probably thinking about this way too much, but I can’t help. I think there is another point that is linked to this that is fodder for a separate post. That being: what is the responsible that we/you have as medical librarians?

    I think as medical librarians we (and I am just gonna use we – I haven’t not been a med librarian that long!) sometimes easily forget that there is a very real clinical component to our work. Academic medical librarians especially may find it very easy to forget that there are patients several floors above or in an adjacent building, whose doctors may base clinical care decisions on information that we provide. For every student I saw at SBU’s HSL, there were plenty of physicians and interns who walked in and said, “my attending/resident told me to find out…” which basically means: “we have a patient upstairs that we are treating and need information.”

    Keeping this in mind, what is our responsibility as medical librarians? I joke around a lot and tell people, “I work in a library, it’s not brain surgery.” Well sometimes it might be.

    Clinicians really enjoy tools like Access Medicine and UptoDate because they help them make clinical decisions and they can trust that the information has undergone some sort of peer review and editorial process.
    They look to us to guide them towards information and tools that they can trust. To do anything less is, IMO, irresponsible.

    So I guess my point is that every recommendation, no matter how casual, does count in some way, shape or form when it comes out of the mouth of a medical librarian.

  12. Consider sites where questions are answered by clinicians as another way to vette information, rather than going thru medical librarians… such as medhelp.org