Troubled Tuesday (Reactions to Marcus Monday)
Dean Giustini likes Marcus’ idea about replacing LIS journals with blogs (see yesterday’s post), but also has concerns:
…my only reservation is when research methods are used such as randomization and the articles would need to go through peer-review.
T. Scott (former editor of the JMLA and one of my favorite contrarians) explains some of his reservations about the idea:
I’m not one who is terribly impressed by the “wisdom of crowds” (a concept that seems to be especially dubious during the US election season). I’ve rarely seen anything approaching substantive discussion and analysis take place in a comment thread, and the longer the thread, the more worthless it typically is. Rather than providing vibrant post-publication review, I’m afraid that posting unedited articles for comment would result in much good work being buried and ignored.
[...snip...]
Marcus is pushing the right questions, and everyone involved in scholarly publishing, at whatever level, should be thinking creatively about how to make the communication and discussion of projects and ideas more effective.
I’ve noticed my neck often aches after reading T. Scott’s blog. After some investigation, I’ve finally figured out that this is because I can’t seem to stop nodding my head while I read him.
I especially like this last bit of his post:
But it isn’t a matter of journals vs blogs. The most effective modes of communication that we develop over the next decade will adopt features that we associate with each, but will be fundamentally different from either.
I couldn’t agree more. I don’t think that blogs or wikis are going to revolutionize medicine, education, or publishing- but some applications of their descendent technologies might.
Since LIS News and LibraryStuff both posted about it, I’m betting there’ll be more interesting opinions to enjoy soon.
Like this post? Subscribe to the RSS feed!
February 12th, 2008 at 4:09 pm
see the post at MLA’s Task Force on Social Networking Software blog to read about an author posting chapters of a book on his blog, using the comments as peer-review: http://sns.mlanet.org/blog/2008/01/28/blog-based-peer-review/.
February 17th, 2008 at 12:59 pm
Thanks for the round-up of reactions David; I’ve been oblivious these past several days!
Regards,
Marcus
February 23rd, 2008 at 12:20 pm
This is going to be a long one, and I apologize for that in advance. I tend to agree with T. Scott on these matters, for several reasons. I don’t see any reason why librarianship journals as blogs should be singled out as a specialty (goose, gander, etc.) in this discussion, so I’ll talk about this more generally.
1) I believe there is value in having a final version of a manuscript on the record. Getting things out quickly isn’t the only goal in publishing a paper, or shouldn’t be. A larger goal is to contribute to the body of knowledge on a topic, in a way that can be cited and referred to and built upon in the future. If the idea is that authors would post their work to a blog and solicit comments, presumably that manuscript is in a constant revisional state, forever and ever, unless the authors finally shut down comments (at what point is the amount of critique enough?) and post a “final” version. This is perhaps more important when large flaws are detected, and it is nice to have a record of the final version of the manuscript. When citing it, are you citing the final version, or some version in which the author tweaked something in the post? If you cite it, will changes happen later that render your reference irrelevant? I think this process would make it harder to talk about what a given author said or did, and it also puts a tremendous amount of trust in the authors not to change things in a way that is dishonest or unethical. With an official “final” version, the author is officially on the record, and I think that’s an important concept.
2)”The argument for pre-publication peer review is that it filters out poor research.” Marcus seems to believe that this isn’t an issue for library research, or at least that the stakes aren’t high enough to matter. I would ask whether librarians seeking tenure and professional respect are really willing to hang themselves out there like this, simply assuming that what they’ve done is good enough for public consumption. Like Scott, I believe this simply isn’t true. Librarians are perfectly capable of producing research that, in its initial state, would be ripped to shreds by competent reviewers. I suspect that many would prefer that critique to happen more privately, to give them a chance to rethink their assumptions and presentation before opening themselves up to professional criticism and embarrasment.
3) Peer review takes work. When a committed board of peer reviewers exists with a demonstrated interest in the process and a deadline for providing feedback, and an editor does the work Scott mentions prior to publication, it is a certainty that an author will receive feedback. Blog comments are an unreliable thing. Commenters may never hit on the one true major flaw of a manuscript, may not have the expertise to critique to manuscript, or may simply not have time to digest a full manuscript in its raw form and suggest all of the appropriate revisions. The manuscript and the professional body of knowledge may suffer from this, as its not just the shiny, catchy papers that need feedback and critique.
4) Related to #3, it would be important to determine whether a manuscript was just open to whoever felt like commenting (or not), or if peer reviewers would be assigned drop by and comment. Would they be allowed to do so anonymously? Could an editor comment anonymously? If not, would the editor continue to make the needed comments Scott mentions about organization and content? I know some have advocated for peer review that is not anonymous, but I suspect that harsher, yet needed, criticisms might be held back if they had to be publicly written with a name and IP attached.
I’m not saying it couldn’t be done. These are just a handful of issues I see as barriers that would have to be considered. Ultimately, I think part of the question is whether we’re so determined as authors to put our unfiltered thoughts out there as fast as possible, or whether we’re really interested in being accountable and on the record and contributing to the professional knowledge base in a substantial way, even if it takes a little longer. I want to think about this a little longer.