1) I believe there is value in having a final version of a manuscript on the record. Getting things out quickly isn’t the only goal in publishing a paper, or shouldn’t be. A larger goal is to contribute to the body of knowledge on a topic, in a way that can be cited and referred to and built upon in the future. If the idea is that authors would post their work to a blog and solicit comments, presumably that manuscript is in a constant revisional state, forever and ever, unless the authors finally shut down comments (at what point is the amount of critique enough?) and post a “final” version. This is perhaps more important when large flaws are detected, and it is nice to have a record of the final version of the manuscript. When citing it, are you citing the final version, or some version in which the author tweaked something in the post? If you cite it, will changes happen later that render your reference irrelevant? I think this process would make it harder to talk about what a given author said or did, and it also puts a tremendous amount of trust in the authors not to change things in a way that is dishonest or unethical. With an official “final” version, the author is officially on the record, and I think that’s an important concept.

2)”The argument for pre-publication peer review is that it filters out poor research.” Marcus seems to believe that this isn’t an issue for library research, or at least that the stakes aren’t high enough to matter. I would ask whether librarians seeking tenure and professional respect are really willing to hang themselves out there like this, simply assuming that what they’ve done is good enough for public consumption. Like Scott, I believe this simply isn’t true. Librarians are perfectly capable of producing research that, in its initial state, would be ripped to shreds by competent reviewers. I suspect that many would prefer that critique to happen more privately, to give them a chance to rethink their assumptions and presentation before opening themselves up to professional criticism and embarrasment.

3) Peer review takes work. When a committed board of peer reviewers exists with a demonstrated interest in the process and a deadline for providing feedback, and an editor does the work Scott mentions prior to publication, it is a certainty that an author will receive feedback. Blog comments are an unreliable thing. Commenters may never hit on the one true major flaw of a manuscript, may not have the expertise to critique to manuscript, or may simply not have time to digest a full manuscript in its raw form and suggest all of the appropriate revisions. The manuscript and the professional body of knowledge may suffer from this, as its not just the shiny, catchy papers that need feedback and critique.

4) Related to #3, it would be important to determine whether a manuscript was just open to whoever felt like commenting (or not), or if peer reviewers would be assigned drop by and comment. Would they be allowed to do so anonymously? Could an editor comment anonymously? If not, would the editor continue to make the needed comments Scott mentions about organization and content? I know some have advocated for peer review that is not anonymous, but I suspect that harsher, yet needed, criticisms might be held back if they had to be publicly written with a name and IP attached.

I’m not saying it couldn’t be done. These are just a handful of issues I see as barriers that would have to be considered. Ultimately, I think part of the question is whether we’re so determined as authors to put our unfiltered thoughts out there as fast as possible, or whether we’re really interested in being accountable and on the record and contributing to the professional knowledge base in a substantial way, even if it takes a little longer. I want to think about this a little longer.

Regards,
Marcus