In this post I'm going to elaborate on a suggestion I saw a few days ago, in an article discussing the role of post-publication commentary in science. (And yes, I'm searching for the source of this idea - if any reader remembers whose it was, please point me to it.)
In modern but pre-internet days, researchers did the research, wrote the paper, submitted it to peer review, made changes, and published it. Other researchers them evaluated this information, using to guide their own work, and discussed its strengths and weaknesses when they cited it in their own papers.
Published papers were also discussed less formally with colleagues, both before and after publication, face-to-face and by mail and phone, and in journal-club presentations and seminars. The ideas from these discussions were incorporated into the formal papers drawing on this work, but they weren't available to anyone but the direct participants.
Now that we're all on line, published papers are also being discussed more publicly, in blogs and other places. Such discussions are extraordinarily valuable for the progress of science - they're written public evaluations, drawn from a wide range of expertise, and usually greatly enriched by comments from and links other researchers. But these pages are all over the place, and finding them requires a lot of active searching.
The Research Blogging site is helping with this problem, by aggregating blog posts that discuss individual research papers. But they can only link to posts that actively insert their code, and so miss quite a lot of the public commentary. So far the journals don't link their papers to this site, so readers who go looking for the paper don't usually think to also check Research Blogging.
A few forward-thinking online journals (PLoS and BMC groups, I'm talking about you) provide their own Comments thread for each paper, so other researchers can provide informal but public feedback . But the researchers don't use these, saying that they don't feel comfortable doing this publicly, or that they don't like the bother of having to register and log on. I know that's true for me, thought I don't know why - I'll happily blog about a paper I've read, but I almost never post comments on its official Comments page.
Sites like The Third Reviewer have tried to solve this by providing journal-independent sites where researchers can post comments about papers. But we won't use these either -the massive wave of discussion about the Wolfe-Simon paper on arsenic bacteria let to exactly zero comments on The Third Reviewer.
Most journals already provide, with each paper they've published, a list of links to the more recent papers that cite it. The suggestion I really liked was that the journals should also aggregate the informal commentary, by providing a separate list of links to ALL the web pages that have link to the paper. Journals could then stop fighting our unwillingness to post comments centrally, and just use our distributed posts to add value to the papers they publish.
I don't think this would be very difficult; lots of sites already have a 'Who links here?' feature, and I think both Google and Yahoo searches can be restricted to sites containing a specific url. The journals could include an explicit disclaimer that the journal in no way vouches for the value or creditability of the information in these links. A few bloggers might have to become a bit more circumspect (probably not a bad thing), and any blogger who didn't want their post linked to the paper could provide a citation to the paper but not link to it.
Jon Eisen, do you think this could work? If PLoS leads, I bet the others will follow.
Why are unfalsifiable beliefs so attractive?
1 day ago in Epiphenom