I enjoyed an excellent meeting with Annette Thomas, CEO of MacMillian Publishing and Nature Publishing group at UC Davis. It was impressive to here that Annette wanted to meet with UC Davis faculty and graduate students interested in open access and open science to discuss our opinions, desires and frustrations in the scientific publishing process. It was particularly valuable to discuss some of the failed experiments in open science at Nature. Following this event I sent the letter below to Annette recording my reflections on those experiments and what I think they might have revealed. The 2 hour discussion with four graduate students, two faculty members and members of UC Davis and Cal Digital Libraries hit on many other topics as well, and these reflections capture just a few of the issues discussed.
Yesterday we discussed the failure of the 2004 experiment in open peer review and the failure of the pre-print server, Nature Precedings. In my humble opinion, both of these were doomed by insufficient incentives to participate.
A more successful model to open up peer review has been that practiced by EMBO, as discussed in a Nature Comment in 2010 (doi: 10.1038/468029a). As we are already doing the work of peer review, their is an immediate incentive to get credit for that work through this mechanism, and more journals should consider it. (Biology Direct has a slightly more extreme take on this policy. Both have been far more successful than either the Nature experiment, comments sections of PLoS, or F1000 approaches to post-publication peer review).
Precedings was a bold and powerful idea that addressed many real gaps in the current publishing infrastructure – a preprint server for fields not represented in arXiv, support for posters and presentations, dois (okay, EZID handles; crossref DOIs might have been better), and a more modern interface than arXiv provided. I was sad to see this close. As you both surely know, preprints run strongly counter to the grain of mainstream biology and their concept of precedence (unlike physics, economics and other fields where the opposite is true). As recent genomic examples have illustrated, the value of faster communication and a broader pre-publication review may slowly shift that culture, but journals such as Nature could play a key role in easing the transition, as your 2004 experiment showed.
What is needed here is a button to submit work to the preprint server integrated into the normal submission form to the journal. This will dispel the persistent myth that the journal is not okay with it, and will meet with much larger uptake than simply providing the server separately would do. I would bet your own data from the 2004 experiment shows this pattern – more authors choose to participate in that, as a fraction of the total submissions, than have independently uploaded work to a preprint server. As you both observed, the failure of the 2004 experiment was in the breadth and depth of comments received, but as a study in willingness to use a preprint, it was an important positive discovery about promoting preprint culture that would only strengthen the final journal copies.