There’s been a lot of calls for reform to the peer review process, and lots of blog posts about problems and bad experiences with peer review (Simply Statistics, SVPow, and this COPE report) . There is lots of evidence that peer review suffers from deficiencies related to author seniority, gender (although see Marsh et al, 2011), and from variability related to the choice of reviewers (see Peters & Ceci, 1982, but the age of this paper should be noted). Indeed, recent work by Thurner and Hanell (2011) and Squazzoni and Gandelli (2012) show how sensitive publication can be to the structure of the discipline (whether homogeneous or fragmented) and the intentions of the reviewers (whether they are competitive or collegial).
To my mind, one of the best, established models of peer review comes from the Copernicus journals of the European Geosciences Union. I’m actually surprised that these journals are rarely referenced in debates about reviewing practice. The journals offer two outlets, I’ve published with co-authors in Climate of the Past Discussions (here, here and here), the papers undergo open review by reviewers who may or may not remain anonymous (their choice), and then the response and revised paper goes to ‘print’ in Climate of the Past (still in review, here and here respectively).
This is the kind of open peer review that people have pushed by posting reviews on their blogs (I saw a post on twitter a couple weeks ago, bt can’t find the blog, if anyone has a reference please let me know). The question is, why not publish in journals that support the kind of open review you want? There are a number of models out there now, and I believe there is increasing acceptance of these models so we have choice, lets use it.
What inspired me to write the post though was my own recent experience as a reviewer. I just finished reviewing a fairly good paper that ultimately got rejected. When I received the editors notice I went to see what the other reviewer had said, only to find that the journal does not release other reviews. This was the first time this has happened to me and I was surprised.
I review for a number of reasons. It helps me give back to my disciplinary community, it keeps me up to date on new papers, it gives me an opportunity to deeply read and communicate science in a way that we don’t ordinarily undertake, and it helps me improve my own skills. The last point comes not only from my own activity, but from reading the reviews of others. If you want a stronger peer review process, having peers see one another’s reviews is helpful.