Fixing Peer Review

Behind closed doors, virtually everyone is willing to say that the double-blind peer review process for journal articles is very badly broken. Yet it is still held up as the sine qua non of quality assurance, intellectual honesty, and academic integrity in our profession. To wit, the only articles that count in tenure reviews are those that appear in peer-reviewed journals. Book chapters, for example, regardless of who edited them or how thoroughly, don't count. If the primacy of the peer review is going to obtain, there need to be some changes to the process. Here are my suggestions for a few:

1) Allow simultaneous submission.

This is a no-brainer. If editors and reviewers know that they are competing for the best articles, the process will move more quickly. Scholars will have some options about where to place their work. And  those whose work is difficult to place because it falls between disciplinary boundaries or deals with unusual groupings of languages or types of texts won't have to gamble six months away at the mercy of a single journal perhaps seeing the wisdom in her work but perhaps erring on the side of intellectual conservatism. Particularly for people whose work is really, truly interdisciplinary this is especially important.

2) Commit to a piece at the "revise and resubmit" stage.

A decision of revise-and-resubmit ought to represent some kind of commitment on the editor's part to seeing an article through to publication. It should not be a guarantee of publication, but it should represent a grant of some time and space to work and of the benefit of the doubt. If, upon resubmission, the reviewers identify new and/or easily correctible issues, the author should have a narrow window in which to fix them, rather than facing automatic rejection.

My recent article rejection was especially frustrating because, after spending a ton of time making revisions based on the original reader reports, the article was rejected on the basis of five or six comments from one reader. All were new, and all were very superficial and easily fixable. Given how far along the article had gotten in the process and how non-fatal all of the issues were, the editor of the journal ought to have given me a chance to address those rather than rejecting the piece. (For example, the reader wanted me to invert the structure of a single paragraph so that my description of the text came before my review of the poem's historiography — something, by the by, that he had not asked me to do for that same paragraph in his original report, which rather foreshadows my next point...)

Again, I'm not saying that R-and-R should represent a commitment to publish, and I'm not saying that an editor should be obligated to work with an author who won't or can't get an article up to standard, but it should represent a commitment to work with the author a little bit to see the article through.

3) Clarify standards.

When a piece is revised, will it be returned to all the readers? To only one? If to all, and if some of the readers' opinions on changes required to the original, how will the difference of opinion be adjudicated? To new readers?

Again, drawing from my own experience: I added a methodologies section to the revised version of my article, in which I set my literary analysis into both medieval and modern poetic theory. For me it was really redundant and I resented having to write it, but Reviewer A had some specific questions about how I was analyzing the imagery of the poem and Reviewer C said he wasn't really clear about whether I was trying to write literary criticism or history. Yet it was Reviewer B, whom I believe to be the reviewer most closely in my own sub-sub-subfield (again, bringing up issues to be addressed in the next point), who was sole decider of the fate of the revised piece, and from his perspective, this new methodologies section "added nothing" to the piece. On the one hand, he was right. On the other hand, working on the assumption that the readership of the journal would be as as disciplinarily varied as the cross-section of the reviewers, he's wrong. But either way, it's up to the editor to have made this clear.

Had I known that the enthusiasm of a single reviewer was going to be the standard by which my article would ultimately have been judged, I would have revised in a somewhat different fashion. I know that fairness doesn't play into the academy or into the real world, but it's not totally fair to ask scholars to be shooting at an invisible moving target; but more to the point, it doesn't serve scholarship or intellectual endeavor at all.

4) Unmask the reviewers.

A decent amount of nastiness and sarcasm and other absurd and unnecessary rhetorical strategies factor into review reports that isn't warranted, necessary, or helpful. Review critically and deeply. Please! Really! But do it in such a way that you are willing to sign your name to it.

Furthermore, in small fields, the identity of the reviewers and the authors is already something of an open secret; no need to perpetuate a charade.

5) Make reader reports available online.

Again, this is a change that might best serve scholars who don't fit neatly into the scope of a single discipline and are likely to garner very different positive and negative reactions from scholars whose work does sit more neatly within a single camp. If reader reports were available, it might be possible for editors to publish more interdisciplinary scholarship that had, for example, garnered a certain type of praise and a certain type of criticism from a literary scholar and the diametrically opposite praise and criticism from a historian. It wouldn't hang the fate of interdisciplinary work on disciplinary concerns, would still allow for the peer review to have a place in the process, and could let journal readers have a greater insight into the debates within and between fields and ultimately make their own methodological judgment on the basis of a wealth of evidence.