How to write a thorough peer review?

Scientists do not receive enough peer-review training. To improve this situation, a small group of editors and I developed a peer-review workflow to guide reviewers in delivering useful and thorough analyses that can really help authors to improve their papers.

We suggest that you perform three readings of a paper, concentrating on a different element each time. At every point, remember to classify your comments as major or minor flaws. Major flaws will need considerable time to explain or correct.

The first reading is to get an overall impression of the paper and its aims. Take notes as you go. Make sure the paper is within the scope of the journal. It’s unlikely that it won’t be, but answering this question forces you to better understand the research and think about the paper’s mission: what it’s trying to achieve.

Take particular note of the parts of the paper that your expertise speaks best to. Editors do not expect you to be an expert in absolutely every aspect of the paper, although they also don’t want you to be a novice. Be upfront and honest with the authors and the editor about which scientific aspects you will focus on in your review.

After the first reading, attempt to ‘mirror’ the article by writing down, in detail, your understanding of the science. This tells the authors how you — the reader — have interpreted the aims, results and novelty of their research. If they disagree with your analysis, they should, from your comments, be able to understand that this is not your fault. Your analysis is a clear message that the authors need to work on how they communicate their intentions.

You might also notice a fatal flaw during your first reading. Forget minor or major flaws: a fatal flaw is one that stops the review process dead in its tracks. There’s not much point in continuing the process if the method is fatally flawed, if an entire section is missing or if the paper is utterly unreadable. Describe all flaws in your review, and submit it. Depending on the journal, you might have the option to ‘reject, but resubmit’. Otherwise, simply reject. There’s no shame in it!

If you don’t find a fatal flaw, continue to the second reading. Remember, you still need time, peace and quiet.

The second reading allows you to concentrate on the scientific nuts and bolts of the research: the method, analysis and conclusions. Remember to distinguish between major and minor issues, and to read the paper in chronological order. Ask the following questions:

• Do the Abstract and Introduction clearly identify the need for this research, and its relevance?

• Does the Methodology target the main question(s) appropriately?

• Are the Results presented clearly and logically, and are they justified by the data provided? Are the figures clear and fully described?

• Do the Conclusions justifiably respond to the main questions posed by the author(s) in the Introduction?

It is particularly important to ensure that the questions put forth in the Introduction are answered properly in the Conclusions. Try to spot anywhere the paper might start to take you on a wild goose chase. The paper should leave wild geese alone, unless it’s a submission to an ornithology journal.

At this point, it’s a good idea to take a few days away from the paper to step back from the details for a while.

During the third and final reading, you should concentrate on the writing and presentation. The science might be great, but heavy composition and messy structure might bog down the main message. If you comment on the writing, make sure you back up your comments. Don’t just note, “This is badly written.” Suggest to the authors how to make the story more cohesive and tightly reasoned. Was the paper hard to read because the paragraphs did not flow together? Did the authors flood the paper with confusing acronyms?

You do not need to copyedit a paper — that is generally the job of the journal that has asked you to review the article. But any suggestions for improving the language more generally will be welcome, and they are an important part of the peer-review process.

You should now have a list of comments and suggestions for a complete peer review. The full peer-review document can comprise the following sections:

1. Introduction: Mirror the article, state your expertise and whether the paper is publishable, or whether there are fatal flaws;

2. Major flaws;

3. Minor flaws;

4. Other, lesser suggestions and final comments.

Now, read your review carefully, and preferably aloud: if you stumble when reciting your own text, then readers will probably do the same. Reading it this way will also draw your attention to how your criticisms might sound to the ears of the authors. Ensure that your critiques are constructive and not offensive. Be helpful and not harmful. Sometimes, you should submit a harsh review, but never a rude one. Remember to adhere always to the “golden rule” of peer reviewing (M. A. McPeek, et al. Am. Naturalist 173, E155–E158; 2009): “review for others as you would have others review for you”.

View our full worksheet here.