13 February 2022

Peer review process

What is peer review?

  • A collaborative process that allows independent experts to evaluate and comment on manuscript submissions
  • Peer review gives authors feedback to improve their work & allows the editor to assess the paper’s suitability for publication

Kinds of peer review

  • Single-blind: reviewer’s name is not disclosed to the author(s)
  • Double-blind: neither identity of the reviewers and authors are disclosed
  • Open: both the author and reviewer names are disclosed
  • Post-publication open: editor mediates readers’ and reviewers’ comments after publication

Why do peer review?

  • Improves the quality of articles that are published
  • Provides an assessment of the science in the literature
  • Assists the editorial decision-making process
  • Screens for unethical practice

Why do peer review?

  • Keep abreast of latest research
  • Improve your own writing
  • Boost your career

Who does peer review?

Generally people with

  • many years of research experience in a subject area
  • in-depth study in a specific area during grad school
  • practical experience in the field
  • recently published articles on a related subject

How do I become a reviewer?

  • Contact the editor
  • Ask a senior colleague to recommend you
  • Look out for calls for reviewers
  • Find a mentor
  • Get on google spreadsheets of potential reviewer (via the Twitter)
  • Write a paper!

Who does what?

Editorial office

  • checks that article files are complete & journal guidelines were followed
  • provides central contact for all inquiries
  • gives essential feedback to all parties so that the publication experience is as straightforward as possible for authors and reviewers
  • ensures final copy for publication is complete & follows guidelines

Who does what?

Editor-in-chief (EIC)

  • manages the strategic direction of the journal
  • makes final decisions on articles based upon
  • aims and scope of the journal
  • associate editor recommendation
  • reviewer comments
  • other articles recently published in the journal
  • journal priorities
  • journal page budgets

Who does what?

Associate editor

  • makes an initial assessment of the article
  • selects appropriate reviewers
  • scrutinizes reviewers’ comments
  • provides their own assessment of the article & guidance on the importance of reviewer comments
  • judges merits of publishing the article in the targeted journal using the feedback from reviewers
  • makes recommendation to EIC regarding a final decision
  • promotes the journal within the scientific community

Who does what?

Reviewers

  • provide a detailed, objective report on the merits of an article
  • identify flaws in methods, analysis & interpretation of results
  • highlight ethical concerns
  • comment on appropriateness of the literature cited
  • comment on suitability for publication in the journal

How does the process work?

Best practices

Reviewer invitation

Things to consider:

  • Does the subject area of the article match my expertise?
  • Do I have time to do the review within the time frame requested (2-3 weeks)?
  • Do I have any conflicts of interest?
  • Do I actually want to review this article?
  • Can I commit the necessary time (it’s OK to ask for an extension)?

Best practices

Reviewer invitation

  • If declining an invitation, do so promptly

  • If possible, suggest alternate reviewers

In-class exercise

What would your response be to an email like this & why?

Dear Dr. Smith:

Pleased to be inviting you to review a manuscript titled “Changes in the age at maturation for two species of Cyprinid minnow raised in an aquaculture setting”, which has been submitted to The Journal of Fisheries Research. Given your extensive background in this field, your advice would be best appreciated.

Click here to accept.

Click here to decline.

Sincerely,
Maya Livingstone
Editor, MDPI Journals

Writing constructive reviews

Best practices

Basic principles

  • Always treat the paper with the utmost confidentiality
  • Always be professional, courteous & collegial
  • Take an objective, independent approach to the work
  • Be attentive & critical
  • Provide evidence for any statements you make
  • Use simple language
  • Never contact the authors directly

Best practices

Questions to consider

  • Does it fit the scope of the journal?
  • Would the paper be of interest to the journal’s readers?
  • Is the work novel (does it advance research)?
  • Are the methods complete & analyses properly conducted?
  • Are all citations & references complete?
  • Does the title reflect the contents & is it engaging?

Best practices

General comments

  • Begin with a summary of methods, results & implications
  • Provide high-level impressions & concerns
  • Offer both positive feedback & constructive criticism
  • Make suggestions on improving clarity, succinctness & quality of presentation

Best practices

Positive comments

The manuscript is well written in an engaging and lively style.

The content is appropriate to our readership.

The subject is timely & one to which the author has made significant contributions.

The authors have distilled a complex idea into tangible results.

The paper offers a useful overview of current research and policy.

Best practices

Constructive criticism

In the Discussion section I would have wished to see more information on…

I would strongly advise the authors to rewrite their introduction and discussion to provide greater context for the study.

There is an interesting finding in this research about… However, there is insufficient discussion of exactly what this finding means and its implications.

The paper would be significantly improved with the addition of more details about…

The abstract is very lengthy and goes into detailed accounts that are best suited for the article’s main discussion sections.

Best practices

Specific comments

  • Provide information to help the editor with decision & authors with revisions
  • Use page & line numbers to reference points of contention or agreement

Specific comments

Abstract or summary

  • Is it concisely written?
  • Does it provide a clear overview of the work?
  • Does it contain the essential facts from the paper?
  • Does the final point place the work in a broader context, highlighting its significance?

Specific comments

Introduction

  • Does it provide a clear, concise background to the study?
  • Can you understand the aims of the study & hypotheses/questions the authors are exploring?
  • Have the authors elaborated sufficiently on the context in which the work is set?
  • Has the motivation for the work been adequately explained?
  • Is there satisfactory citation of prior literature?

Specific comments

Materials & methods

  • Have the procedures followed been sufficiently described?
  • Is there enough detail here for the study to be replicated?
  • Is it clear what was recorded and which units of measurement were used?
  • Are the statistical design and analyses appropriate?
  • Have important details been left out?
  • Where appropriate, has ethical approval been obtained for the work?

Specific comments

Results

  • Are they provided in a form that is easy to interpret & understand?
  • Have results for all the questions asked been provided?
  • Are the data of sufficient quality & quantity to answer the questions?
  • Are the figures & tables appropriate?
  • Have the correct units of measurement been used?

Specific comments

Discussion

  • Have the authors answered their questions/hypotheses?
  • Are the conclusions drawn from the results justified?
  • Has the significance of the study been fully explained?
  • How much has the study advanced our current understanding of the science?

Best practices

Post review

  • Whenever possible, agree to review revisions or resubmissions of articles you have previously reviewed
  • Do not include anything that appears to be a decision about the paper in your comments to the authors
  • Most journals will provide the other reviewers’ comments, which help you see how different people review papers & issues that you may have missed
  • If you signed your review & a dispute arises about a decision or your comments, do not engage with the authors directly

In-class exercise

Take 15 minutes to read this abstract & provide feedback via this Discussion

Owing to a number of concerns about climate change, we set out to understand how fish respond to warming water temperature. Using a collection of mesocosms set out in a large field near our research station located in the central United States, we compared fish growth rates under ambient temperature compared to elevated temperatures indicative of a possible warming climate regime. This was important because fish will be even more important as a food source in the future as the number of humans on earth continues to grow and will ultimately exceed the carrying capacity of traditional land-based agricultural systems and evidence suggests already that aquaculture may be a necessary and readily available source of important protein. Our experimental results were compared to experimental controls using a 2-way ANOVA using post-hoc Tukey tests. We discovered that warmer temperatures of water in the tanks was pretty significant because our p-values were generally less than would be expected by chance alone, except in two cases where they were really close to 0.1, which is still pretty low but perhaps greater than some people have suggested is a critical threshold for statistical validity. Owing to the importance of our results, we now know that water temperatures in the future will be warmer and therefore fish may change in the future as well. Clearly, our extraordinary findings have huge implications for the future of fisheries management for a healthy and sustainable planet.

Ethics in peer review

Confidentiality

  • You must not take ideas presented in articles you review & pass them as your own
  • You must not disclose any data presented in the article
  • However, you can ask a colleague for advice as long as necessary details remain confidential (eg, author names)

Bias

We all have unconscious biases that affect our ability to conduct reviews

  • gender bias: articles by authors of either sex are subjected to different standards of review
  • geographical bias: authors’ country of origin will influence the manner in which their work is assessed
  • seniority bias: articles by authors at different stages in their careers will be subject to more or less favorable review
  • confirmation bias: articles reporting controversial or new ideas will be less favorably reviewed than articles that do not challenge conventional wisdom

Duplicate submission & publication

  • Do not submit a paper to various journals concurrently
  • Do not submit a paper with overlapping material, especially results, to different journals

Authorship

  • Recall that authorship is usually granted to those who have substantially contributed to the work presented in an article
  • Unjustified authorship: honorary or guest authorship assigned to people who have not made substantial contribution to the conception, design, acquisition of data, or analysis and interpretation
  • Ghost authorship: people who have contributed to authorship of the paper are omitted from the list of authors

Data

Unfortunately, not all data are what they appear

  • Fabricated data are made up rather than the result of actual measurements
  • Falsified data arise from measurements that have been unjustifiably altered in order to yield more impressive/convenient results
  • Theft of data someone else’s data without their consent
  • Animal welfare practices need to be followed when carrying out research

Conflict of interest

  • Reviewers: Colleagues or friends of authors who cannot provide an impartial review
  • Authors: Funding or association with an organization may raise questions about motivation and findings

Responding to reviews

Journal decisions

Generally fall along 3 lines:

  1. accept with minor revisions

  2. accept with major revisions

  3. reject

Journal decisions

BEWARE: reviewer comments can be blunt, unnecessarily degrading & utterly demoralizing

  • ask your advisor about their approach / policy

Responding to reviews

  • if not rejected, paper is revised along the lines suggested by editor & reviewers
  • lead author writes cover letter summarizing and detailing changes made
  • should reference & address every comment point-by-point; be as polite & flattering as possible

Responding to reviews

Example

P 7 L 21-23: Here the authors say they used data from 3 sites, but the first paragraph of the methods section mentions 4 sites were sampled. Why was one of the sites excluded?

We thank the reviewer for catching what was an error on our part. We did, in fact, use the data from all 4 sites and have made the appropriate edit to the sentence.

Responding to reviews

Example

P 13 L 11-12: It’s simply absurd to think that an increase in water temperature of 1.9 degrees C could elicit the change in growth you observed.

Our findings are very much in line with previous studies showing the effect of temperatuure on growth. For example, the reference we cited (Smith 2020) presents rather compelling evidence that a difference of only 1.5 degrees C increased growth by 20%.

Responding to reviews

Example

P 9 L 4-6: It would seem that in addition to the growth study, the authors could also add an analysis of the diet data to examine possible relationships between shifts in composition accompanied the observed changes in growth.

We agree with the reviewer that such an analysis could reveal some interesting patterns, but it is simply out of scope for this project given the resources and time required to undertake it.

Responding to reviews

  • if not rejected, paper is revised along the lines suggested by editor & reviewers

  • lead author writes letter summarizing and detailing changes made

  • should reference & address every comment point-by-point; be as polite & flattering as possible

  • note that you do not have to agree with everything, but you do need to justify everything
  • this process can take a long time; ask for an extension if you need one

Responding to reviews

Paper rejections

  • If your paper is rejected, are the reviews sensible & reasonable?
  • If not, consider contacting the editor to express your concerns and ask for clarification or re-evaluation
  • If so, consider revising along the lines suggested by the reviews & submit elsewhere