The Evolution of Peer Review and the Role of Editors: Thoughts from Nature Communications’ Editor in Chief

To help us understand more about peer review, the role that editors play, and how accountability and transparency promote ethics and integrity in the process, we invited Nature Communications’ Editor in Chief, Elisa De Ranieri, to share her thoughts with us.

How has the peer review process evolved over time?

Peer review as the main vehicle for assessing the integrity of research output is a less established practice than commonly thought – for example, it was only in the 1970’s that it became the norm at Nature. In any case, it is now a cornerstone of scientific publishing and as such all actors (authors, reviewers, editors, publishers, readers) are keen to keep innovating to make peer review more robust. Recent trends aim to make it also more transparent: traditional peer review – where an editor mediates comments from anonymous reviewers – tends to be perceived as a black box, and its effectiveness has been questioned. Transparency leads to trustworthiness, hence the recent focus on transparent or open peer review, in which reviewer reports and/or their identities, as well as editorial notes and authors’ rebuttal are published. On the other hand, in the attempt to remove unconscious bias from the process, double-blind peer review (where the authors’ identities are kept from reviewers until a final decision on the manuscript is taken) has been introduced by some publishers as an optional service, but it’s efficacy in removing bias has yet to be proven.

The emergence of preprint servers can also be seen as a push towards transparency: while in the physical sciences the arXiv has been around for almost 30 years, the rise of bioRxiv and the birth of servers in other disciplines are popularising the trend to a wider set of disciplines. By making results public in this way, the community is welcome to comment thus delivering in effect a community-mediated peer review, which can go in parallel to the traditional model or not. Challenges of this model include difficulties in ensuring that a suitable number of qualified experts provides comments on the work, in the absence of a mediator such as an editor. Finally, there are now platforms such as F1000 in which the peer review happens post-publication in a transparent way, and reviewers are selected by the authors themselves. In this model, it might be difficult to ensure impartiality in reviewer selection, making it more difficult to ensure the integrity and ethics of the process.

What is the future of peer review? What innovations should we expect?

Looking ahead, we can expect the transparency trend to gain even more momentum and I do see a world in which all elements of the process are made public, for all journals. Innovations will depend on what technology is available, but ultimately the greater issue to overcome is rather a cultural one. There is still a lot of resistance from a fraction of the research community towards transparency, and it will take time to mitigate the risks that this fraction perceives from a fully transparent process. Other trends that might take hold could include a decentralisation of the figure of the reviewer, and we might see more of the crowd-sourcing of reviewers, as in the preprint model. Could we also see reviewers replaced by AI? I doubt it, but wouldn’t be too surprised to be proven wrong.

In addition, I expect innovations that will enable reviewers to more easily assess the quality of the data. For code, things are moving already – for example, some of the Nature-branded titles are running a pilot scheme in collaboration with Code Ocean that allows reviewers to test the author’s code as part of the peer review process. I can imagine that we are going to make similar progress in the data space.

Ultimately, I think that we will not move completely away from a model that has editors at its heart, at least not in the near future, as they perform the essential functions of selection and curation of the content.

What is the role of editors in peer review?

The editor plays a crucial role in championing and safeguarding the integrity of peer review. This is particularly true of professional editors, as they are less prone to competing interests when assessing a research work and can thus be more objective in their decisions. A key task performed by editors is reviewer selection, which is essential to maintain the integrity of published material. Professional editors have the competence and time to provide a high-quality and robust peer review experience.

Editors will also work in collaboration with the research community to develop quality and reproducibility standards for doing and reporting research that can be implemented via journal policy, for example using checklists during the peer review process. In this way, they foster integrity in the publishing process and promote ethics in research practices. A study by Macleod et al.  reported that “there was a substantial improvement in the reporting of risks of bias in in vivo research in NPG journals following a change in editorial policy” suggesting a positive impact of journal-led policies.

 How do we keep peer review trustworthy?

Accountability and transparency are means to make a process more trustworthy, so I argue that publishers need and should make all contributors to peer review more accountable and the process more transparent. In turn, this will support and enable ethics and integrity in research and publishing practices. Besides the obvious steps mentioned above (publishing reviewer and editor names, reviewer reports, correspondence among the parties) publishers should be more open about peer review’s times and procedures, they should support early disclosure of results with appropriate policies as well as the dissemination of negative results to avoid large waste of public money in repeating research works doomed to fail. They can also mandate the sharing of raw data, and enforce more transparent competing interests statements.

Elisa has been working at Nature Communications since 2012 where she has held various editorials roles, she also has a PhD in Physics from the University of Cambridge. In January, Elisa attended the Academic Publishing in Europe Conference in Berlin where she participated in a panel discussion on Integrity, Ethics and Peer Review. You can view her slides here.