Peer review serves as a rigorous vetting process that ensures the quality and integrity of published research. Its roots can be traced back to 1665, to the Philosophical Transactions of the Royal Society, one of the original scholarly journals. The founding editor, Henry Oldenberg, envisioned it as a platform to establish provenance over original ideas, evaluate them critically, and disseminate them to a broader audience. Fast forward to today, and technology has revolutionized how this “conversation between scholars” takes place, expanding it beyond the confines of anonymous peer review to open forums like preprint servers and post-publication commenting.
This blog post draws insights from a recent installment of Highwire’s Best Practice Webinar Series, featuring a distinguished panel of experts: Fiona Hutton, Head of Publishing at eLife Sciences, Monica Granados from PREreview; and Boris Barbour from PubPeer. The webinar delved into the future of peer review by exploring community and third-party peer review services. It examined how these emerging models fit into the broader ecosystem of scientific peer review, offering a timely look at the innovations that are shaping this critical aspect of scholarly communication.
Let’s dive in.
Reviewing the Future of Scientific Publishing: Fiona Hutton on eLife’s New Model
The Current Landscape: What’s Wrong?
The existing system of scientific publishing is fraught with inefficiencies and limitations. It’s slow, often taking nine months between submission and publication, which undervalues the time of academics. The system also lacks author control and places greater emphasis on where an article is published rather than its content. This has led to a cascade effect, where articles are shifted from one journal to another based on arbitrary indicators, further slowing down the process.
Monica from PREreview echoes Fiona’s sentiments about the limitations of the current peer review system. She adds another layer by highlighting that the pool of reviewers is not only small but also homogeneous. This pool is usually composed of male, mid-to-late career researchers, often selected based on existing relationships with journal editors. This perpetuates a cycle of bias and limits the diversity of perspectives in the review process.
The Winds of Change: Open Science
The scientific community is increasingly advocating for open science, with initiatives aimed at ensuring the adoption of open, equitable, and secure science. Preprints, according to eLife, serve as the first step in this new process.
eLife’s New Model: An Overview
eLife’s new model aims to make the process faster, more transparent, and author-centric. It follows a similar process to traditional journals at the initial stages—submission, decision to review, and peer review. However, eLife introduces an “eLife assessment,” published along with the articles. This assessment is a detailed review that gives readers more insight into what the reviewers and the editor thought of the article.
Author Choice: The Core Principle
A significant aspect of eLife’s new model is the emphasis on author choice. Authors can choose to revise their article based on comments, publish it as a version of record, or even opt for a different journal. This model values the time of both authors and reviewers, making the process more efficient.
Community Reception and Progress
Since its launch, eLife’s new model has sparked considerable discussion within the scientific community. It has received nearly 4,000 submissions and shows that the validation of research through reviewed preprints is happening much faster—within one to three months of the publication of the preprint. Interestingly, public reviews are being actively engaged with, indicating a high level of interest in what reviewers have to say about articles.
Editorial Support: A Positive Outlook
Feedback from eLife’s senior editors and reviewing editors indicates strong support for the new model, with over 80% being very to moderately supportive. This endorsement from the editorial backbone of eLife is a promising sign for the model’s future.
Decentralizing Peer Review: Monica Granados on PREreview’s Vision for an Equitable System
The Need for Training and Equity
Interestingly, despite the long-standing nature of the peer review system, many reviewers have never received formal training on how to conduct reviews. PREreview aims to address this by providing training and mentorship, focusing on traditionally marginalized groups who have been excluded from the review process. The goal is to measure reviewers by their constructive contributions and community engagement rather than by seniority, which is often the case in the current model.
The PREreview Model: Openness and Transparency
PREreview’s model is built on the backbone of preprints, aiming for a system where scholarly communication is open, rapid, and conducted by a diverse global community. They offer a community platform for open preprint review, providing training across career stages to ensure that reviews are constructive rather than destructive.
Lowering Barriers and Increasing Participation
PREreview is committed to lowering the barriers to participation in the review process. Through their platform, anyone with an ORCID ID can give feedback on a preprint, without needing an invitation from a journal editor. They also host live, collaborative reviews, moderated to ensure constructive feedback.
Community-Centric Approach
PREreview offers “PREreview Clubs,” small self-organized groups that focus on reviewing papers around common themes. These clubs foster a sense of community and ownership, aiming to make the review process more enjoyable and welcoming.
Recognition and Ownership
All reviews conducted through PREreview receive DOIs and are published under a CC BY license, allowing for citation and broader sharing. This ensures that reviewers are acknowledged and have a sense of ownership in the process.
Partnerships and Community Feedback
PREreview has partnered with various organizations and adopts a user-centered approach. They have recently relaunched their platform and are actively seeking community feedback to continually improve their model.
By lowering barriers and providing training, PREreview is working towards building a more inclusive and constructive system for scholarly communication.
Scientific Accountability: Boris on the Evolution and Impact of PubPeer
PubPeer has carved a unique space in the scientific community, serving as a platform for post-publication peer review. Boris, an academic scientist involved with PubPeer, provided a comprehensive look into the platform’s reach, legal resilience, and future directions.
How PubPeer Works
With just a DOI, PubPeer populates a page with metadata for every scientific article, providing a text box for comments. These comments can range from minor corrections to pointing out significant errors or inconsistencies. Importantly, the platform allows for anonymous commenting, a feature that has significantly increased user engagement.
With over 200,000 user comments on nearly 100,000 papers, and monthly views ranging from half a million to a million, PubPeer’s influence is undeniable. One of the platform’s most notable features is its legal robustness. Despite allowing anonymous comments—a point of contention in academic circles—PubPeer has never faced legal challenges. This speaks volumes about the platform’s stringent moderation policies, which Boris emphasized are designed to ensure comments are factual, relevant, and publicly verifiable.
Advanced Features for Publishers
For publishers seeking deeper engagement, PubPeer offers specialized dashboards as a subscription service. These dashboards provide real-time alerts about comments, historical commentary access, and the ability to respond officially. This feature is particularly timely, as PubPeer’s data is currently part of a trial by the STM Consortium, a development that could revolutionize how publishers interact with community feedback.
The Unexpected Outcomes
While the platform was initially designed to foster scientific discourse, it has also become a watchdog for more severe issues. These range from data duplications to machine-generated fake papers, highlighting the platform’s role in maintaining the integrity of scientific publications.
Moderation is Key
To ensure the quality of the discourse, PubPeer employs strict moderation guidelines. Every comment must be scientific, factual, polite, and relevant. Explicit allegations of misconduct are not allowed, but implicit allegations must pass through a rigorous moderation process. This combination of expert commenters and moderators ensures that any allegations are highly likely to be accurate.
Closing Thoughts
The push for diversification and decentralization is not merely a trend but a necessary evolution. eLife’s author-centric model, PREreview’s focus on training and inclusivity, and PubPeer’s commitment to post-publication scrutiny collectively signal a paradigm shift.
The challenges are manifold, from ingrained biases to the lack of formal training for reviewers. The collective endeavor should be to foster a system that upholds the integrity of scientific research while adapting to the changing needs and expectations of its stakeholders. The question is not whether the peer review system will change, but how swiftly and effectively the academic community can steer this change towards a more equitable and transparent future.
