Retractions: their effect on the scholarly record, and what’s being done about it

The sudden global emergence of the COVID-19 pandemic in the first quarter of 2020 was soon followed by a flurry of research papers that hoped to provide some useful insights or hypotheses to medical professionals and policymakers. The large volume of briskly produced research, the high stakes at the time and the relaxed standards that some journals decided to temporarily implement made it more or less inevitable that a fair number of these papers would be retracted, as did indeed happen.

Seen a certain way, this is a cause for celebration, as it is a demonstration of the self-correcting nature of scholarly research in action. Indeed, each paper that is retracted can, in principle, give pause to the stakeholders concerned and force them to consider what could have been done differently. However, this is also cause for concern since retracted scholarship can remain in the scholarly record as cited references, something that academia has been grappling with for quite some time.

Let’s take a closer look at the challenges retractions present and how academia has been tackling them. But first, let’s set the context for that discussion.

The fundamentals of retractions

Retractions, which involve the withdrawal of pieces of research from the scholarly record and the invalidation of their conclusions, should be distinguished from two other common corrective mechanisms: errata and expressions of concern.

1. Errata, also often referred to as ‘corrigenda’, are relatively minor corrections that do not have an impact on the conclusion of a paper. Issues such as typos, mislabelled images and missing author names are usually addressed through errata.
2. An expression of concern is a note that aims to inform readers that a certain paper (or certain parts of one) might not be fully reliable. Such notes are typically published by journal editors only if the paper in question is subject to an ongoing investigation, or if the relevant evidence is inconclusive.

Papers can get retracted for a wide variety of reasons, not all of which are indicative of wrongdoing. For instance, retractions might be motivated by honest but fundamental mistakes with respect to methodology or data interpretation. Other potential reasons for retractions are: original data that is unverifiable, data falsification, the unreproducibility of results, plagiarism, undisclosed conflicts of interest and unethical conduct.

Retractions, whether originating from an author, editor or publisher, serve several key purposes in the world of academic publishing:

• They help maintain the integrity of the scholarly record, and ensure that future research is not built on a shaky foundation.
• The risk of having one’s paper retracted, and the attendant consequences thereof, can serve as a deterrent against unethical practices.
• The retraction of papers in fields such as medicine can prevent patients from being subjected to potentially harmful treatments.

Ideally, for retractions to benefit the scholarly record as much as possible:

1. Papers that have pervasive errors or that are found to have involved unethical conduct of any kind should be retracted as quickly as possible.
2. The fact that a paper has been retracted should be disseminated as broadly and as quickly as possible.
3. Any future papers citing a retracted paper should indicate that it has been retracted.
4. Any existing research reliant on the results of a retracted paper should be reviewed and amended (or possibly even retracted!) in an appropriate manner.

However, as might be expected, the actual situation is far from ideal. For instance, journals can often take several years to retract papers even after misconduct on the part of an author has been proven (with one such retraction having taken 12 years).

Similarly, it continues to be far from trivial to effectively disseminate the retracted status of papers and ensure that citations referring to such papers explicitly indicate their status; this is probably even truer of Point 4 above.

How stakeholders are (and have been) trying to address issues surrounding retractions

Of course, the academic community has long been cognizant of the importance of the aforementioned challenges, and has taken and continues to take steps to tackle them effectively.

Such measures can be considered to fall under two broad categories:

• the publication of recommendations and guidelines that are mostly relevant to editors and publishers, and
• the creation of resources that are primarily meant to make it easier for researchers to search for and appropriately handle retracted work, as well as to flag potentially serious issues with published work.

Let’s take a closer look at each of these categories in turn.

Recommendations and guidelines

Journal editors and reviewers have traditionally played the biggest role in determining which papers should be retracted and how seriously flawed papers should be dealt with; they continue to hold a lot of power in these regards. And sometimes, their interests and those of the global research community as a whole might not be fully aligned.

For instance, in general, it is in the best interest of a journal to simply issue corrections to a paper rather than retract it, as each retraction has the potential to hurt the journal’s credibility and raise uncomfortable questions about its review process.

In addition, while some journals or disciplines might have certain de facto norms regarding retraction-related nomenclature and metadata and the communication of retraction-related information, such norms might not overlap across journals or disciplines, or be strictly enforced.

Lastly, there may be cases where authors are aware of major issues with their papers but may not be willing to bring them to the attention of the relevant journal because they fear the stigma that comes with retraction.

Several bodies have drawn up guidelines and recommendations that aim to address such issues. Such bodies may be mainly oriented towards specific disciplines (e.g. the International Committee of Medical Journal Editors (ICMJE): its recommendations can be accessed here) or aim to be cross-disciplinary in nature (e.g. the Committee on Publication Ethics (COPE): its flowcharts on how to deal with issues related to publication ethics can be accessed here).

However, it has been found that publishers do not uniformly follow such recommendations (see here, for instance).

More recently (in September 2022, to be precise), a project known as ‘Reducing the Inadvertent Spread of Retracted Science’ (RISRS), whose goal was to “[…] develop an actionable agenda for reducing the inadvertent spread of retracted science” published a set of four recommendations in a report that was the culmination of a year-long process. These recommendations point to useful directions for progress in the matter of retractions, and also serve as an implicit summary of the state of the art; for these reasons, they are reproduced below:

“(1) Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions; 

(2) Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders; 

(3) Develop best practices for coordinating the retraction process to enable timely, fair, unbiased outcomes; and 

(4) Educate stakeholders about pre- and post-publication stewardship, including retraction and correction of the scholarly record.”

There is also some potentially fruitful work being currently carried out by bodies such as the National Information Standards Organization through the CREC (Communication of Retractions, Removals, and Expressions of Concern) Working Group, which aims to describe “[…] the involved parties, along with their responsibilities, actions, notifications, and the metadata necessary to communicate retracted research across stakeholders’ systems”.

Resources for researchers

There was a time when even conscientious researchers who wished to be as thorough as possible would have found it difficult to correctly determine a given paper’s status. Similarly, individual scholars who spotted questionable data, figures, studies or methodologies in published papers might have found it difficult to have transparent discussions about them not too long ago.

Today, however, there are several researcher-oriented platforms that have made such problems much more tractable. Two such platforms are:

1. Retraction Watch, which started out in 2010 as a blog that reported on retractions and the issues surrounding them. In 2018, a searchable database of the retracted papers the site’s operators had collected went live, and contained more than 37,000 retractions as of early 2023.
2. The post-publication peer review (PPPR) platform PubPeer, whose goal is to enable issues in papers to be discussed publicly so that authors can be incentivized to also respond publicly. PubPeer also provides browser extensions that make it easy to check if a given paper has been a subject of discussion on it, thus allowing researchers planning to cite a certain paper to get an idea of any potential issues with it.

Retractions going forward

Thus, it should be clear that effectively disseminating the retracted status of a paper to all relevant stakeholders, and using common nomenclature and formats for presenting retractions, are going to play a key role in the future if publishers and editors are to truly preserve the integrity and reliability of the scholarly record. The need of the hour is for all stakeholders to come together and standardize how they handle retractions and their associated processes: there certainly is no dearth of existing recommendations to choose from.

It has been proposed that retraction and correction notices should include details such as the problems a paper had and who was responsible for them: this would prevent potentially incorrect assumptions of wrongdoing, and would also prevent co-authors from having their careers affected by unethical colleagues. It has even been suggested that the term ‘retraction’ should only be reserved for cases of intentional misconduct, with alternative terms being used for other kinds of issues, with the purpose of this being to encourage more authors to flag aspects of their papers they know to be problematic.

Is this the direction scholarly publishing will go in? Only time will tell.

The recording of HighWire’s best practices webinar series on ‘The effects of retractions on the scholarly record’ is available now. Watch here.