Bleak Prospects for the Peer Review System

For decades, our knowledge has relied on the peer review system, which is intended to ensure that only scientifically sound studies—methodologically and factually—see the light of day. In the academic community, legitimate concerns are beginning to emerge about what the rise of artificial intelligence may do to this system. Let us look at the main issues.

The number of publications had already been growing rapidly even before the emergence of artificial intelligence—to such an extent that it is beyond the capacity of any individual to actually read all the articles. As early as 2022, around 5,140,000 articles were published annually in approximately 50,000 scientific journals. If a person devoted just one minute to each article, it would take ten years—without sleep or food—to read that year’s output. This situation is driven by the principle of publish or perish, according to which scholars must publish in order to advance their careers, obtain grants, and so on. This trend began long before the widespread adoption of ChatGPT, and even then the situation was already becoming unsustainable.

The rise of artificial intelligence is accelerating this growth. Data suggest that those who actively use AI publish 50–60 % more articles. Even if we acknowledge that AI may improve the formal—and perhaps also the methodological and argumentative—quality of these studies, the peer review system will likely reach its limits. If the productivity of individuals increases dramatically while the number of willing reviewers remains constant, an imbalance inevitably emerges.

Editors and authors are already reporting problems with the length of the review process. Simply put, the number of reviewers is becoming dramatically disproportionate to the volume of articles that need to be reviewed. If today the acceptance rate of review invitations is between 5–10 %, in the future a paper may remain “stuck” in the system for an unacceptably long time before someone is willing to review it. This problem is compounded by the fact that reviewing is highly qualified and intellectually demanding work, yet it is unpaid and essentially relies on collegial solidarity. However, such solidarity has its limits.

It is therefore no surprise that reviewers themselves are beginning to evaluate manuscripts with the help of AI. Yet this effectively moves us outside the peer review system itself (“peer” implies evaluation by a colleague, not by artificial intelligence). When artificial intelligence evaluates the work of artificial intelligence, we enter a strange loop whose outcome is unclear—and where responsibility for potential errors becomes uncertain.

While I have heard the above complaints from colleagues on social networks, few people see a clear solution. At present, we simply do not know where this development will lead or what we will do about it, but optimism is certainly not widespread. These trends may lead to changes in the typical structure of scientific papers (introduction – theoretical background – methodology – results – discussion – conclusion). Because reviewers and readers alike may become overwhelmed by texts that often look “copy-pasted,” it may become necessary to remove much of the narrative padding from papers. The main emphasis will likely be placed on methodology and results, while sections such as the Introduction and Conclusion may recede into the background—perhaps replaced by a few bullet-point “highlights.” Greater emphasis will likely be placed on open data and preregistration to prevent manipulation. We may also see trends toward open peer review (where reviews and author responses are public) and post-publication peer review (where articles are evaluated by the community after publication).

Another possibility is that the publishing world will effectively split into two layers: one consisting of top journals with extremely long waiting times, and another where massive production of papers continues—many of which will be more or less ignored by parts of the scientific community.

The worst-case scenario, however, is an erosion of trust in scientific publishing. This is why it is important to have this debate before it is too late. We are already seeing some early signs of adaptation, at least in education. I am pleased that the Faculty of Business Administration at the Prague University of Economics and Business has abolished bachelor’s theses and replaced them with a bachelor’s project, where the main emphasis will be on the output itself rather than on the surrounding textual apparatus. It is a small but symbolic shift, and it is possible that other universities will follow.