Preprints and what peer review actually does
What peer review checks and what it does not, why a preprint is not worthless, and how predatory journals complicate the picture.
Peer review is a check by other researchers before publication, assessing whether methods support conclusions, whether the work is novel and whether reporting is adequate. It is valuable and it is limited: reviewers rarely see raw data, do not repeat experiments, and cannot detect well constructed fabrication.
A preprint is a manuscript posted publicly before that check. It is not worthless and it is not verified. The practical distinction is that a preprint has had no independent scrutiny, while a peer-reviewed paper has had some, of unknown depth, from an unknown number of people, in a journal of unknown standards.
What peer review checks
At its best, review asks whether the question is clear, whether the methods can answer it, whether the analysis is appropriate, whether the conclusions follow, and whether the reporting is complete enough for a reader to judge. Reviewers request changes and editors decide.
What it generally does not do: verify raw data, repeat the experiment, audit the statistics independently, or confirm that the work was carried out as described. Review is a check on plausibility and coherence, not an audit.
Why it still matters
Because it catches a great deal. Unsupported conclusions, missing controls, inappropriate statistics and overclaiming are all things a competent reviewer notices. The difference between a manuscript before and after review is often substantial, particularly in the discussion, where the strongest claims live.
It also creates accountability. A paper carries a journal's name and reviewers' judgement, and there is a mechanism, imperfect but real, for correction and retraction.
Peer-reviewed publication establishes that a study's findings are correct.
- Proposed mechanism
- Independent expert scrutiny before publication verifies the work.
- What has been shown
- Peer review assesses whether methods support conclusions and whether reporting is adequate. It does not usually include verification of raw data or repetition of experiments. Retractions and post-publication corrections occur across all fields, which demonstrates that errors pass review. Replication remains the mechanism by which findings become reliable.
- Highest level reached
- Not shown
- Main confounders
- Journal standards vary enormously. Review depth varies between manuscripts at the same journal. Predatory journals perform little or no review while presenting as peer reviewed.
GradeNOT SUPPORTED
What would change thisNothing about the general claim. Peer review is a filter, not a verification. What establishes a finding is independent replication, and the strongest signal available to a reader is whether other groups have reproduced the result.
Preprints, fairly assessed
Preprint servers let researchers post manuscripts publicly before or during review. The arguments for them are good: faster dissemination, public timestamping, and open commentary. Many preprints are later published essentially unchanged.
The risk is misuse. A preprint cited as though it were reviewed, particularly in marketing, presents an unchecked manuscript as a verified finding. In this sector that occurs, and it is worth checking the status of any citation offered in support of a product. Preprint servers label their content clearly, so the check takes seconds.
Predatory journals
A more difficult problem than preprints. Predatory journals charge publication fees, perform minimal or no review, and present as legitimate. A citation to such a journal looks like a citation to a peer-reviewed paper and carries almost no assurance.
Signals worth noticing: a journal with a name closely resembling a well known one, an implausibly fast time from submission to publication where stated, an editorial board that is hard to verify, and aggressive solicitation. Indexing in established databases is a useful, imperfect check, and a journal not indexed anywhere recognised deserves scrutiny.
| Status | Scrutiny received | How to check |
|---|---|---|
| Preprint | None independent | Hosted on a preprint server, labelled as such |
| Peer reviewed, established journal | Some, depth unknown | Journal indexed in recognised databases |
| Peer reviewed, unknown journal | Unknown, possibly none | Check indexing, editorial board, publisher |
| Conference abstract | Minimal, often none | Short, no full methods, published in a supplement |
Conference abstracts
A category that deserves its own warning. Abstracts are short, frequently not peer reviewed in any meaningful sense, contain no full methods, and a substantial share never become full papers. A claim supported only by a conference abstract is supported by a summary of work that may not have withstood writing up.
They are cited in marketing precisely because they are easy to generate and look like publications. Checking whether a full paper followed is a reasonable step, and often the answer is that none did.
Where a paper appeared, and why it is a weak signal on its own
Journal prestige is a rough proxy for scrutiny and a poor one for correctness. High-profile journals publish work that is later corrected or retracted, and specialist journals with modest profiles publish careful work that nobody disputes. Selecting evidence by the name on the cover substitutes one heuristic for the reading you were trying to avoid.
A more useful signal is the specialism of the journal relative to the claim. A paper on vesicle characterisation in a journal whose readership does that work for a living has been seen by reviewers equipped to notice a missing control. The same paper in a general journal outside the field may have been reviewed by people assessing plausibility rather than method.
The most useful signal of all remains whether anyone else has reproduced the result. A modest paper that three groups have replicated carries more weight than a striking one that nobody has repeated, and that ordering is the opposite of how citation counts and press coverage rank them.
How this shows up in our grading
Our vocabulary distinguishes EARLY, UNREPLICATED from SUPPORTED, LIMITED for exactly this reason. The first says human data exist but have not been independently repeated. The second says more than one group points the same way. That distinction is invisible in a claim that says clinical studies show, which is why we do not use that phrasing.
The full vocabulary and what each grade requires is published at the grading vocabulary, so that a reader can check our use of it against our own definitions.
Questions readers ask
What does peer review check?
Whether the question is clear, the methods can answer it, the analysis is appropriate, the conclusions follow and the reporting is complete. It does not usually verify raw data or repeat experiments.
Is a preprint reliable?
A preprint has had no independent scrutiny. Many are later published essentially unchanged, and many are not. The problem is citing one as though it were reviewed, which presents an unchecked manuscript as a verified finding.
What is a predatory journal?
A journal that charges publication fees while performing minimal or no review, and presents as legitimate. Signals include names resembling established journals, unverifiable editorial boards, aggressive solicitation and absence from recognised indexes.
Are conference abstracts evidence?
Weak evidence at best. They are short, often not meaningfully reviewed, contain no full methods, and a substantial share never become full papers. Checking whether a full paper followed is worthwhile.
What actually makes a finding reliable?
Independent replication. Publication is a filter rather than a verification, and in a young field an unreplicated result is the normal state and should be described as such.