How many reviewers does a systematic review need? Why two independent reviewers is the standard, what a third is for, when single screening is acceptable, and how to handle small teams.
The short version is two, plus someone to break ties. But the reasoning behind that number is more interesting than the number itself, and knowing it tells you what to do when you cannot meet the standard.
The requirement is not arbitrary bureaucracy. It exists because one person, screening thousands of records against written criteria, makes mistakes at a rate that would compromise the review. Two people working independently catch each other's errors. That is the entire logic.
The standard is two reviewers screening independently at both the title/abstract and full-text stages, with a third reviewer available to resolve disagreements. Two reviewers extracting data independently is also expected where feasible.
So the practical minimum team is three people: two screeners and an arbiter. Two can work if the pair can resolve disagreements by discussion, but with no tiebreaker you will eventually hit a deadlock.
Because single screening is measurably unreliable. A reviewer working alone through thousands of abstracts will wrongly exclude eligible studies, simply through fatigue, ambiguity, and the ordinary error rate of human judgment. Nobody notices, because there is no second opinion to reveal it. The missed studies simply never appear in the review, and neither the author nor the reader can tell.
Two independent reviewers fix this in a specific way. Each screens the same records without seeing the other's decisions, and then the decisions are compared. Where they agree, confidence is high. Where they disagree, a real question has been surfaced that would otherwise have been silently resolved one way by one person. The disagreements are the point. They are the mechanism working.
This is also why the screening must be genuinely independent. Two people screening together in the same room, discussing as they go, is not double screening. It is one screening decision made by two people, and it inherits all the bias of a single reviewer plus a dose of whoever is more assertive.
The third reviewer is an arbiter, not a screener. When two reviewers disagree and cannot settle it by discussion, the third makes the call. This person is often the lead reviewer or the content expert, and they should be agreed in advance rather than recruited in the middle of an argument.
You do not need a third person screening every record. You need someone available to adjudicate the fraction of records where the first two disagree.
| Stage | Standard | Notes |
|---|---|---|
| Title and abstract screening | Two, independently | The stage with the most records and the most errors |
| Full-text screening | Two, independently | Exclusion reasons recorded for each |
| Disagreements | A third reviewer | Only for cases the first two cannot resolve |
| Data extraction | Two, independently, where feasible | Reconcile differences |
| Risk-of-bias assessment | Two, or one with a checker | Assessments should be verified |
Not everything, and the expectations differ by stage.
Screening is where the double-reviewer standard is firmest. This is the stage that determines which studies exist in your review at all, and errors here are unrecoverable.
Data extraction should also be done in duplicate where feasible, because transcription errors are common and consequential, especially for the numbers feeding a meta-analysis. Some teams have one person extract and a second verify, which is a reasonable compromise.
Risk-of-bias assessment benefits from two assessors, or at minimum one assessor with a second person checking. These are judgment calls, and a single judgment is more variable than a reconciled one.
Searching does not need two people running the search, but it strongly benefits from a second pair of eyes on the strategy, ideally an information specialist.
Some fields expect you to report how well your reviewers agreed, typically with a statistic such as Cohen's kappa, which measures agreement beyond what you would expect by chance. Reporting it demonstrates that your criteria were applied consistently.
Low agreement is not primarily a sign of careless reviewers. It usually means the eligibility criteria are ambiguous, and two reasonable people reading them arrive at different conclusions. The fix is to sharpen the criteria, which is exactly why you pilot them on a sample before full screening.
This is a real constraint, particularly for students working alone on a thesis. Be honest about the options.
The best answer is to find a second screener, even a colleague who can screen a subset rather than the whole set. Some teams have a second reviewer independently screen a random sample of records as a quality check, which is weaker than full double screening but far better than nothing, and it lets you report an agreement rate.
If you genuinely cannot, you have two defensible routes. Conduct the review with single screening and disclose it clearly as a limitation, so readers can weigh the review accordingly. Or reframe the project as an explicit rapid review, which permits single screening as a disclosed methodological shortcut.
What you should not do is perform single-reviewer screening while presenting the work as a full systematic review that met the standard. That is the failure mode reviewers punish, and rightly.
How many reviewers do you need for a systematic review? Two screening independently, plus a third to resolve disagreements. Three is the practical minimum team size.
Can one person do a systematic review? It is possible but falls short of the standard, because independent double screening cannot be performed. Disclose it as a limitation, or run an explicit rapid review instead.
Do both reviewers screen every record? At the screening stages, yes. Both screen the same records independently, then compare. The third reviewer only handles disagreements.
Does data extraction need two people? Where feasible, yes. Duplicate extraction catches transcription errors, which matter especially for numbers feeding a meta-analysis. One extractor plus a verifier is a common compromise.
What if my two reviewers disagree a lot? That usually points to ambiguous eligibility criteria rather than careless reviewers. Revisit and sharpen the criteria, ideally during a pilot before full screening.
Two independent reviewers for screening, a third to arbitrate, and duplicate extraction where you can manage it. The redundancy exists because a single reviewer's errors are invisible, and invisible errors are the ones that quietly change what your review concludes.
If you cannot meet the standard, say so plainly or run a rapid review. The one unacceptable option is claiming a rigor you did not have.
Coordinating independent screening across reviewers? Verflux supports blinded dual screening with built-in conflict resolution.
All analytical features included in the free trial. No credit card, no installation, no R or Python.
Create free account