When a topic has accumulated a dozen systematic reviews, each answering a slightly different slice of the question, a reader drowns rather than learns. The umbrella review exists for exactly that situation. Instead of going back to the primary studies, it synthesizes the systematic reviews themselves, standing one level up and summarizing what the reviews collectively say.
It is a powerful tool for mature fields and a poor one for young ones. The trick is knowing when you have enough reviews to justify sitting above them.
An umbrella review, also called a review of reviews or an overview of reviews, is a systematic review whose included studies are themselves systematic reviews. Rather than screening individual trials, you gather, appraise, and synthesize the existing systematic reviews on a topic.
Use one when a field already has multiple systematic reviews and you want a high-level summary across them. It answers "what does the whole body of review evidence tell us" rather than "what do the primary studies show."
The precondition is simple: enough systematic reviews must already exist. If a topic has one or two reviews, an umbrella review has nothing to stand on. If it has many, especially several that overlap or reach different conclusions, an umbrella review earns its place.
Typical situations include a broad question that individual reviews have each addressed in part, a field where reviews disagree and someone needs to make sense of the divergence, or a decision-maker who needs the big picture across many interventions without reading twenty separate reviews. Umbrella reviews are common for summarizing the evidence on a condition across multiple treatments, or the effects of an exposure across many outcomes.
If the primary studies have never been systematically reviewed, you need a systematic review, not an umbrella review. The umbrella sits on top of existing reviews; it does not replace them.
An umbrella review follows the systematic review method, with the unit of analysis shifted up to the review level.
1. Define the question and protocol. As with any systematic review, start with a focused question and a registered protocol. The difference is that your inclusion criteria specify systematic reviews rather than primary studies.
2. Search for systematic reviews. Build a search designed to find reviews, using review-oriented filters and databases that index them, including the Cochrane Database of Systematic Reviews. Your target is published systematic reviews on the question, not the underlying trials.
3. Screen and select. Screen the reviews against your criteria in duplicate, just as you would primary studies. Decide upfront what counts as a systematic review for your purposes, since review quality varies widely.
4. Appraise the included reviews. This is where umbrella reviews differ most. You appraise the methodological quality of each included review, commonly with AMSTAR 2, which is designed to assess systematic reviews. A tool like ROBIS can also assess risk of bias in the reviews. This tells you how much to trust each review's conclusions.
5. Handle overlap. Included reviews often share primary studies, so the same trial can appear in several of your reviews and get double-counted. Managing this overlap is the signature methodological challenge of umbrella reviews. Approaches include mapping which primary studies appear in which reviews and reporting the degree of overlap, sometimes with a measure such as the corrected covered area. You must acknowledge overlap rather than ignore it.
6. Synthesize. Summarize the findings across reviews, usually narratively, presenting what the body of review evidence shows, where reviews agree, and where they diverge. Umbrella reviews rarely pool results statistically, because the reviews are too heterogeneous and the overlap complicates any pooling.
7. Report. Report against an appropriate standard and account for your reviews in a flow diagram, as in any systematic review.
Overlap deserves its own attention because it is where umbrella reviews most often go wrong. Suppose three of your included reviews each analyzed the same landmark trial. If you simply tally conclusions, that trial's influence is tripled, and your umbrella review overstates the consistency of the evidence. Reviews that appear to agree may agree only because they drew on the same handful of primary studies.
A rigorous umbrella review maps this. It records which primary studies feed which reviews, reports how much they overlap, and interprets agreement in that light. Two reviews reaching the same conclusion from entirely different studies is strong evidence. Two reviews reaching it from the same studies is one piece of evidence counted twice. Distinguishing the two is the umbrella review's real job.
| Feature | Systematic Review | Umbrella Review |
|---|---|---|
| Unit of analysis | Primary studies | Systematic reviews |
| Question | Focused, specific | Broad, cross-review |
| Appraisal tool | RoB 2, ROBINS-I, etc. | AMSTAR 2, ROBIS |
| Main challenge | Comprehensive search | Managing overlap between reviews |
| Synthesis | Narrative or meta-analysis | Usually narrative |
| Precondition | Primary studies exist | Multiple systematic reviews exist |
Running one too early. Without several existing reviews, an umbrella review has insufficient material. Confirm the field is mature enough first.
Ignoring overlap. Treating overlapping reviews as independent inflates the apparent consistency of the evidence. Map and report the overlap.
Skipping review appraisal. The included reviews vary in quality. Appraising them with AMSTAR 2 or ROBIS is essential, because a conclusion drawn from weak reviews deserves less confidence.
Trying to meta-analyze. Pooling across reviews is usually inappropriate given their heterogeneity and shared studies. Umbrella reviews are typically narrative for good reason.
What is an umbrella review? A systematic review of systematic reviews. It synthesizes existing reviews on a topic rather than the primary studies beneath them.
How is an umbrella review different from a systematic review? Its included studies are systematic reviews, not primary studies. It appraises those reviews with tools like AMSTAR 2 and must manage overlap between them.
What tool do I use to appraise the included reviews? AMSTAR 2 is commonly used to assess the methodological quality of systematic reviews. ROBIS assesses their risk of bias.
Do umbrella reviews include a meta-analysis? Rarely. The heterogeneity between reviews and the overlap in their primary studies usually make pooling inappropriate, so synthesis is typically narrative.
When should I do an umbrella review instead of a systematic review? When multiple systematic reviews already exist on your question and you want to summarize across them. If the primary studies have not been reviewed, do a systematic review instead.
An umbrella review synthesizes existing systematic reviews to give a high-level view of a mature field. It follows the systematic review method with the unit of analysis raised to the review level, appraises included reviews with tools like AMSTAR 2, and lives or dies on how well it handles the overlap between reviews that share primary studies.
Use one only when the field has enough reviews to synthesize, and treat overlap as the central methodological task rather than an afterthought. For where umbrella reviews sit among other designs, see types of systematic reviews.
Running any kind of evidence synthesis? Verflux supports screening, appraisal, and synthesis in one workflow.
All analytical features included in the free trial. No credit card, no installation, no R or Python.
Create free account