ROBIS: A Risk of Bias Assessment Tool for Systematic Reviews

A Summary of

University of Bristol (2016). Tool to assess risk of bias in systematic reviews. Retrieved from:


ROBIS is a tool designed to assess risk of bias in systematic reviews. ROBIS differs from other tools that assess systematic reviews as it was specifically designed to assess risk of bias, while other tools focus on broader goals such as critical appraisal and quality assessment. The use of ROBIS can lead to more robust recommendations and improvements in public health and patient care.

Access the Tool

Format of the Tool


  • English
  • Portuguese, Italian, Spanish





Using the Tool

Time for Participation/Completion

Information not available

Additional Resources and/or Skills Needed for Implementation

ROBIS users are likely to need expertise in both subject content and the methodologies used to complete an assessment.

Steps for Using Method/Tool

The tool includes three phases:

  1. Assess relevance (optional)
  2. Identify concerns with the review process
  • Study eligibility criteria
  • Identification and selection of studies
  • Data collection and study appraisal
  • Synthesis and findings
  1. Judge risk of bias
  • Determine whether the limitations identified in phase 2 were considered in the overall risk of bias in the interpretation of review findings

Phase 1 asks users to assess the relevance of the systematic review in question by comparing the target question of the overview or guideline being developed with the question being addressed in the systematic review being assessed. Ideally, the review addresses the same target question as the overview or guideline being developed. This phase of the ROBIS tool is optional and is not applicable if the user is not developing a guideline or overview.

Phase 2 consists of four domains: study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis and findings. Each domain has five or six signalling questions that are answered as “yes,” “probably yes,” “probably no,” “no” or “no information.” Once the signalling questions have been answered, the user can identify concerns regarding the synthesis and findings for each of the four domains as “low,” “high” or “unclear.”

Phase 3 is a summary of the concerns identified in phase 2 of the assessment. Each domain is assessed by concern and rationale for the concern. Finally, the risk of bias in the review is assessed based on three questions:

  1. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4?
  2. Was the relevance of identified studies to the review’s research question appropriately considered?
  3. Did the reviewers avoid emphasizing results on the basis of their statistical significance?

In order to ensure transparency in ratings, the information used to make each judgement is recorded.

Who is involved

ROBIS can be used by anyone interested in assessing risk of bias in systematic reviews. The authors state that their target audience includes guideline developers, authors of overviews of systematic reviews and review authors who want to assess and avoid risk of bias in their reviews.

Conditions for Use

© 2016 The Authors

Evaluation of the Tool


Information not available


Not applicable


Not applicable

Methodological Rating

Not applicable

Development of the Tool


University Hospitals Bristol NHS Foundation Trust
9th Floor, Whitefriars, Lewins Mead,
Bristol BS1 2NT
Phone: +44 117 34 212 73

Method of Development

ROBIS was developed using a four-stage approach: define the scope, review the evidence base, hold a face-to-face meeting and refine the tool through piloting.

Release Date


Contact Person/Source

Penny Whiting
University of Bristol
Phone: +44 117 34 212 73

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Have you used this resource? Share your story!