ROBIS: A Risk of Bias Assessment Tool for Systematic Reviews

A Summary of

University of Bristol (2016). Tool to assess risk of bias in systematic reviews. Retrieved from: www.robis-tool.info

Relevance for Public Health

Public health practitioners require evidence to guide their decision making. Systematic reviews are considered the most reliable form of evidence, but they need to be appraised to ensure that results are not biased. The ROBIS tool is designed to assess risk of bias in reviews within health care settings in terms of four main categories: interventions, diagnosis, prognosis and etiology.

Description

ROBIS is a tool designed to assess risk of bias in systematic reviews. ROBIS differs from other tools that assess systematic reviews as it was specifically designed to assess risk of bias, while other tools focus on broader goals such as critical appraisal and quality assessment. The use of ROBIS can lead to more robust recommendations and improvements in public health and patient care.

Accessing the Method/Tool


Language(s)

  • English
  • Portuguese, Italian, Spanish

Format(s)

Periodical

Cost

None

Implementing the Method/Tool


Time for Participation/Completion

Information not available

Additional Resources and/or Skills Needed for Implementation

ROBIS users are likely to need expertise in both subject content and the methodologies used to complete an assessment.

Steps for Using Method/Tool

The tool includes three phases:

  1. Assess relevance (optional)
  2. Identify concerns with the review process
  • Study eligibility criteria
  • Identification and selection of studies
  • Data collection and study appraisal
  • Synthesis and findings
  1. Judge risk of bias
  • Determine whether the limitations identified in phase 2 were considered in the overall risk of bias in the interpretation of review findings

Phase 1 asks users to assess the relevance of the systematic review in question by comparing the target question of the overview or guideline being developed with the question being addressed in the systematic review being assessed. Ideally, the review addresses the same target question as the overview or guideline being developed. This phase of the ROBIS tool is optional and is not applicable if the user is not developing a guideline or overview.

Phase 2 consists of four domains: study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis and findings. Each domain has five or six signalling questions that are answered as “yes,” “probably yes,” “probably no,” “no” or “no information.” Once the signalling questions have been answered, the user can identify concerns regarding the synthesis and findings for each of the four domains as “low,” “high” or “unclear.”

Phase 3 is a summary of the concerns identified in phase 2 of the assessment. Each domain is assessed by concern and rationale for the concern. Finally, the risk of bias in the review is assessed based on three questions:

  1. Did the interpretation of findings address all of the concerns identified in Domains 1 to 4?
  2. Was the relevance of identified studies to the review’s research question appropriately considered?
  3. Did the reviewers avoid emphasizing results on the basis of their statistical significance?

In order to ensure transparency in ratings, the information used to make each judgement is recorded.

Who is involved

ROBIS can be used by anyone interested in assessing risk of bias in systematic reviews. The authors state that their target audience includes guideline developers, authors of overviews of systematic reviews and review authors who want to assess and avoid risk of bias in their reviews.

Conditions for Use

© 2016 The Authors

Evaluation and Measurement Characteristics


Evaluation

Information not available

Validity

Not applicable

Reliability

Not applicable

Methodological Rating

Not applicable

Method/Tool Development


Developer(s)

NIHR CLAHRC West
University Hospitals Bristol NHS Foundation Trust
9th Floor, Whitefriars, Lewins Mead,
Bristol BS1 2NT
Phone: +44 117 34 212 73

Method of Development

ROBIS was developed using a four-stage approach: define the scope, review the evidence base, hold a face-to-face meeting and refine the tool through piloting.

Release Date

2016

Contact Person/Source

Penny Whiting
University of Bristol
Phone: +44 117 34 212 73
Email: penny.whiting@bristol.ac.uk

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Resources

Title of Primary Resource ROBIS: Tool to Assess Risk of Bias in Systematic Reviews
File Attachment None
Web-link
Reference

University of Bristol (2016). Tool to assess risk of bias in systematic reviews. Retrieved from: www.robis-tool.info

Type of Material Online resource
Format On-line Access
Cost to Access None.
Language English, Portuguese, Italian, Spanish
Conditions for Use None
Title of Supplementary Resource ROBIS tool
File Attachment None
Web-link
Reference

University of Bristol (2017). ROBIS tool. Retrieved from: http://www.bristol.ac.uk/population-health-sciences/projects/robis/robis-tool/

Type of Material Website
Format On-line Access
Cost to Access None.
Language English
Conditions for Use None
Title of Supplementary Resource ROBIS: A New Tool to Assess Risk of Bias in Systematic Reviews Was Developed
File Attachment None
Web-link
Reference

Whiting, P., Savovic, J., Higgins, J.P.T., Caldwell, D.M., Reeves, B.C., Shea, B., et al. (2016). ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology, 69, 225-234.

Type of Material Journal Article
Format Periodical
Cost to Access None.
Language English
Conditions for Use

© 2016 The Authors

Have you used this resource? Share your story!