Higgins, J.P.T., Sterne, J.A.C., Savović, J., Page, M.J., Hróbjartsson, A., Boutron, I., Reeves, B., Eldridge, S. (2016). A revised tool for assessing risk of bias in randomized trials. In: Chandler, J., McKenzie, J., Boutron, I., Welch, V. (editors). Cochrane Methods. Cochrane Database of Systematic Reviews, 10 (Suppl 1). dx.doi.org/10.1002/14651858.CD201601.
Description
A risk of bias tool was created to assess the potential for bias in randomized trials included in systematic reviews and meta-analyses. In 2005, the Cochrane Collaboration decided to improve upon the existing tools at the time, mainly scales and checklists, with and without scoring respectively. The original risk of bias (RoB) tool was launched in 2008, and its most current update, version 2.0, is summarized here.
The RoB tool 2.0 provides a framework for considering the risk of bias in the findings of any type of randomized trial. The current tool (Oct 20, 2016) is based on the potential sources of biases that may arise in clinical trials.
The RoB tool 2.0 is for use with individual randomized trials, randomized parallel group trials, cluster-randomized trials and randomized cross-over trials.
The following five domains are to be used in the assessment of the risk of bias in individual randomized trials:
- Bias arising from the randomization process
- Bias due to deviations from intended interventions
- Bias due to missing outcome data
- Bias in measurement of the outcome
- Bias in selection of the reported result
Access the Tool
Language(s)
English
Format(s)
On-line Access
Cost
None
Time for Participation/Completion
30 to 60 minutes
Additional Resources and/or Skills Needed for Implementation
The efficiency of the use of the tool will likely depend on the user's background knowledge of bias and related terms and the ability to apply them within domains, as well as the user's content knowledge for the given public health topic area under study.
The background information and detailed guidance for using the RoB 2.0 tool is provided in the Supplemental Resources, and on the website: http://www.riskofbias.info/
Steps for Using Method/Tool
The RoB tool 2.0 involves the user making a series of judgements about the extent of bias arising from the conduct of the trial. The tool focuses on the risk of bias rather than reporting characteristics. The tool thus allows readers of the review to decide whether they agree or not with the judgements made in assessing the risk of bias for trials.
A key feature is the inclusion of signalling questions within each domain of bias. These questions are used to help make judgements about the risk of bias. The response options for signalling questions are:
- Yes
- Probably yes
- Probably no
- No
- No information
The signalling questions should be answered independently from one another, after which domain-level judgements about the risk of bias are determined. The risk of bias judgements are:
- Low risk of bias
- Some concerns
- High risk of bias
The key is consistency, ensuring domain-level judgements about risk of bias mean the same across the five domains and thus constitute a similar view on the trustworthiness of the trial conduct. The five domains include:
- Bias arising from the randomization process
- Bias due to deviations from intended interventions
- Bias due to missing outcome data
- Bias in measurement of the outcome
- Bias in selection of the reported result
Finally, an overall risk of bias judgement can be made for a specific outcome for a given trial, using the following qualifiers:
- Low risk of bias: the study is judged to be at low risk of bias for all domains for this result.
- Some concerns: the study is judged to have some concerns of bias in at least one domain for this result.
- High risk of bias: the study is judged to be at high risk of bias in at least one domain for this result OR the study is judged to have some concerns for multiple domains in a way that substantially lowers confidence in the result.
In summary, the steps to using the tool include:
- Determining the risk of bias in each of the five domains
- Determining the risk of bias across all five domains for a specific outcome
Additional features of the tool are further described in the Manual (See Supplementary Resources). The developer's website also provides slightly modified tools depending on the trial design. The tool reviewed here (primary resource) is for individually randomized, parallel group trials.
Who is involved
Systematic reviewers use the tool, but systematic review readers can also see the judgements made by the reviewers.
Conditions for Use
© 2016 by the authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
Evaluation
Has been evaluated.
The original tool has been evaluated (see Supplemental Resources, Higgins et al., 2011).
Validity
Validity properties meet accepted standards.
Reliability
Reliability properties meet accepted standards.
Reliability has not been extensively studied (BMJ), however some items are more difficult to assess by users. Ongoing guidance and training materials have been developed to help improve reliability.
Methodological Rating
Not applicable
Developer(s)
MRC Network of Hubs for Trials Methodology Research
Website: http://www.riskofbias.info/
Method of Development
In 2005, a group of experts, including statisticians, epidemiologists and others, developed the tool and used informal consensus to refine it. Specific criteria for the tool were developed by this group, and further refinements were made after pilot work of the tool at meetings and workshops. The first version of the tool was launched in 2008, with refinements made in 2016.
Release Date
2016
Contact Person/Source
Julian PT Higgins
MRC Biostatistics Unit
Institute of Public Health
Cambridge, UK CB2 0SR
Email: julian.higgins@mrc-bsu.cam.ac.uk.
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.
