A Summary of
Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JPT. ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ 2016; 355; i4919.
Relevance for Public Health
The ROBINS-I tool can be applied to any public health research question that examines the intervention effect of a public health intervention. This includes a number of observational study designs such as cohort studies, quasi-randomized trials and other concurrently controlled studies, as well as case-control studies, cross-sectional studies, interrupted time series and controlled before-after studies.
The ROBINS-I tool and manual (Risk Of Bias In Non-randomized Studies - of Interventions) can be used to evaluate the risk of bias in non-randomized studies of interventions (NRSI) that compare the health effects of two or more interventions.
The ROBINS-I tool is an update to the previous ACROBAT-NRSI (A Cochrane Risk of Bias Assessment Tool for Non-Randomized Studies of Interventions). The tool is based on the Cochrane "risk of bias" (RoB) tool for randomized trials, and also builds on related tools such as QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies).
The ROBINS-I is a tool to rate risk of biases in a non-randomized study of interventions. It outlines seven domains where biases might occur. The first two domains of bias occur in the "pre-intervention" phase, one domain of bias occurs in the "at intervention" phase and four domains of bias occur in the "post-intervention" phase. The specific bias domains are as follows:
- Bias due to confounding
- Bias in selection of participants into the study
- Bias in classification of interventions
- Bias due to deviations from intended interventions
- Bias due to missing data
- Bias in measurement of outcomes
- Bias in selection of the reported result
The tool also offers a number of explanations for ratings, terms, definitions and theory.
Accessing the Method/Tool
Implementing the Method/Tool
Time for Participation/Completion
1 to 2 hours
This tool requires an intense review of study methodology. With an increasing number of studies, the time to implement this tool increases.
Additional Resources and/or Skills Needed for Implementation
The ease with which this tool can be used coincides with knowledge and theory of study design.
Steps for Using Method/Tool
The users begin by outlining signalling questions, explanations and response options for each domain of bias. The typical response options are listed below, with notable exceptions outlined in the manual:
- Probably yes
- Probably no
- No information
Then, users judge each domain to determine if it falls within a risk of bias category, including:
- Low risk of bias
- Moderate risk of bias
- Serious risk of bias
- Critical risk of bias
The tool then provides an approach to map the RoB judgements within domains to a single RoB judgement across domains for an outcome. In addition, the tool provides a framework for reaching an overall RoB judgement across multiple outcomes in a study.
Therefore, the overall use of ROBINS-I includes:
- RoB judgements for signalling questions
- RoB judgements within domains
- RoB judgements across domains for one outcome
- RoB judgements across domains for more than one outcome
Who is involved
Anyone who reads public health research.
Conditions for Use
© 2016 by the authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
Evaluation and Measurement Characteristics
Has been evaluated.
The tool was piloted with a working group that implemented the use of the tool across a number of topic areas using NRSI, and the revisions needed were addressed. Further pilot testing was conducted and feedback was collected with the target audience of first-time users using a telephone interview format. The tool was then launched in 2014 in a workshop format and user feedback from this event and additional training events in 2015 lead to final tool refinement.
Cochrane Methods Bias
Method of Development
The development of the tool is described in detail in the corresponding BMJ publication (Supplemental materials). In brief, the tool was developed over three years, largely by expert consensus, and following the seven principles previously described for assessing risk of bias in clinical trials.
School of Social and Community Medicine
University of Bristol
Bristol, UK BS8 2PS
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.