Shea, B., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., et al. (2017). AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008.
Relevance for Public Health
AMSTAR (A Measurement Tool to Assess Systematic Reviews) was the original critical appraisal tool for systematic reviews of randomized studies. AMSTAR 2 is an update that accounts for both randomized studies and observational (non-randomized) studies. It is practical for use for health professionals and public health practitioners involved with program or policy decision making. The goal for AMSTAR 2 is to increase its applicability to include critical appraisal of a broad range of study designs on healthcare interventions used in systematic reviews.
AMSTAR (A Measurement Tool to Assess Systematic Reviews), originally created in 2007, dealt with critical appraisal of systematic reviews. Since then there has been a shift in focus toward evidence-informed decision making using a broader scope of evidence, including observational studies or non-randomized studies. AMSTAR 2, an update to the original AMSTAR tool, was created to address the contemporary need for clinical and policy decision making that accounts for real-world observational evidence.
Accessing the Method/Tool
Implementing the Method/Tool
Time for Participation/Completion
10 to 30 minutes
Additional Resources and/or Skills Needed for Implementation
Steps for Using Method/Tool
AMSTAR 2 consists of 16 items in total, 10 of which are from the original AMSTAR. Each item allows for the following response options: Yes, Partial Yes, or No. AMSTAR 2 is not intended to be scored.
The following 16 items are covered in AMSTAR 2:
- Did the research questions and inclusion criteria for the review include the components of PICO?
- Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?
- Did the review authors explain their selection of the study designs for inclusion in the review?
- Did the review authors use a comprehensive literature search strategy?
- Did the review authors perform study selection in duplicate?
- Did the review authors perform data extraction in duplicate?
- Did the review authors provide a list of excluded studies and justify the exclusions?
- Did the review authors describe the included studies in adequate detail?
- Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?
- Did the review authors report on the sources of funding for the studies included in the review?
- If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?
- If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?
- Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review?
- Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?
- If they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?
- Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?
Who is involved
AMSTAR 2 is practical for use by individuals involved in appraising research evidence from a clinical, public health or policy standpoint. Managers and decision makers should also be familiar with the components of the appraisal tool when they are using systematic reviews to inform the basis of evidence-informed decision making.
Conditions for Use
Evaluation and Measurement Characteristics
Has been evaluated.
The tool has been evaluated.
Validity properties meet accepted standards.
The validity of the tool has been evaluated.
Reliability properties meet accepted standards.
The reliability of the tool has been evaluated using three pairs of raters across three sets of reviews. Kappa scores were in the acceptable range.
Method of Development
The original AMSTAR tool was developed in 2007. Since then, a need to update the tool was identified based on published critiques, user feedback received at workshops, and developments in the science of systematic reviews. The original AMSTAR tool was created based on a scoping review of rating instruments available at the time; the use of analysis validation techniques; and pilot testing, including reliability and user testing, external validation and content expert review.
AMSTAR 2 was developed based on information from an expert working group, review of current appraisal tools, review of extensive user feedback of the original tool, and a brainstorming session of ideas to create a list of items to consider. The draft instrument underwent pilot testing at workshops, feedback and further modifications, inter-rater reliability testing, and usability testing.
School of Epidemiology and Public Health, Faculty of Medicine
University of Ottawa
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.