AMSTAR 2: A Critical Appraisal Tool for Systematic Reviews that Include Randomised and/or Non-Randomised Studies of Healthcare Interventions

Shea B. J., Reeves B. C., Wells G., Thuku M., Hamel C., Moran J. et al. (2017). AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358. https://doi.org/10.1136/bmj.j4008

Canadian Template available Evaluated/validated

Description

AMSTAR (A Measurement Tool to Assess Systematic Reviews), originally created in 2007, dealt with critical appraisal of systematic reviews. Since then there has been a shift in focus toward evidence-informed decision making using a broader scope of evidence, including observational studies or non-randomized studies. AMSTAR 2, an update to the original AMSTAR tool, was created to address the contemporary need for clinical and policy decision making that accounts for real-world observational evidence.

Steps for Using Method/Tool

AMSTAR 2 consists of 16 items in total, 10 of which are from the original AMSTAR. Each item allows for the following response options: Yes, Partial Yes, or No. AMSTAR 2 is not intended to be scored.

The following 16 items are covered in AMSTAR 2:

  1. Did the research questions and inclusion criteria for the review include the components of PICO?
  2. Did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol?
  3. Did the review authors explain their selection of the study designs for inclusion in the review?
  4. Did the review authors use a comprehensive literature search strategy?
  5. Did the review authors perform study selection in duplicate?
  6. Did the review authors perform data extraction in duplicate?
  7. Did the review authors provide a list of excluded studies and justify the exclusions?
  8. Did the review authors describe the included studies in adequate detail?
  9. Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?
  10. Did the review authors report on the sources of funding for the studies included in the review?
  11. If meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?
  12. If meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis?
  13. Did the review authors account for RoB in primary studies when interpreting/discussing the results of the review?
  14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?
  15. If they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?
  16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?

Evaluation

Shea, B., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., et al. (2017). AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008.

Validity

Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., et al. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed.), 358, j4008. https://doi.org/10.1136/bmj.j4008

Reliability

Lorenz, R.C., Matthias, K., Pieper, D., Wegewitz, U., Morche, J., Nocon, M., et al. (2020). AMSTAR 2 overall confidence rating: Lacking discriminating capacity or requirement of high methodological quality? Journal of Clinical Epidemiology, 119, 142-144. https://doi.org/10.1016/j.jclinepi.2019.10.006.

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

We have provided the resources and links as a convenience and for informational purposes only; they do not constitute an endorsement or an approval by McMaster University of any of the products, services or opinions of the external organizations, nor have the external organizations endorsed their resources and links as provided by McMaster University. McMaster University bears no responsibility for the accuracy, legality or content of the external sites.

Have you used this resource? Share your story!