GRADE for appraising the quality of evidence and strength of recommendations

A Summary of

Guyatt, G.H., Oxman, A.D., Vist, G.E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(26), 924-926.

Relevance for Public Health

The GRADE approach is useful when answering questions about interventions and when evidence-informed decision making is needed and recommendations are being produced. Originally developed for clinical interventions, the GRADE approach is designed to assess the quality of evidence for both randomized controlled trials and observational studies.

Description

The GRADE framework (Grading of Recommendations Assessment, Development and Evaluation) applies a rating of quality (synonyms: confidence, certainty) of evidence and a grading of strength of recommendations for systematic reviews and clinical practice guidelines. For the quality of evidence, randomized trials start as high quality, and observational studies as low (in the high, moderate, low, very low classification). Quality can be rated down for risk of bias, inconsistency, indirectness, imprecision and publication bias or rated up, most frequently for large or very large effect. Knowing the quality of evidence can help formulate conclusions, and recommendations are then graded as strong or weak. The GRADE system recognizes that the relationship between evidence and recommendations is influenced by other factors beyond the evidence, including patient values and preferences and the balance of desirable and undesirable effects.

Accessing the Method/Tool


Language(s)

English

Format(s)

Periodical

Cost

None

Implementing the Method/Tool


Time for Participation/Completion

Information not available

Additional Resources and/or Skills Needed for Implementation

Yes. Standard risk of bias appraisal tool.

Steps for Using Method/Tool

The GRADE system classifies the quality of evidence and gives an overall rating of very low quality of evidence, low quality of evidence, moderate quality of evidence or high quality of evidence.

The quality of evidence rating depends on a summary of different factors, one of which is risk of bias. A standard appraisal tool can be used to determine the risk of bias present in individual studies gathered from a systematic review. However, it is important to note that GRADE addresses rating the quality of a body of evidence rather than individual studies.

Factors in the assessment of the quality of evidence include the following:

  • Risk of bias/study limitations
  • Inconsistency of results
  • Indirectness of evidence
  • Imprecision
  • Reporting bias
  • Magnitude of effect
  • Dose-response gradient
  • Direction of plausible biases

When making recommendations based on a systematic review, additional issues beyond the quality of evidence should also be considered. This includes the balance of desirable effects (e.g., increased “health”) and undesirable effects (e.g., reduced quality of life), along with patient preferences and values (e.g., would all patients choose the intervention?). When the desirable/undesirable effects are clearly “black and white,” that is, one substantially outweighs the other, then a strong recommendation can be made. When there is a close balance between the two (desirable/undesirable effects) or when low quality of evidence forms the basis, then weak recommendations are usually made.

Factors that affect the strength of recommendation include the following:

  • Quality of evidence
  • Uncertainty on balance between desirable and undesirable effects
  • Uncertainty or variability in patient values and preferences
  • Uncertainty about the use of resources

Who is involved

The GRADE system of rating the quality of evidence and recommendation formulation depends on the staff conducting the systematic review or meta-analysis, which could also include a librarian, managers, senior leaders, panels of experts or clinicians, and community representatives or patients.

Conditions for Use

© 2017 BMJ Publishing Group Ltd.

Evaluation and Measurement Characteristics


Evaluation

Has been evaluated.

Has advantages over other rating systems including its transparency, explicitness, comprehensiveness and usefulness.

Validity

Not applicable

Reliability

Not applicable

Methodological Rating

Not applicable

Method/Tool Development


Developer(s)

The GRADE working group, including the following:
G Guyatt
Andrew Oxman
Gunn Vist
Regina Kunz
Yngve Falck-Ytter
Pablo Alonso-Coello
Holger Schunemann

Method of Development

The GRADE system was developed by a widely representative group of international guideline developers.

Release Date

2008

Contact Person/Source

G Guyatt
CLARITY Research Group
McMaster University
1200 Main St West
Hamilton, ON L8N 3Z5
Email: guyatt@mcmaster.ca

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Resources

Title of Primary Resource GRADE: An emerging consensus on rating quality of evidence and strength of recommendations
File Attachment None
Web-link
Reference

Guyatt, G.H., Oxman, A.D., Vist, G.E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(26), 924-926.

Type of Material Journal article
Format Periodical
Cost to Access None.
Language English
Conditions for Use

© 2017 BMJ Publishing Group Ltd.

Title of Supplementary Resource GRADE: What is “quality of evidence” and why is it important to clinicians?
File Attachment None
Web-link
Reference

Guyatt, G.H., Oxman, A.D., Vist, G.E., Kunz, R., Falck-Ytter, Y., Schunemann, H.J. (2008). GRADE: What is “quality of evidence” and why is it important to clinicians? BMJ, 336(3), 995-998.

Type of Material Journal article
Format Periodical
Cost to Access None.
Language English
Conditions for Use

© 2017 BMJ Publishing Group Ltd.

Have you used this resource? Share your story!