Guyatt, G.H., Oxman, A.D., Vist, G.E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., et al. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(26), 924-926.
Relevance for Public Health
The GRADE approach is useful when answering questions about interventions and when evidence-informed decision making is needed and recommendations are being produced. Originally developed for clinical interventions, the GRADE approach is designed to assess the quality of evidence for both randomized controlled trials and observational studies.
The GRADE framework (Grading of Recommendations Assessment, Development and Evaluation) applies a rating of quality (synonyms: confidence, certainty) of evidence and a grading of strength of recommendations for systematic reviews and clinical practice guidelines. For the quality of evidence, randomized trials start as high quality, and observational studies as low (in the high, moderate, low, very low classification). Quality can be rated down for risk of bias, inconsistency, indirectness, imprecision and publication bias or rated up, most frequently for large or very large effect. Knowing the quality of evidence can help formulate conclusions, and recommendations are then graded as strong or weak. The GRADE system recognizes that the relationship between evidence and recommendations is influenced by other factors beyond the evidence, including patient values and preferences and the balance of desirable and undesirable effects.
Accessing the Method/Tool
Implementing the Method/Tool
Time for Participation/Completion
Information not available
Additional Resources and/or Skills Needed for Implementation
Yes. Standard risk of bias appraisal tool.
Steps for Using Method/Tool
The GRADE system classifies the quality of evidence and gives an overall rating of very low quality of evidence, low quality of evidence, moderate quality of evidence or high quality of evidence.
The quality of evidence rating depends on a summary of different factors, one of which is risk of bias. A standard appraisal tool can be used to determine the risk of bias present in individual studies gathered from a systematic review. However, it is important to note that GRADE addresses rating the quality of a body of evidence rather than individual studies.
Factors in the assessment of the quality of evidence include the following:
- Risk of bias/study limitations
- Inconsistency of results
- Indirectness of evidence
- Reporting bias
- Magnitude of effect
- Dose-response gradient
- Direction of plausible biases
When making recommendations based on a systematic review, additional issues beyond the quality of evidence should also be considered. This includes the balance of desirable effects (e.g., increased “health”) and undesirable effects (e.g., reduced quality of life), along with patient preferences and values (e.g., would all patients choose the intervention?). When the desirable/undesirable effects are clearly “black and white,” that is, one substantially outweighs the other, then a strong recommendation can be made. When there is a close balance between the two (desirable/undesirable effects) or when low quality of evidence forms the basis, then weak recommendations are usually made.
Factors that affect the strength of recommendation include the following:
- Quality of evidence
- Uncertainty on balance between desirable and undesirable effects
- Uncertainty or variability in patient values and preferences
- Uncertainty about the use of resources
Who is involved
The GRADE system of rating the quality of evidence and recommendation formulation depends on the staff conducting the systematic review or meta-analysis, which could also include a librarian, managers, senior leaders, panels of experts or clinicians, and community representatives or patients.
Conditions for Use
© 2017 BMJ Publishing Group Ltd.
Evaluation and Measurement Characteristics
Has been evaluated.
Has advantages over other rating systems including its transparency, explicitness, comprehensiveness and usefulness.
The GRADE working group, including the following:
Method of Development
The GRADE system was developed by a widely representative group of international guideline developers.
CLARITY Research Group
1200 Main St West
Hamilton, ON L8N 3Z5
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.