Health Evidence™ Quality Assessment Tool

A Summary of

Relevance for Public Health

The quality appraisal tool can be used to appraise systematic reviews of public health interventions. Specifically, the tool would be useful for program managers who use research evidence to make decisions about program development or program de-implementation. The tool can also be used as a teaching tool for critical appraisal.

Description

Health Evidence™ is a free online registry of systematic review-level evidence evaluating the effectiveness of public health and health promotion interventions. This website is one component of a larger knowledge translation resource.

The goal is to support public health decision makers working in program planning and policy to engage in evidence-informed decision making. Monthly searches of the published literature are done to identify public health-relevant systematic reviews. Two independent raters critically appraise all reviews that meet relevance criteria (review article; relevant to public health practice; focuses on effectiveness of an intervention; reports evidence on health outcomes; describes search strategy), and create a summary statement for selected high-quality reviews.

The “Health Evidence™ Quality Assessment Tool – Review Articles” is a tool developed by Health Evidence™ and is used for appraising systematic reviews on the effectiveness of public health interventions. All included systematic reviews then become part of the Health Evidence™ repository.

Accessing the Method/Tool


Language(s)

  • English
  • French

Format(s)

On-line Access

Cost

None

Implementing the Method/Tool


Time for Participation/Completion

30 to 60 minutes

Additional Resources and/or Skills Needed for Implementation

Yes. The time to use the tool may depend on the knowledge of study design concepts, systematic review methodology, and related definitions.

Steps for Using Method/Tool

The tool is made of 10 questions, each dealing with a different part of critical appraisal. In brief, the 10 questions address the following critical appraisal key elements;

Q1: Research question

Q2: Inclusion/exclusion criteria

Q3: Search strategy

Q4: Search dates

Q5: Level of evidence

Q6: Methodological quality

Q7: Results

Q8: Analysis

Q9: Appropriateness of analysis methods

Q10: Interpretation

Each question requires a final answer of YES or NO, where a yes answer is assigned one point and a no answer, zero points. After all 10 questions are answered by the reviewer, a final score is calculated. Based on the final score, the following qualifiers are used to provide an overall rating of the systematic review.

Quality assessment rating:

Strong: total score of 8 to 10

Moderate: total score of 5 to 7

Weak: total score of 4 or less

Each of the 10 questions listed above are further described in the tool in terms of the study design issues and rating decision making. The tool also describes any caveats (e.g., qualitative reviews, reviews of reviews).

Who is involved

Any systematic reviewers may be interested to use this tool, as well as managers and other staff interested in health-based decision making using evidence.

Conditions for Use

Not specified

Evaluation and Measurement Characteristics


Evaluation

Has been evaluated.

Shown to have acceptable reliability (see Supplemental Resources)

Validity

Not applicable

Reliability

Not applicable

Methodological Rating

Not applicable

Method/Tool Development


Developer(s)

Maureen Dobbins

Method of Development

Release Date

2000

Contact Person/Source

Maureen Dobbins
Health Evidence
McMaster University
175 Longwood Road S, Suite 210a
Hamilton, ON L8P 0A1
Email: dobbinsm@mcmaster.ca

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Resources

Title of Primary Resource Health Evidence™ Quality Assessment Tool – Review Articles
File Attachment None
Web-link
Reference
Type of Material Online tool
Format On-line Access
Cost to Access None.
Language English, French
Conditions for Use Not specified
Title of Supplementary Resource A knowledge management tool for public health: Health-evidence.ca
File Attachment None
Web-link
Reference

Dobbins, M., DeCorby, K., Robeson, P., Husson, H., Tirilis, D., & Greco, L. (2010). A knowledge management tool for public health: Health-evidence.ca. BMC Public Health, 10, 496.

Type of Material Journal
Format Periodical
Cost to Access None.
Language English
Conditions for Use Not specified

Have you used this resource? Share your story!