Quality Assessment Tool for Quantitative Studies

A summary of Effective Public Health Practice Project. (1998). Quality Assessment Tool For Quantitative Studies. Hamilton, ON: Effective Public Health Practice Project. Available from: http://www.ephpp.ca/index.html

How to cite this NCCMT summary:

National Collaborating Centre for Methods and Tools (2008). Quality Assessment Tool for Quantitative Studies. Hamilton, ON: McMaster University. (Updated 13 April, 2010). Retrieved from http://www.nccmt.ca/registry/view/eng/14.html.

Keywords: Appraise

Tool Knowledge Doing

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Relevance for Public Health

This instrument was developed for use in public health, and can be applied to articles of any public health topic area. Some of these areas include the promotion of family and sexual health and the prevention of chronic disease, injuries and substance misuse. Various types of public health professionals would find this tool relevant to utilize sources of high quality literature to support the decision-making process, especially when designing, implementing and evaluating public health programs and policy.

Description

The “Quality Assessment Tool For Quantitative Studies” developed by the Effective Public Health Practice Project (EPHPP) is a tool for doing knowledge synthesis. This instrument, along with a user manual, provides a standardized means to assess study quality and develop recommendations for study findings. The quality appraisal tool was developed by the Effective Public Health Practice Project (EPHPP) as a discrete step within the systematic review process. The EPHPP began in 1998 with funding from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The goal of the project was to address the lack of high quality evidence to support public health interventions known as Ontario Mandatory Health Programs and Service Guidelines (MHPSG, 1997) as outlined in Ontario’s standards for public health. This standardized tool was developed in order to provide high quality systematic reviews to address the public health sector’s need for evidence to support practice. Final results of the “Quality Assessment Tool For Quantitative Studies” lead to an overall methodological rating of strong, moderate or weak in eight sections: 1. selection bias 2. study design 3. confounders 4. blinding 5. data collection methods 6. withdrawals and dropouts 7. intervention integrity 8. analysis A reviewer’s dictionary is provided to assist reviewers and maintain standardized results.

Implementing the Method/Tool
Steps for Using Method/Tool

The tool can be applied after a research article has been selected for critical appraisal. After the article has been reviewed, the “Quality Appraisal Tool For Quantitative Studies” is completed according to the instructions provided in the accompanying companion manual.

Who is involved

This tool requires individuals to have specialized expertise in critical appraisal methods. Having knowledge of public health interventions and the ability to synthesize results into applicable practice recommendations is also important.

Conditions for Use

A standardized approach for the assessment of qualitative research has not been completed. Currently, the EPHPP uses qualitative research in order to explain and interpret the outcomes and interventions under review.

Evaluation and Measurement Characteristics
Evaluation
Has been evaluatedHas been evaluated.

This tool has been evaluated for content and initial construct validity and inter-rater reliability.

Validity
Validity properties meet accepted standardsValidity properties meet accepted standards.

Thomas, Ciliska, Dobbins & Miccucci (2004) assessed content validity using an iterative process. An expert group was formed to review the tool for consistency of coding, interpretation, and examination of evidence tables and/or recommendations. The validation process involved assessing the instrument’s content and categories for clarity, completeness and relevance, and an overall comparison with similar types of tools.

Reliability
Reliability properties meet accepted standardsReliability properties meet accepted standards.

Test-retest reliability on this tool was calculated over two occasions using a random selection of primary studies and two reviewers. Results indicated that agreement for the two reviewers was Kappa 0.74 and Kappa 0.61 (Thomas, et al., 2004).

Methodological Rating
StrongStrong
Method/Tool Development
Developer(s)

The Effective Public Health Practice Project (EPHPP), McMaster University, Faculty of Health Sciences, School of Nursing. The original authors are B.H. Thomas, Associate Professor, McMaster University School of Nursing, D. Ciliska, Professor, McMaster University School of Nursing, M. Dobbins, Associate Professor, McMaster University School of Nursing, and S. Micucci, Project Coordinator.

Method of Development

This tool was developed by reviewing available instruments in peer review literature and devising a list of components. Content validity was assessed by circulation to six experts in appraising study quality; comments were incorporated. A pilot version tested ten primary studies by four experts in critical appraisal and community health. The tool was further clarified with feedback from the reviewers. A detailed dictionary was developed, as well as guidelines for assessing the overall strength of the articles.

Release Date
1998
Contact Person/Source

Project Coordinator Effective Public Health Practice Project 1685 Main St. W., Suite 302 Hamilton Ontario L8S 1G5 905-525-9140 www.ephpp.ca

Resources

Title of Primary Resource
Quality Assessment Tool For Quantitative Studies
File Attachment
QATool.pdf
Web-link
Reference

Effective Public Health Practice Project. (1998). Quality Assessment Tool For Quantitative Studies. Hamilton, ON: Effective Public Health Practice Project. Available from: http://www.ephpp.ca/index.html

Type of Material
Tool - Data Abstraction
Format
On-line Access
Cost to Access
None 
Language
English
Conditions for Use
Not specified 

Title of Supplementary Resource
Dictionary for the Effective Public Health Practice Project Quality Assessment Tool For Quantitative Studies
File Attachment
QADictionary.pdf
Web-link
Reference

Effective Public Health Practice Project. (1998). Dictionary for the Effective Public Health Practice Project Quality Assessment Tool For Quantitative Studies. Retrieved October 2008 from above web address. Corresponding article of interest: Thomas, B.H., Ciliska, D., Dobbins, M., & Micucci, S. (2004). A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing, 1(3), 176-184.

Type of Material
Tool - Supplementary Material
Format
On-line Access
Cost to Access
None 
Language
English
Conditions for Use
Not specified 

Title of Supplementary Resource
A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions.
File Attachment
None
Web-link
Reference

Thomas, B.H., Ciliska, D., Dobbins, M., & Micucci, S. (2004). A process for systematically reviewing the literature: Providing the research evidence for public health nursing interventions. Worldviews on Evidence-Based Nursing, 1(3), 176-184.

Type of Material
Method - Doing Synthesis
Format
Periodical
Cost to Access
 
Language
English
Conditions for Use
Copyright © 2004 Worldviews on Evidence-Based Nursing 


Have you used this resource? Share your story!