Effective Public Health Practice Project. (1998). Quality Assessment Tool For Quantitative Studies. Hamilton, ON: Effective Public Health Practice Project. Available from: https://merst.ca/ephpp/
Relevance for Public Health
This instrument was developed for use in public health, and can be applied to articles of any public health topic area. Some of these areas include the promotion of family and sexual health and the prevention of chronic disease, injuries and substance misuse. Various types of public health professionals would find this tool relevant to utilize sources of high quality literature to support the decision-making process, especially when designing, implementing and evaluating public health programs and policy.
The “Quality Assessment Tool for Quantitative Studies” developed by the Effective Public Health Practice Project (EPHPP) is a tool for doing knowledge synthesis. This instrument, along with a user manual, provides a standardized means to assess study quality and develop recommendations for study findings. The quality appraisal tool was developed by the Effective Public Health Practice Project (EPHPP) as a discrete step within the systematic review process.
The EPHPP began in 1998 with funding from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The goal of the project was to address the lack of high quality evidence to support public health interventions known as Ontario Mandatory Health Programs and Service Guidelines (MHPSG, 1997) as outlined in Ontario’s standards for public health.
This standardized tool was developed in order to provide high quality systematic reviews to address the public health sector’s need for evidence to support practice. Final results of the “Quality Assessment Tool for Quantitative Studies” lead to an overall methodological rating of strong, moderate or weak in eight sections:
- selection bias;
- study design;
- data collection methods;
- withdrawals and dropouts;
- intervention integrity;
A reviewer’s dictionary is provided to assist reviewers and maintain standardized results.
Accessing the Method/Tool
Implementing the Method/Tool
Time for Participation/Completion
30 to 60 minutes
The authors of this tool suggest that it takes an individual approximately 30 minutes to complete the “Quality Assessment Tool for Quantitative Studies”.
Additional Resources and/or Skills Needed for Implementation
A companion document "Dictionary for the Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies" is provided to assist with scoring the tool. Potentially, some expertise might be required when analyzing results. See Resources.
Steps for Using Method/Tool
The tool can be applied after a research article has been selected for critical appraisal. After the article has been reviewed, the “Quality Appraisal Tool for Quantitative Studies” is completed according to the instructions provided in the accompanying companion manual.
Who is involved
This tool requires individuals to have specialized expertise in critical appraisal methods. Having knowledge of public health interventions and the ability to synthesize results into applicable practice recommendations is also important.
Conditions for Use
A standardized approach for the assessment of qualitative research has not been completed. Currently, the EPHPP uses qualitative research in order to explain and interpret the outcomes and interventions under review.
Evaluation and Measurement Characteristics
Has been evaluated.
This tool has been evaluated for content and initial construct validity and inter-rater reliability.
Validity properties meet accepted standards.
Thomas, Ciliska, Dobbins & Micucci (2004) assessed content validity using an iterative process. An expert group was formed to review the tool for consistency of coding, interpretation, and examination of evidence tables and/or recommendations. The validation process involved assessing the instrument’s content and categories for clarity, completeness and relevance, and an overall comparison with similar types of tools.
Reliability properties meet accepted standards.
Test-retest reliability on this tool was calculated over two occasions using a random selection of primary studies and two reviewers. Results indicated that agreement for the two reviewers was Kappa 0.74 and Kappa 0.61 (Thomas, et al., 2004).
The Effective Public Health Practice Project (EPHPP)
Method of Development
This tool was developed by reviewing available instruments in peer review literature and devising a list of components. Content validity was assessed by circulation to six experts in appraising study quality; comments were incorporated. A pilot version tested ten primary studies by four experts in critical appraisal and community health. The tool was further clarified with feedback from the reviewers. A detailed dictionary was developed, as well as guidelines for assessing the overall strength of the articles.
The Effective Public Health Practice Project (EPHPP)
1685 Main St W., Suite 302
Hamilton, ON L8S 1G5
Phone: (905) 525-9140
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.