Quality Checklist for Health Care Intervention Studies

A summary of

Downs, S.H., & Black, N. (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. Journal of Epidemiology Community Health, 52, 377-384.

How to cite this NCCMT summary:

National Collaborating Centre for Methods and Tools (2008). Quality Checklist for Health Care Intervention Studies. Hamilton, ON: McMaster University. (Updated 02 August, 2017) Retrieved from http://www.nccmt.ca/resources/search/9.

Categories: Tool, Appraise

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Relevance for Public Health

Downs & Black (1998) do not explicitly refer to public health practices; however, they do state that the “Checklist for Measuring Quality” could be applied to any study detailing a health care intervention. Therefore, this instrument is applicable to the critique of studies involving public health interventions.

Description

The “Checklist for Measuring Quality” (Downs & Black, 1998) addresses the increasing demand for the use of evidence from systematic reviews and meta-analyses to support program and policy decisions in public health decision-making. This tool can be used to assess the quality of original or primary source research articles and to synthesize evidence from quantitative studies for public health practitioners, policy makers and decision-makers. The “Checklist for Measuring Quality” is discussed in a three page article and contains 27 ‘yes’-or-'no’ questions across five sections. The tool is easy to use and provides both an overall score for study quality and a numeric score out of a possible 30 points. The five sections include questions about:

1. Study quality (10 items) – the overall quality of the study;

2. External validity (3 items) – the ability to generalize findings of the study;

3. Study bias (7 items) – to assess bias in the intervention and outcome measure(s);

4. Confounding and selection bias (6 items) – to determine bias from sampling or group assignment; and

5. Power of the study (1 items) – to determine if findings are due to chance.

Administration of the tool can happen either within a systematic review process, or as a quality assessment tool for individual articles. Within a systematic review, a group guides the review and one or two people administer the tool.

Implementing the Method/Tool
Steps for Using Method/Tool

Not specified.

Who is involved

The “Checklist for Measuring Study Quality” is completed by those individuals who are interested and able to critically appraise research studies for quality and applicability to public health. Practitioners, decision-makers and/or policy makers who require fast, synthesized public health oriented research for program planning and decision-making may find this tool useful.

Conditions for Use

Copyright © 1998 Journal of Epidemiology and Community Health.

Evaluation and Measurement Characteristics
Evaluation
Has been evaluated.

This instrument has received pilot testing and a second testing after revisions.

Validity
Validity properties meet accepted standards.

Downs & Black (1998) describe assessing face, content and criterion validity. Face and content validity were assessed by three experienced reviewers, including two senior epidemiologists and a medical statistician. Modifications to the checklist were made based on the feedback of these reviewers. Criterion validity for this instrument was tested by comparing the total scores of the tool with another tool used only for randomized controlled trials. The resulting correlation between the two instruments was high (r=0.90).

Reliability
Reliability properties meet accepted standards.

Reliability testing was completed during the pilot testing phase and after development of the new, modified version of the tool. Internal consistency reliability scores were high (Cronbach alpha > 0.69) on all subscales, except for the external validity subscale (Cronbach alpha=0.54). Test-retest reliability scores were high for all subscales when randomized and non-randomized studies were assessed (r: 0.69-0.90). The correlation score for external validity was low (r=-0.37). Inter-rater reliability scores were high for most of the subscales (r>0.70) with the exception of external validity (r=-0.14). The variation across scores for the external validity subscale was thought to be related to the rating ability of reviewers who did not have health care backgrounds and who may not have appreciated the applicability of the interventions to health care.

Methodological Rating
Strong
Method/Tool Development
Developer(s)

University of London

Website: http://www.london.ac.uk/

Method of Development

The development of the “Checklist for Measuring Study Quality” was based on epidemiologic principles, reviews of study designs and existing tools for the assessment of randomized trials.

Release Date
1998
Contact Person/Source

Nick Black

University of London

Room 34, Keppel Street

London WC1E 7HT, UK

Phone: (020) 7927-2228

Fax: (020) 7580-8183

Email: nick.black@lshtm.ac.uk

Resources

Title of Primary Resource
The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions
File Attachment
None
Web-link
Reference

Downs, S.H., & Black, N. (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. Journal of Epidemiology Community Health, 52, 377-384.

Type of Material
Periodical
Format
On-line Access
Cost to Access
Language
English
Conditions for Use
Source document is copyrighted (©1998) by the Journal of Epidemiology and Community Health

Have you used this resource? Share your story!