Description
The “Guide to Community Preventive Services Data Abstraction Form” is a tool to aid in knowledge synthesis. A critical appraisal tool can reduce inconsistencies by providing a standardized instrument and procedure to collect and evaluate data from individual research studies.
This tool was developed to address a lack of consistency in approaches to synthesize relevant public health research. This instrument is applicable to numerous areas in public health as it provides a template to:
- organize study information;
- extract key data summaries;
- appraise the quality of the study;
- discuss the impact of the results and intervention.
Using this tool can help public health practitioners, decision-makers and researchers examine a study and make decisions about the quality of evidence to support or refute a proposed intervention.
This instrument consists of three sections:
- Classification of the study (e.g. the study design, the nature of the intervention and its components, and the primary outcome measures);
- Intervention description (e.g. theoretical basis, implementation, clinical area administered, treatment of the comparison group, evaluation of key study characteristics, and the extraction of numerical data for summary table development);
- Study quality assessments (e.g. evaluate the study design, methods and analysis, and interpret results)
The tool is constructed as a 26-page booklet with instructions for use comprising 26 questions related to study content and 23 questions for quality execution of the study.
Steps for Using Method/Tool
Appraisers review a study design algorithm, which allows each study to be included on one specified study design category. Relevant outcomes are determined for abstraction. Appraisers document the methods and the results of the study, including the intervention, theories relating to the intervention, the organization implementing the intervention, and a description of the intervention.
Evaluation
Over 40 reviewers have provided feedback and suggestions for improvement of the critical appraisal tool. This included improving instructions for use of the tool and the overall format of the tool.
Validity
Zaza et al. (2000) assessed face and content validity using an iterative process. An expert group was formed to review the tool for consistency of coding, interpretation, and examination of evidence tables and/or recommendations. The validation process involved assessing the instrument’s content and categories for clarity, completeness and relevance, and an overall comparison to similar types of tools.
Reliability
Zaza et al. (2000) describe the inter-rater reliability process; yet, the authors of the tool do not provide overall scores. The authors acknowledge that reliability testing was completed on an earlier version of the tool, but not on the final version of the tool. However, special attention was paid to questions with lower reliability scores when revising the tool. Based on what is currently found in the published literature on this data abstraction tool, there is not enough evidence provided to confirm reliability according to Registry of Knowledge Translation standards.
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.
We have provided the resources and links as a convenience and for informational purposes only; they do not constitute an endorsement or an approval by McMaster University of any of the products, services or opinions of the external organizations, nor have the external organizations endorsed their resources and links as provided by McMaster University. McMaster University bears no responsibility for the accuracy, legality or content of the external sites.
