How to assess the best available evidence when time is limited

A summary of

European Centre for Disease Prevention and Control (2011). Evidence-based methodologies for public health—How to assess the best available evidence when time is limited and there is lack of sound evidence. Stockholm, Sweden: European Centre for Disease Prevention. doi: 10.2900/58229. Available from: http://www.ecdc.europa.eu/en/publications/Publications/1109_TER_evidence_based_methods_for_public_health.pdf

How to cite this NCCMT summary:

National Collaborating Centre for Methods and Tools (2012). How to assess the best available evidence when time is limited. Hamilton, ON: McMaster University. (Updated 03 February, 2012) Retrieved from http://www.nccmt.ca/resources/search/115.

Categories: Method, Appraise, Synthesize, Consensus building

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Relevance for Public Health
Individuals appraising public health evidence to inform a rapid risk assessment process would benefit from the list of critical appraisal tools provided and the framework for conducting an assessment.
Description

Developed by the European Centre for Disease Prevention and Control (ECDC), this technical report, Evidence-based methodologies for public health, provides a framework for rapid risk assessment and evaluates critical appraisal tools for public health evidence. This method looks at how evidence-based methodologies can be applied in public health. Specifically, this report examines how to provide evidence-informed advice on public health issues when there is little evidence and when advice is sought within a short time frame. The report also discusses the need to adapt methods from evidence-based medicine for a public health setting, and the usefulness of critical appraisal tools to grade evidence.

Rapid risk assessment is a core public health function used to evaluate the risk to human health when an incident is confirmed as being of potential public health concern (ECDC, 2011). Rapid risk assessments should be based on the best available evidence to determine:

  • if a response is needed;
  • the urgency and magnitude of the response;
  • the design and selection of intervention; and
  • broader management of the incident.
Implementing the Method/Tool
Steps for Using Method/Tool

This method has five sections.

I: Background

Evidence-based public health includes two main areas:

  1. evidence of the effectiveness of interventions and how evidence can be applied to programs and policies
  2. defining health risks and risk assessment, where uncertainties in the decision-making process have an impact on risk assessment

There are three basic requirements for addressing uncertainties in risk assessments:

  1. Systematically identify and evaluate the sources of uncertainties.
  2. Evaluate their combined effect on the outcome of the assessment.
  3. Communicate this to decision-makers.

II: How to give evidence-based guidance when evidence is scarce and time is limited?

The risk assessment process consists of:

  • Stage 0: Preparatory
  • Stage 1: Threat detection/verification
  • Stage 2: Assessment of risk
  • Stage 3: Developing advice
  • Stage 4: Implementation
  • Stage 5: Monitoring and evaluation
To see tools for each stage, see Annex 2.

III: The usefulness of EBM grading tools for grading evidence and recommendations for the field of public health

This method assesses tools for grading the quality of evidence and the development of public health guidelines. The report provides brief descriptions of each organization's approach to developing guidelines and making recommendation:

  • World Health Organization – Subcommittee on the Use of Research Evidence (SURE)
  • GRADE, The Grading of Recommendations Assessment, Development and Evaluation Working Group, provides a framework to move from assessing evidence to making a recommendation. The GRADE approach is being used in different ways by other organizations to appraise evidence for guideline development. To see challenges on applying GRADE in public health settings, see Annex 5 in the appendix.
  • The National Institute for Health and Clinical Excellence (NICE)
  • The Scottish Intercollegiate Guidelines Network (SIGN) (see Annex 6)
  • Centers for Disease Control and Prevention (CDC)
  • National Health and Medical Research Council (NHMRC)
  • Centre for Evidence Based Medicine (CEBM)

The following tools for appraising evidence are discussed:

  • AGREE (Appraisal of Guidelines Research & Evaluation) tool to appraise guidelines (to see a summary statement on AGREE II tool in the Registry, click here)
  • CONSORT (Consolidated Standards of Reporting Trial) statement for reporting randomized controlled trials
  • TREND (Transparent Reporting of Evaluations with Non-randomized Designs) statement – a checklist to guide reporting of non-randomized controlled trials of public health intervention.
(For more information on appraising evidence on public health interventions, see a module developed by NCCMT - click here)

IV: Assessing and assuring quality to guideline development for health protection and control of communicable diseases

The report provides a description of the AGREE II instrument and its limitations, and discusses its application for communicable disease guidelines. Also, the Guideline Evaluation Tool (GET5), a shorter and adapted version of the AGREE II, is discussed (see Annex 7 and 8 for the GET5 tool).

V: EBM methods for public health—the use of consensus methods

Consensus methods can be used in different ways for public health guidance. For example, when:

  • defining the key questions and selecting relevant data sources and type of evidence needed;
  • assessing which outcomes are essential when grading evidence and informing the strength of the recommendation;
  • translating the summary of evidence into recommendations;
  • revising recommendations; and
  • endorsing the final recommendations.
Who is involved
Individuals appraising public health evidence to inform a rapid risk assessment process would benefit from the list of critical appraisal tools provided and the framework for conducting a rapid risk assessment.
Conditions for Use
Not specified
Evaluation and Measurement Characteristics
Evaluation
Information not available
Validity
Not applicable
Reliability
Not applicable
Methodological Rating
Not applicable
Method/Tool Development
Developer(s)
European Centre for Disease Prevention and Control
SE-171 83 Stockholm
Sweden
Email: info@ecdc.europa.eu
Method of Development
Information not available
Release Date
2011
Contact Person/Source
European Centre for Disease Prevention and Control
SE-171 83 Stockholm
Sweden
Email: info@ecdc.europa.eu

Resources

Title of Primary Resource
Evidence-based methodologies for public health - How to assess the best available evidence when time is limited and there is lack of sound evidence.
File Attachment
None
Web-link
Reference
European Centre for Disease Prevention and Control (2011). Evidence-based methodologies for public health—How to assess the best available evidence when time is limited and there is lack of sound evidence. Stockholm, Sweden: European Centre for Disease Prevention. doi: 10.2900/58229. Available from: http://www.ecdc.europa.eu/en/publications/Publications/1109_TER_evidence_based_methods_for_public_health.pdf
Type of Material
Technical report
Format
On-line Access
Cost to Access
None.
Language
English
Conditions for Use
Copyright © 2011 European Centre for Disease Prevention and Control

Title of Supplementary Resource
Operational guidance on rapid risk assessment methodology
File Attachment
None
Web-link
Reference
European Centre for Disease Prevention and Control (2011). Operational guidance on rapid risk assessment methodology. Stockholm, Sweden: European Centre for Disease Prevention and Control. doi: 10.2900/57509. Available from: http://ecdc.europa.eu/en/publications/Publications/1108_TED_Risk_Assessment_Methodology_Guidance.pdf
Type of Material
Technical report
Format
On-line Access
Cost to Access
None.
Language
English
Conditions for Use
Copyright © 2011 European Centre for Disease Prevention and Control.

Have you used this resource? Share your story!