How to assess the best available evidence when time is limited

A Summary of

European Centre for Disease Prevention and Control (2011). Evidence-based methodologies for public health—How to assess the best available evidence when time is limited and there is lack of sound evidence. Stockholm, Sweden: European Centre for Disease Prevention. doi: 10.2900/58229. Available from: http://www.ecdc.europa.eu/en/publications/Publications/1109_TER_evidence_based_methods_for_public_health.pdf

Description

Developed by the European Centre for Disease Prevention and Control (ECDC), this technical report, Evidence-based methodologies for public health, provides a framework for rapid risk assessment and evaluates critical appraisal tools for public health evidence. This method looks at how evidence-based methodologies can be applied in public health. Specifically, this report examines how to provide evidence-informed advice on public health issues when there is little evidence and when advice is sought within a short time frame. The report also discusses the need to adapt methods from evidence-based medicine for a public health setting, and the usefulness of critical appraisal tools to grade evidence.

Rapid risk assessment is a core public health function used to evaluate the risk to human health when an incident is confirmed as being of potential public health concern (ECDC, 2011). Rapid risk assessments should be based on the best available evidence to determine:

  • if a response is needed;
  • the urgency and magnitude of the response;
  • the design and selection of intervention; and
  • broader management of the incident.
Access the Method

Format of the Method


Language(s)

Not specified

Format(s)

Not specified

Cost

Not Specified

Using the Method


Time for Participation/Completion

Information not available

Additional Resources and/or Skills Needed for Implementation

Individuals using the critical appraisal tools listed would require skills in critically appraising research evidence.

Steps for Using Method/Tool

This method has five sections.

I: Background

Evidence-based public health includes two main areas:

  1. evidence of the effectiveness of interventions and how evidence can be applied to programs and policies
  2. defining health risks and risk assessment, where uncertainties in the decision-making process have an impact on risk assessment

There are three basic requirements for addressing uncertainties in risk assessments:

  1. Systematically identify and evaluate the sources of uncertainties.
  2. Evaluate their combined effect on the outcome of the assessment.
  3. Communicate this to decision-makers.

II: How to give evidence-based guidance when evidence is scarce and time is limited?

The risk assessment process consists of:

  • Stage 0: Preparatory
  • Stage 1: Threat detection/verification
  • Stage 2: Assessment of risk
  • Stage 3: Developing advice
  • Stage 4: Implementation
  • Stage 5: Monitoring and evaluation

To see tools for each stage, see Annex 2.

III: The usefulness of EBM grading tools for grading evidence and recommendations for the field of public health

This method assesses tools for grading the quality of evidence and the development of public health guidelines. The report provides brief descriptions of each organization's approach to developing guidelines and making recommendation:

  • World Health Organization – Subcommittee on the Use of Research Evidence (SURE)
  • GRADE, The Grading of Recommendations Assessment, Development and Evaluation Working Group, provides a framework to move from assessing evidence to making a recommendation. The GRADE approach is being used in different ways by other organizations to appraise evidence for guideline development. To see challenges on applying GRADE in public health settings, see Annex 5 in the appendix.
  • The National Institute for Health and Clinical Excellence (NICE)
  • The Scottish Intercollegiate Guidelines Network (SIGN) (see Annex 6)
  • Centers for Disease Control and Prevention (CDC)
  • National Health and Medical Research Council (NHMRC)
  • Centre for Evidence Based Medicine (CEBM)

The following tools for appraising evidence are discussed:

  • AGREE (Appraisal of Guidelines Research & Evaluation) tool to appraise guidelines (to see a summary statement on AGREE II tool in the Registry, click here)
  • CONSORT (Consolidated Standards of Reporting Trial) statement for reporting randomized controlled trials
  • TREND (Transparent Reporting of Evaluations with Non-randomized Designs) statement – a checklist to guide reporting of non-randomized controlled trials of public health intervention.

(For more information on appraising evidence on public health interventions, see a module developed by NCCMT - click here)

IV: Assessing and assuring quality to guideline development for health protection and control of communicable diseases

The report provides a description of the AGREE II instrument and its limitations, and discusses its application for communicable disease guidelines. Also, the Guideline Evaluation Tool (GET5), a shorter and adapted version of the AGREE II, is discussed (see Annex 7 and 8 for the GET5 tool).

V: EBM methods for public health—the use of consensus methods

Consensus methods can be used in different ways for public health guidance. For example, when:

  • defining the key questions and selecting relevant data sources and type of evidence needed;
  • assessing which outcomes are essential when grading evidence and informing the strength of the recommendation;
  • translating the summary of evidence into recommendations;
  • revising recommendations; and
  • endorsing the final recommendations.

Who is involved

Individuals appraising public health evidence to inform a rapid risk assessment process would benefit from the list of critical appraisal tools provided and the framework for conducting a rapid risk assessment.

Conditions for Use

Not specified

Evaluation of the Method


Evaluation

Information not available

Validity

Not applicable

Reliability

Not applicable

Methodological Rating

Not applicable

Development of the Method


Developer(s)

European Centre for Disease Prevention and Control
Website: https://ecdc.europa.eu/en

Method of Development

Information not available

Release Date

2011

Contact Person/Source

European Centre for Disease Prevention and Control
SE-171 83 Stockholm, Sweden
Email: info@ecdc.europa.eu

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Have you used this resource? Share your story!