user story webinar summary

Applicability and Transferability of Evidence Tool (A&T Tool)

A summary of

Buffett, C., Ciliska, D., & Thomas, H. (2007). Can I use this evidence in my program decision? Assessing applicability and transferability of evidence. Hamilton, ON: National Collaborating Centre for Methods and Tools (NCCMT).

How to cite this NCCMT summary:

National Collaborating Centre for Methods and Tools (2009). Applicability and Transferability of Evidence Tool (A&T Tool). Hamilton, ON: McMaster University. (Updated 30 August, 2017) Retrieved from http://www.nccmt.ca/knowledge-repositories/search/24.

Categories: Tool, Adapt, Organizational capacity and management

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Relevance for Public Health

This tool was created to help public health managers and planners make decisions about local programming priorities. Public health decision-makers who want to incorporate high quality evidence in their programming can use this tool to help determine whether a policy or program is relevant or feasible. Created for public health, this tool includes questions relevant to public health context such as collaboration with stakeholders, needs for local implementation, assessment of the political and organizational climates, and evaluation of the costs related to outcome. This tool can be used by decision-makers in any public health program area.

Description

This tool is part of an overall process that explores whether and how to apply evidence into public health decision-making and policy-making. Public health decision-makers base clinical and policy decisions on high quality evidence when it is available. Decision-makers must also consider whether or not an intervention that worked in a research study could apply to local programs and policy.

Applicability assesses the feasibility of providing an intervention in a local setting. Applicability considers cost-effectiveness, organizational culture and capacity (Buffet, Ciliska, & Thomas, 2007). The tool includes questions that assess the following applicability factors:

  • political acceptability or leverage;
  • social acceptability;
  • available resources; and
  • organizational expertise and capacity

Transferability assesses the likelihood that the intervention developed and delivered in one setting can achieve the same outcomes when applied in a different local setting. Considerations for transferability include generalizability and community effectiveness (Buffet et al., 2007). The tool includes questions that assess the following transferability factors:

  • magnitude of the health issue in a local setting;
  • magnitude of the ‘reach’ of the intervention; and
  • target population characteristics

Users of the tool can assign their own scoring system for the tool. A methodological rating of ‘unknown’ was assigned for this tool since the evaluation of its measurement characteristics is in progress.

Implementing the Method/Tool
Steps for Using Method/Tool

Preliminary steps prior to use of the tool:

  • generate a question to drive literature search and review process;
  • search and retrieve relevant literature;
  • critically appraise identified literature.

When valid, higher quality evidence is found, this tool is used to assess the feasibility and transferability of research findings into public health practice.

Step 1: Establish a facilitator for the overall process who can act as group leader and maintain timelines.

Step 2: Select key stakeholders to form a group who can commit to completing the tool and working to achieve consensus.

Step 3: Select as a group applicability and transferability assessment questions that are most important for the intervention of interest and local context.

Step 4: Since not all criteria are relevant for the particular intervention considered, the group can rate some criteria as more important than others.

Step 5: Identify a scoring system for the assessment questions (e.g. a 1-5 point scale where 1 is low level impact and 5 is high level impact). Priority goes to the highest scoring intervention or program.

Step 6: Document scoring process used by the group.

Who is involved

Authors of this tool recommend the tool be completed by a group comprising interested and concerned parties from inter-sectoral, multidisciplinary and consumer groups. A facilitator is required to orient the group and establish timelines.

Conditions for Use
Not specified
Evaluation and Measurement Characteristics
Evaluation
Information not available
Validity
Validity properties meet accepted standards.

Content validity completed. Content was changed after it was reviewed for content validity. No further validity testing has been completed to date.

Reliability
Reliability testing in progress
Methodological Rating
Unknown/No evidence
Method/Tool Development
Developer(s)

C. Buffet
D. Ciliska
H. Thomas

Method of Development

This tool was developed from synthesizing literature related to knowledge synthesis, translation and exchange and relevant to applicability and transferability. A skilled librarian conducted an initial search (1996-2006) with an update in February 2007. Over 250 potentially relevant articles were appraised for applicability and transferability. Overall findings revealed no consensus on the criteria to evaluate the applicability and transferability of evidence for public health decision-making. Relevant literature informed the assessment questions for applicability (e.g. can it work?) and transferability (e.g. will it work?)

Release Date
2007
Contact Person/Source

Donna Ciliska
McMaster University
National Collaborating Centre for Methods and Tools (NCCMT)
Hamilton, ON
Phone: (905) 525-9140
Email: ciliska@mcmaster.ca

Resources

Title of Primary Resource
Tool for assessing applicability and transferability of evidence
File Attachment
None
Web-link
Reference

Buffett, C., Ciliska, D., & Thomas, H. (2007). Can I use this evidence in my program decision? Assessing applicability and transferability of evidence. Hamilton, ON: National Collaborating Centre for Methods and Tools (NCCMT).

Type of Material
Tool
Format
Paper copy
Cost to Access
Language
English, French
Conditions for Use
Not specified

Have you used this resource? Share your story!