Buffett, C., Ciliska, D., & Thomas, H. (2007). Can I use this evidence in my program decision? Assessing applicability and transferability of evidence. Hamilton, ON: National Collaborating Centre for Methods and Tools (NCCMT).
Relevance for Public Health
This tool was created to help public health managers and planners make decisions about local programming priorities. Public health decision-makers who want to incorporate high quality evidence in their programming can use this tool to help determine whether a policy or program is relevant or feasible. Created for public health, this tool includes questions relevant to public health context such as collaboration with stakeholders, needs for local implementation, assessment of the political and organizational climates, and evaluation of the costs related to outcome. This tool can be used by decision-makers in any public health program area.
This tool is part of an overall process that explores whether and how to apply evidence into public health decision-making and policy-making. Public health decision-makers base clinical and policy decisions on high quality evidence when it is available. Decision-makers must also consider whether or not an intervention that worked in a research study could apply to local programs and policy.
Applicability assesses the feasibility of providing an intervention in a local setting. Applicability considers cost-effectiveness, organizational culture and capacity (Buffet, Ciliska, & Thomas, 2007). The tool includes questions that assess the following applicability factors:
- political acceptability or leverage;
- social acceptability;
- available resources; and
- organizational expertise and capacity
Transferability assesses the likelihood that the intervention developed and delivered in one setting can achieve the same outcomes when applied in a different local setting. Considerations for transferability include generalizability and community effectiveness (Buffet et al., 2007). The tool includes questions that assess the following transferability factors:
- magnitude of the health issue in a local setting;
- magnitude of the ‘reach’ of the intervention; and
- target population characteristics
Users of the tool can assign their own scoring system for the tool. A methodological rating of ‘unknown’ was assigned for this tool since the evaluation of its measurement characteristics is in progress.
Accessing the Method/Tool
- Paper copy
- On-line Access
Implementing the Method/Tool
Time for Participation/Completion
Information not available
Additional Resources and/or Skills Needed for Implementation
Steps for Using Method/Tool
Preliminary steps prior to use of the tool:
- generate a question to drive literature search and review process;
- search and retrieve relevant literature;
- critically appraise identified literature.
When valid, higher quality evidence is found, this tool is used to assess the feasibility and transferability of research findings into public health practice.
Step 1: Establish a facilitator for the overall process who can act as group leader and maintain timelines.
Step 2: Select key stakeholders to form a group who can commit to completing the tool and working to achieve consensus.
Step 3: Select as a group applicability and transferability assessment questions that are most important for the intervention of interest and local context.
Step 4: Since not all criteria are relevant for the particular intervention considered, the group can rate some criteria as more important than others.
Step 5: Identify a scoring system for the assessment questions (e.g. a 1-5 point scale where 1 is low level impact and 5 is high level impact). Priority goes to the highest scoring intervention or program.
Step 6: Document scoring process used by the group.
Who is involved
Authors of this tool recommend the tool be completed by a group comprising interested and concerned parties from inter-sectoral, multidisciplinary and consumer groups. A facilitator is required to orient the group and establish timelines.
Conditions for Use
Evaluation and Measurement Characteristics
Information not available
Validity properties meet accepted standards.
Content validity completed. Content was changed after it was reviewed for content validity. No further validity testing has been completed to date.
Reliability testing in progress
Method of Development
This tool was developed from synthesizing literature related to knowledge synthesis, translation and exchange and relevant to applicability and transferability. A skilled librarian conducted an initial search (1996-2006) with an update in February 2007. Over 250 potentially relevant articles were appraised for applicability and transferability. Overall findings revealed no consensus on the criteria to evaluate the applicability and transferability of evidence for public health decision-making. Relevant literature informed the assessment questions for applicability (e.g. can it work?) and transferability (e.g. will it work?)
National Collaborating Centre for Methods and Tools (NCCMT)
Phone: (905) 525-9140
These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.