Evaluating knowledge translation interventions: A systematic review

A Summary of

Van Eerd, D., Cole, D., Keown, K., Irvin, E., Kramer, D., Brenneman Gibson, J., Kohn, M.K., Mahood, Q., Slack, T., Amick III, B.C., Phipps, D., Garcia, J., & Morassaei, S. (2011). A systematic review of the quality and types of instruments used to assess KTE implementation and impact. Toronto, ON: Institute for Work & Health.

Description

The Institute for Work and Health (IWH) conducted a systematic review to identify instruments to evaluate knowledge translation (KT) initiatives and their impact. With an increasing emphasis on KT in health and public health, attention is shifting to the effectiveness of KT interventions. However, the effectiveness of KT practices has not been consistently assessed; this may be due to a lack of valid and reliable tools.

The IWH team conducted a comprehensive and systematic review of peer-reviewed literature to find instruments to evaluate KT practices. Instruments included questionnaires, surveys and interviews that collected information from the end-users of research evidence (such as practitioners).

Knowledge translation, or KTE (knowledge transfer and exchange), was defined as "an iterative and dynamic process by which relevant research information is created, synthesized, disseminated and exchanged through interactive engagement with decision-makers/knowledge-users to improve outcomes, provide more effective services and products, and strengthen the use of evidence in decision-making, practice, planning and policy-making."

The Institute for Work and Health's resources on tools for evaluating KT interventions can be accessed from their website (http://www.iwh.on.ca/sys-reviews/kte-evaluation-tools). The resources include:

  • a summary highlighting key evidence points;
  • a list of promising KT evaluation tools; and
  • the full report.

Format of the Method


Language(s)

Not specified

Format(s)

Not specified

Cost

Not Specified

Using the Method


Time for Participation/Completion

Information not available

Additional Resources and/or Skills Needed for Implementation

Not Specified

Steps for Using Method/Tool

The main findings of the Institute for Work & Health's systematic review are:

  • There are few well-developed instruments to evaluate the implementation and impact of KTE practices.
  • A total of 16 articles describe promising tools to evaluate KTE intitiatives. These instruments show some degree of validity and reliability, and are able to measure changes in knowledge, attitudes and behaviours among decision-makers and stakeholders.

The implications of this systematic review for practitioners and researchers include the following:

  • Consider evaluation instruments found in this review when planning an evaluation of a KTE initiative. When choosing an instrument, look for a clear presentation of the measurement properties.
  • Select or construct well-developed instruments for evaluation.
  • Consider developing instruments that can be used in various contexts. It is important to separate implementation, which is dependent on context, from instrument development, which can be based on theory and be independent of context.
  • Systematically develop and use evaluation instruments with known measurement properties to advance KTE evaluation.

Who is involved

Anyone interested in assessing the effectiveness of a knowledge translation activity could use the tools listed in this systematic review.

Conditions for Use

Not specified

Evaluation of the Method


Evaluation

Information not available

Validity

Not applicable

Reliability

Not applicable

Methodological Rating

Strong

Development of the Method


Developer(s)

Dwayne Van Eerd
Donald Cole
Kiera Keown
Emma Irvin
Desre Kramer
Jane Brenneman Gibson
Melanie Kazman Kohn
Quenby Mahood
Tesha Slack
Benjamin Amick III
David Phipps
John Garcia
Sara Morassaei

Method of Development

The review was conducted by a multidisciplinary team who searched for articles on knowledge translation in nine databases from different disciplines (including medicine, psychology, education, agriculture and others), in addition to hand searching specific journals and consulting with content experts. The initial search yielded 9,998 articles after duplicates were removed.

The articles were reviewed in terms of their relevance to the research question: "Does the article describe a KTE outcome or a tool to measure a KTE outcome as a result of a KTE application?"

The review team identified 346 potential articles as relevant, and examined qualitative and quantitative articles. Qualitative articles described specific approaches to evaluating KTE, while quantitative articles included instruments used to evaluate KTE (such as questionnaires, surveys and interviews).

A total of 12 qualitative and 54 quantitative articles met the team's inclusion criteria. Information from all 66 articles was collated and details about the measurement properties of instruments from the quantitative articles were collected. The review team collected information on the validity, reliability and responsiveness of instruments. Of these 54 instruments, 16 were identified as promising based on their measurement properties.

Release Date

2011

Contact Person/Source

Dwayne Van Eerd
Institute for Work & Health
481 University Avenue, Suite 800
Toronto, ON M5G 2E9
Email: dvaneerd@iwh.on.ca

These summaries are written by the NCCMT to condense and to provide an overview of the resources listed in the Registry of Methods and Tools and to give suggestions for their use in a public health context. For more information on individual methods and tools included in the review, please consult the authors/developers of the original resources.

Have you used this resource? Share your story!