Methodological Quality Rating

This rating sheet is used to rate the methodological quality of systematic reviews that are included in the Registry of Methods and Tools. The AMSTAR tool is used for the rating sheet (to learn more about the AMSTAR, click here to see a summary statement on this tool in the Registry). There are 11 questions in the AMSTAR tool, which are the basis for providing an overall qualitative assessment of a strong, moderate or weak methodological rating for the systematic review. Only systematic reviews are rated using AMSTAR. Surveys and instruments are rated by their validity and reliability measurements. AMSTAR is being used since it the international gold standard for appraising systematic reviews.

1.

Was an ‘a priori’ design provided?

The research question and inclusion criteria should be established before the conduct of the review.
  • Yes
  • No
  • Can't answer
  • Not applicable
2.

Was there duplicate study selection and data extraction?

There should be at least two independent data extractors and a consensus procedure for disagreements should be in place.
  • Yes
  • No
  • Can't answer
  • Not applicable
3.

Was a comprehensive literature search performed?

At least two electronic sources should be searched. The report must include years and databases used (e.g. Central, EMBASE, and MEDLINE). Key words and/or MESH terms must be stated and where feasible the search strategy should be provided. All searches should be supplemented by consulting current contents, reviews, textbooks, specialized registers, or experts in the particular field of study, and by reviewing the references in the studies found.
  • Yes
  • No
  • Can't answer
  • Not applicable
4.

Was the status of publication (i.e. grey literature) used as an inclusion criterion?

Status of publication was not used as an inclusion criterion. The literature search was not limited by language. However, only articles written in English or French were selected for review.

×

The authors should state that they searched for reports regardless of their publication type. The authors should state whether or not they excluded any reports (from the systematic review), based on their publication status, language etc.
  • Yes
  • No
  • Can't answer
  • Not applicable
5.

Was a list of studies (included and excluded) provided?

A list of excluded studies was not provided.

×

A list of included and excluded studies should be provided.
  • Yes
  • No
  • Can't answer
  • Not applicable
6.

Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes. The ranges of characteristics in all the studies analyzed e.g. age, race, sex, relevant socioeconomic data, disease status, duration, severity, or other diseases should be reported.
  • Yes
  • No
  • Can't answer
  • Not applicable
7.

Was the scientific quality of the included studies assessed and documented?

‘A priori’ methods of assessment should be provided (e.g., for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo controlled studies, or allocation concealment as inclusion criteria); for other types of studies alternative items will be relevant.
  • Yes
  • No
  • Can't answer
  • Not applicable
8.

Was the scientific quality of the included studies used appropriately in formulating conclusions?

The purpose of this review was to determine whether there are existing reliable, valid and/or useful instruments to apply in the assessment of KTE implementation and its impact.

×

The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the review, and explicitly stated in formulating recommendations.
  • Yes
  • No
  • Can't answer
  • Not applicable
9.

Were the methods used to combine the findings of studies appropriate?

The review was not structured in a way to permit the pooling of results (e.g. meta-analysis). The purpose of this review was to determine whether there are existing reliable, valid and/or useful instruments to apply in the assessment of KTE implementation and its impact.

×

For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity (i.e. Chi-squared test for homogeneity, I²). If heterogeneity exists a random effects model should be used and/or the clinical appropriateness of combining should be taken into consideration (i.e. is it sensible to combine?).
  • Yes
  • No
  • Can't answer
  • Not applicable
10.

Was the likelihood of publication bias assessed?

Publication status (i.e. grey literature) was not used an inclusion criterion but an assessment of publication bias (using graphical aids or statistical tests) was not presented. This is not considered a fatal flaw as a result of the nature and purpose of this study.

×

An assessment of publication bias should include a combination of graphical aids (e.g., funnel plot, other available tests) and/or statistical tests (e.g., Egger regression test).
  • Yes
  • No
  • Can't answer
  • Not applicable
11.

Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both the systematic review and the included studies.
  • Yes
  • No
  • Can't answer
  • Not applicable