National Collaborating Centre for Methods and Tools

Earlier this year the Medical Office of Health at my health department asked for 5 (maximum) bullet points regarding effective strategies to reduce the spread of viruses be developed. The request came at a time when I was slightly overwhelmed with the pending H1N1 pandemic. Given all the talk about the importance of using evidence, I wondered if this was a topic that we could use to walk through the steps of Evidence-Informed Public Health.
Here’s how I would start the process…As the manager of the CD program, short on time, I would refer this request to my team. I am aware there have been conflicting messages about the use of hand sanitizers versus soap and water, so I would ask my team to look for evidence related to interventions to reduce the spread of viruses, rather than getting bogged down in the detail of which hygiene strategy may be more effective. Is that a good idea?
I also told them that there must be evidence to back up the recommendations. I would probably also suggest they look for a systematic review as the first step, appreciating that if they can find one, a lot of time and work will be saved. I don’t know a lot about systematic reviews except they come highly recommended.

It would be great if we could walk through this one as an example.

This is a great example to work through – thanks for suggesting. The way you framed the question related to interventions to reduce the spread of viruses is great; as was your suggestion to start with a systematic review. Why have your staff spend days looking through potentially hundreds of articles when someone else has already done that? Someone else has searched for any studies related to the question, rated each study for relevance to the question, then quality of the methods and done the data extraction, compilation and conclusion. In some cases, the outcomes can be statistically combined in a meta-analysis.

For more information on systematic reviews, see Ciliska, D., Thomas, H., & Buffett, K. (2008). Introduction to Evidence-Informed Public Health and a Compendium of Critical Appraisal Tools for Public Health Practice.
English Compendium
French Compendium

Let's pick this up in the new year.

Happy Holidays!!

Donna

Happy New Year!

Now that we are back from holidays - could we start the discussion again?

Where do we start?

You start by clearly defining the question. The acronym that is often used to work through this process is PICO.

P=population: who is the population of interest?
I=intervention: what is the strategy that you are considering?
C-comparison: no intervention or standard care
O=outcome: the effect you are hoping to achieve by using the intervention.

Why don’t you try and work out the PICO for this particular illustration and then we can review it.

Donna

Hi -sorry for the delay in responding. Here are my thoughts for the PICO

P (population) – community at large, all ages
I (intervention) – physical interventions e.g., personal hygiene such as hand washing,
C (comparison) – no physical interventions
O (outcome) - spread of respiratory viruses for example, H1N1

Great – this is very clear.
Now we will go on to the search for articles. From the question, you can see in this case the comparison and the population terms will not help you in the search.
As we said above, the most useful type of evidence would be a good systematic review. For public health in Canada, the best source of relevant systematic reviews is Health-Evidence.ca. (www.health-evidence.ca)

If you use Health-evidence.ca, you are likely to get a manageable number of reviews. If there are several, you will see that health-evidence has done a quality rating of the reviews. Go with the highest score for the highest quality review. If there are several with the same scores, retrieve the reviews with the most recent date of publication. If a strong review is 10 years old, it is too old to consider.

Donna

Hi Donna,

Before going to Health-Evidence, I wanted to see what I would get using Google Scholar…I typed in “prevention of respiratory viruses” and got 101,000 results…a few too many results to review. That’s in fact a lot less than what I got with Google, which gave me 4,420,000 results!!!

I took your advice and went to Health-Evidence. When I used the same search terms “prevention of respiratory viruses”, I got four results. Three were considered strong and one was considered moderate. Of the three that were strong, one had a summary statement. I’m assuming that I should probably work with the article that is rated as strong and has a summary statement. Is that correct?

If this is the article of choice, what’s the next step?

Sounds like you have found a great article!

The next step is to appraise the quality of the review. To do this you can refer to a compendium that NCCMT has on its website.

However, if you find the systematic review on the health-evidence site, which is what happened in this example, you can actually see how the rater scored the review. When you click on the title of the systematic review, you will go to a new page, at the top of this new page you will see a link to the actual assessment that was completed by the rater. Click on the link ‘View quality assessment” and you will get the details.

If having used the compendium or having reviewed the quality assessment on the Health-Evidence site you are confident that you have a strong systematic review, the next step is to interpret or synthesize or make recommendations from the findings in the systematic review. We’ll discuss that next.

Before moving on, appraising evidence is a challenging step…do you have any questions? Does anyone else reading this forum have any questions?

Donna

I have a question....Can you tell me what the quality rating provided by Health Evidence means? How is it determined? What makes one systematic review strong and another moderate and still another weak?

Health-Evidence uses a standardized rating tool (“quality assessment tool”) to evaluate each systematic review. Two people do the rating independently. There are ten criterion included in the tool and the rating depends on how many criterion an individual systematic review meets. Reviews that meet seven or more of the criterion are considered strong. Reviews that meet between five and six criterion are considered moderate and those with four or less are considered weak. A review with a weak score shouldn’t immediately be tossed out, particularly if it’s the only evidence you can find, however, if you decide to move forward with this evidence, you should do so with caution. Another caution is that the rating is not related to the strength of the literature reviewed – only the process of the review. So a systematic review that rated at 9 may contain studies that are all considered weak….and this may be the state of the research around this question. You will find the answer to this concern in the review findings (and usually conclusions); as well you can look at the quality criteria #6 in the Health-Evidence.ca form.

The results of the rating are posted on the Health-Evidence website so you can see how different systematic reviews have been rated – where were the strengths and weaknesses of each.

Donna