II Seven-step process for making evidence-informed public health decisions
-- Submitted by Lina Sherazy, Dr. Susan Snelling, Kristin Read and Dr. Maureen Dobbins, the National Collaborating Centre for Methods and Tools
The National Collaborating Centre for Methods and Tools (NCCMT) supports and engages with those working in public health across Canada to put what works into practice and policy. Incorporating research evidence in public health decision-making allows for efficient and effective use of resources. Evidence-Informed Public Health (EIPH) is the process of distilling and disseminating the best available evidence from a variety of sources as shown in the figure below (including research findings, local context, political climate, and financial and human resources) and integrating this evidence to inform and improve public health practice and policy. The NCCMT has identified a seven-step process for making evidence-informed public health decisions. This article provides an example of working through these seven steps using healthy weight promotion as an example. Recommended resources from the NCCMT’s Registry of Methods and Tools (http://www.nccmt.ca/resources/registry) are included to support each step. The registry is a searchable, online collection of knowledge translation methods and tools for public health.
II Seven-step process for making evidence-informed public health decisions
Consider this scenario: You are a health promoter and you have been asked by your organization to design a program that supports children’s healthy weights in your community. You have become aware of the community’s concern about weight stigma. Your organization has recently partnered with the early childhood education sector on other issues and there is potential for partnership related to healthy weight promotion in early childhood education (ECE) settings.
Step 1 - Define: Clearly define the issue or problem
The first step of the EIPH process is to clearly define the issue in relation to who it affects, possible solutions, and the desired impact. PICO is a useful acronym that can be used to outline the Population, Intervention, Comparison, and Outcomes of the issue. Population identifies the target group and outlines what characteristics they have in common. The Intervention proposes an approach to addressing the issue. Comparison is the alternative to the intervention and Outcomes are factors related to the desired impact. Here is an example PICO based on the scenario above.
P: Children aged 0-5 in an ECE setting
I: Promotion of healthy habits (nutrition, physical activity)
C: Usual programs
O: Change in eating habits or physical activity or BMI
- Defining your question: PICO and PS (http://www.nccmt.ca/resources/search/138)
Step 2 - Search: Efficiently search for research evidence
The terms identified using PICO can help inform the search strategy; based on the PICO, use the 6S pyramid (http://www.nccmt.ca/professional-development/eiph/search) to structure where to look for evidence. Starting the search at the top of the pyramid is the most efficient way to access the most highly synthesized (and ideally, quality appraised) research evidence because each level of the pyramid draws on evidence from the layers below. Single research studies are the lowest level of the 6S pyramid, and could comprise searches of thousands of studies. Searching for single studies creates a risk of bias because people often pick studies with findings they favour.
Starting at the top of the pyramid, searching for summaries, one could look at the National Guidelines Clearinghouse (https://www.guideline.gov/) The National Guidelines Clearinghouse database contains evidence-based clinical practice guidelines with structured, standardized summaries. When doing our search in the Clearinghouse, we find several guidelines, of which one contains recommendations for early childhood (https://guideline.gov/summaries/summary/49132/primary-prevention-of-chil...) settings pertaining to childhood healthy weights. When making decisions about public health practice using the seven-step model, once you find the most synthesized evidence that is relevant to the PICO question, is of high quality, and published recently, you can stop searching. However, for the purpose of this article, we will move down the pyramid to also search for synopses of syntheses and syntheses. Health Evidence™ (https://www.healthevidence.org/) provides access to thousands of quality-rated systematic reviews that evaluate the effectiveness of public health interventions. A search using Health Evidence™ on nutrition and physical activity in children 1-4 years old in day care centres, published from 2014-2017 and quality rated as ‘strong’ yielded seven results. Of these seven systematic reviews, all appear relevant to our topic.
- The 6S Pyramid
- The National Guidelines Clearinghouse
- Health Evidence™
Step 3 - Appraise: Critically and efficiently appraise the information sources
Now that you have found relevant research evidence, you need to consider the quality of the evidence. Critical appraisal allows you to assess the quality of the study’s methods to determine whether the findings are trustworthy and to look for sources of bias. Public health research has numerous types of study designs, and different critical appraisal tools exist for these various designs. The guideline summary our search yielded has not been critically appraised, so we would use a tool intended to appraise the quality of guidelines such as the AGREE II tool (http://www.nccmt.ca/resources/search/100). In our search of Health Evidence™, we specifically selected articles already assessed as being of high methodological quality, so that step is done for us when using Health Evidence™.
- Critically Appraising Practice Guidelines: The AGREE II Instrument(http://www.nccmt.ca/resources/search/100)
- Tools to Assess the Methodological Quality of Systematic Reviews: AMSTAR(http://www.nccmt.ca/resources/search/97)
- Critical Appraisal Skills Programme: Checklists (http://www.nccmt.ca/resources/search/87)
Step 4 - Synthesize: Interpret information and form recommendations for practice
The fourth step of EIPH is to summarize the research evidence, identifying key action messages that can inform practice. If your search identifies more than one piece of evidence, you will need to synthesize the evidence across studies or reviews. What if the evidence has conflicting conclusions? In that case, you need to consider which source has stronger methodological quality, is the most recent, and has a population and/or setting most similar to your own. At this stage, it is recommended that you prepare a briefing note to document the findings from the research evidence and to share the synthesized findings with decision makers. The briefing note should include:
- a description of the issue
- the types and nature of the evidence (research and practice-based)
- the options considered (and the rationale for these options)
- the recommendation(s) for action.
- Briefing Note: Decisions, Rationale and Key Findings Summary (http://www.nccmt.ca/resources/search/137)
Step 5 - Adapt: Adapt the information to the local context
Not all research findings will work or fit your context; each situation is unique. The fifth step involves tailoring the key messages from synthesized research evidence to the local context. There are several factors to consider in adaptation including: the importance of the public health issue, the proposed intervention, the local community, and your organization and partners’ resources. The local context can include the culture of the population or location, which can affect nutrition habits and food availability. Resources should include both budget and human resources. Because the scenario describes a sensitivity to concerns about the effect of healthy weight interventions on weight bias and weight stigma, these issues should be taken into account as you determine the suitability of an intervention for your context.
- The Applicability and Transferability Tool (http://www.nccmt.ca/resources/search/24) developed by the NCCMT includes a list of questions about collaboration, needs, political climate and costs that can be used to guide discussion about the fit of the key messages to your context.
- The “Is this Evidence Useful to Me” tool (http://www.nccmt.ca/resources/search/227) from the Centre for Research on Inner City Health (CRICH) is an adapted version of the Applicability and Transferability Tool and incorporates a health equity perspective.
Step 6 - Implement: Decide whether (and plan how) to implement the adapted evidence into practice or policy
Now it’s time to put research evidence into action! In this step you plan how to implement the identified, appraised, synthesized and possibly adapted evidence into practice or policy. While planning how to implement the evidence into practice, it is also important to consider how you will evaluate the impact that the implementation had on practice as well as relevant client or population outcomes.
Here are three steps to consider while developing an implementation plan:
1. Conducting a situational assessment
This involves assessing the facilitators and barriers for the change you intend to implement. Think about the people who will be involved from the organization and from the community, and consider the type of change in practice that will be required. Which strategies will maximize facilitators and minimize barriers? For example, one barrier to implementing a nutrition or physical activity program could be limited resources. Perhaps you would have to plan to build capacity or partner with other organizations to help mitigate this potential barrier.
2. Planning a program
At this point you want to think about the possible implementation strategies. If you plan to partner with early childhood education, what will their degree of involvement be? A comprehensive plan should consider all possible stakeholder groups.
3. Disseminating the intervention
You must identify who needs to be informed about the change (educators, parents, community members, others?) and determine how best to reach them.
- Planning and assessment tool for chronic disease prevention and management (http://www.nccmt.ca/resources/search/81)
Step 7 - Evaluate: Assess the effectiveness of implementation efforts
The last step is evaluating the effectiveness of the implementation strategies. There are two kinds of evaluation that may be relevant: process and outcome. Process evaluation assesses whether the program or implementation plan was delivered as intended. Outcome evaluation determines if the program or implementation achieved the anticipated results or if there were any unanticipated outcomes. Many tools for evaluation agree on these basic components:
- identify variables of interest
- select indicators to measure those variables
- collect outcome data before and after implementation
- interpret these results to inform further planning.
You may also want to evaluate your efforts in finding and using evidence. The “Improving future decisions” tool can be helpful as a guide to reflective practice of EIPH.
- Improving future decisions: Optimizing the decision process from lessons learned (http://www.nccmt.ca/resources/search/194)
EIPH is essential for closing the gap between research evidence and practice. It has the potential to lead to implementing the most ethical and effective interventions, as well as increasing public health credibility and accountability to funders and the public. The seven-step EIPH process outlines a systematic way to incorporate the best available research evidence across all public health domains.
The EIPH webinar focused on resources to support public health practitioners involved in healthy weights promotion is available on our YouTube channel. https://www.youtube.com/watch?v=WsuGNE4GWGQ&list=PL50nkPezdX7mBtoN87Kovg...
Only a few of the tools included in the NCCMT’s Registry of Methods and Tools are highlighted in this paper. The registry includes resources related to all seven steps in the EIPH process and can be filtered by each of the EIPH steps, or by other relevant knowledge translation activities (e.g., program planning, policy development). The NCCMT assesses tools and methods for their applicability in public health and summarizes these products for easy review. The NCCMT’s Learning Centre houses a series of free, self-paced, interactive online learning modules related to the steps of EIPH process. Also helpful are the NCCMT’s Understanding Research Evidence videos; these short videos use plain language and cartoon illustrations to explain complex research terms and concepts you may come across when looking at research evidence.
The NCCMT strives to provide support, knowledge, skills, and capacity to use the best available evidence in public health decision making and meet identified needs of those working in public health. To learn more about the NCCMT visit www.nccmt.ca or sign up to receive the Round-up newsletter. You can also connect with us through email ([email protected]) or social media (@nccmt) with any questions and/or feedback.