Back to top

Evaluate for Action: Developing a Communication and Reporting Plan for Evaluation Results

Contents

I  Introduction
II Identifying and understanding your stakeholders
III Defining communication objectives
IV Considerations and barriers to dissemination
V Channels and formats
VI Conclusion
VII References
VIII Resources

--Submitted by Laura Bellissimo, Health Promotion Coordinator and Allison Meserve, Health Promotion Consultant; Health Promotion Capacity Building at Public Health Ontario

I Introduction

A key purpose of evaluation is to use the results to make more informed decisions [1]. The Health Promotion Capacity Building team at Public Health Ontario provides ten defined steps for conducting an evaluation to better inform decisions [1]. This article will specifically focus on step nine of this model: interpreting and disseminating the results of an evaluation. Although this step comes near the end of the evaluation process, planning for how you are going to communicate evaluation results should occur during early stages of evaluation planning. The communication and reporting plan is developed by identifying and understanding  evaluation stakeholders, defining  communication objectives and choosing the best channels or the way in which your message is distributed (e.g., social media, conference presentation) and the best formats (e.g., the means used to disseminate and convey the message such as infographics, executive summaries) [2].

The content for this article was presented at the HC Link 2015 conference, Linking for Healthy Communities: Action for Change. The presentation slides can be accessed through the link below, along with a step-by-step worksheet to assist with developing a communication and reporting plan: http://hclinkontario.ca/images/2015conference/slides/BlockA_What_do_I_do...  

II Identifying and understanding your stakeholders

A key factor when determining what and how you’re communicating are your stakeholders. Evaluations typically include a number of stakeholders with diverse information needs including those who will make decisions based on results, those involved in planning or creating the program being evaluated, or those who are directly affected by the results [3].

Often there is more than one type of stakeholder who will use evaluation results to inform decisions. Program staff may use results to improve the program and to make resource decisions, while management may use evaluation results to make funding and policy decisions. External stakeholders may use findings to create or improve their own programs. Consider the following audience characteristics when deciding what to communicate:

  • Familiarity: How familiar an audience is with the program can influence the type and quality of background information needed to provide sufficient context in interpreting the results [3].
  • Attitude: Your audience’s attitude and interest in the program and in evaluation in general, will influence the content, format and delivery method for communicating results [3].
  • Role in decision making: The more influence an audience has in decision making, the more detailed information is needed to inform decisions [3].
  • Literacy: It is important to assess literacy level and familiarity with research and evaluation methods of your audience to best communicate results [3].

III Defining communication objectives

There are three general purposes to communicating evaluation findings. First, we want to build awareness and support for the program. Communications that describe a program, how it works, and to what effect, can be used to let audiences know about the existence of a program, its uses and benefits [3].

Another communication objective is to facilitate growth and make improvements. Program staff and those responsible for organizational functions can learn from their experiences participating in the evaluation and from the findings [3]. Further, implementation or formative evaluations can help inform decision making about possible program improvements.

Lastly, through communicating results, we want to demonstrate accountability. Outcome or summative evaluations are frequently designed to show results and impact and to be accountable to funders, board members, and senior management. Evaluation communications demonstrating impact and accountability will help stakeholders and audiences in decision making about continued funding, new funding proposals, or implementing a similar program elsewhere.

IV Considerations and barriers to dissemination

At this time, you may have identified that it may be more complicated to reach your desired stakeholders than first anticipated, so at this stage of creating a communications and reporting plan it is important to consider some implications of the audience characteristics you identified, including possible barriers.

  • Content and complexity: Using the appropriate channel and format will determine whether your message is received by your audience. It is important to understand the media habits and communication preferences of each audience, as well as their literacy level and how familiar they are with this type of information. Apply plain language principles to keep content clear and concise, while considering which information will be most important to your audience [3, 4]. When literacy may be limited, consider your format to tailor your reporting through verbal presentation or visual aids such as infographics.
  • Audience reach: A stakeholder audience will vary in how accessible they are to you. As an internal evaluator, if your primary audience is program staff, it may be quite feasible for you to communicate regularly and easily. However, other audiences, such as senior management or funding organizations, may require more effort and resources to reach. Therefore it is important to consider the resources needed to reach your desired audience(s) [3].
  • Cultural and issue appropriateness: Cultural differences should be considered as they could influence communication, behaviour, social norms and values related to the information different audiences receive. Having an awareness of the cultural context and any sensitivity to the information presented should influence how it is delivered to your audience, through selected channels and formats [6].   
  • Cost-effectiveness: Costs associated with reaching your desired audience need to be considered.  Possible channels will need to be assessed on the basis of budget, time and priority [3].

V Channels and formats

Reporting should always be strategic and driven by the evaluation purpose along with the information needs of stakeholders [5]. Various reporting strategies can be used to reach different audiences, with a common goal of organizing and presenting findings to facilitate understanding and promoting use.

Communication channels refer to the paths that are used to reach and exchange information with stakeholders whereas formats are the materials, events, activities or means used to convey a message through these communication channels. You should learn the media habits and communication preferences of your audience to determine the best channels and formats. As outlined previously, these preferences will influence the style, format, content and process of reporting to your intended user [5]. Some stakeholders may want to read concise executive summaries instead of a full report, whereas a stakeholder involved in the day to day operations of the program may be more interested in the detailed version of results [5]. Channels and formats will also need to be considered based on timelines, priorities and available resources [5]. When determining the feasibility of your communication plan, it’s helpful to identify dates for the release of various communications or reports, since often there is an impending decision the evaluation is meant to inform.

Below are examples of common channels and formats conducive to both individual and interactive learning:

Channels:

  • Mass media channels: print, broadcast media
  • New media channels: Facebook, Twitter, LinkedIn, YouTube
  • Interpersonal channels: one-on-one meetings, stakeholder-led meetings
  • Community-specific channels and venues: poetry, theater, churches
  • Professional channels and venues: conference, online forums [6]

Formats:

  • Final reports: Reports are the most common format, typically 20–30 pages long. These allow for the most comprehensive reporting and are often provided to those who will be making a decision with the evaluation results [3].
  • Executive summaries: Another common format, typically one to five pages long. These can emphasize evaluation findings and recommendations concisely without losing the attention of your audience [3].
  • Short communications: Memos, e-mails, bulletins and newsletters can be a cost-effective means for communicating with a broad range of audiences, and can provide a timely update on the activities and findings throughout the evaluation [3].
  • Verbal presentations: A more interactive learning format, best for conveying complex information as this format allows participants to provide feedback and highlight what may not be clear. Working sessions with various stakeholders allow a chance to identify concerns with the evaluation, receive feedback and establish program buy-in.  From these meetings smaller committees can be formed to implement the proposed recommendations that resulted from the evaluation [3].
  • Video presentations: Although these often require more resources, video presentations can assist in documenting the evaluation process and can help disseminate findings to broader audiences. They can also be successful at delivering information to audiences with lower literacy levels [3].

VI Conclusion

Your evaluation efforts are not complete after collecting and analyzing your data. By applying the principles above when developing communication and reporting plans, you can begin to connect with stakeholders in a more strategic, meaningful and engaging way. Ultimately, this will create better understanding of evaluation results and greater likelihood of informed action.

For more information about the evaluation step model and the concepts presented, please watch the OHPE bulletin for the release of our updated introductory evaluation workbook [7]. If you are considering conducting an evaluation, but would like further support, please complete a service request at http://www.publichealthontario.ca/en/ServicesAndTools/HealthPromotionSer... with the Health Promotion Capacity Building Services (HPCB) or email at [email protected] for a tailored consultation.

VII References

Ministry of Health and Long-Term Care. Ontario Public Health Standards 2008. Toronto, ON: Queen's Printer for Ontario; 2014. Available from: http://www.health.gov.on.ca/en/pro/programs/publichealth/oph_standards/d...  
Ontario Agency for Health Protection and Promotion (Public Health Ontario). At a Glance: the ten steps for conducting an evaluation [Internet]. Toronto, ON: Queen’s Printer for Ontario; 2015 [cited 2015 Nov2]. Available from: https://www.publichealthontario.ca/en/eRepository/At_A_Glance_Evaluation...  

Torres R, Preskill H, Piontek ME. Evaluation strategies for communicating and reporting: enhancing learning in organizations. 2nd ed. Thousand Oaks, CA: Sage Publications;2005.

The Plain Language Action and Information Network [homepage on the Internet]. Simple words and phrases; [cited 2015 Mar 17]. Available from: http://www.plainlanguage.gov/howto/wordsuggestions/simplewords.cfm
Patton MQ. Utilization-focused evaluation. 4th ed. Thousand Oaks, CA: Sage Publications; 2008.

Schiavo R. Health communication: from theory to practice. 2nd ed. San Francisco, CA: Jossey-Bass; 2014.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Evaluating health promotion programs: introductory workbook. Toronto, ON: Queen's Printer for Ontario; [forthcoming].

VIII Resources

General
The Plain Language Action and Information Network [homepage on the Internet]. Simple words and phrases; [cited 2015 Mar 17]. Available from: http://www.plainlanguage.gov/howto/wordsuggestions/simplewords.cfm

Knowledge Mobilization Toolkit [homepage on the Internet]. Ontario Centre of Excellence for Child and Youth Mental Health; c2014 Welcome to the knowledge mobilization (KMb) toolkit; [cited 2015 Mar 17]. Available from: http://www.kmbtoolkit.ca/

Data visualizations

Dark Horse Analytics [blog on the Internet]. Data looks better naked; 2013 Aug 03 [cited 2015 Mar 17]. Available from: http://darkhorseanalytics.com/blog/data-looks-better-naked/

Dark Horse Analytics [blog on the Internet]. Clear off the table; 2014 Mar 27 [cited 2015 Mar 17]. Available from: http://darkhorseanalytics.com/blog/clear-off-the-table/

Dark Horse Analytics [blog on the Internet]. Salvaging the pie; 2014 Sep 26 [cited 2015 Mar 17]. Available from: http://darkhorseanalytics.com/blog/salvaging-the-pie/

Ann K. Emery [blog on the Internet]. Ann's blog equipping you to collect, analyze, and visualize data; c2012-2014 [cited 2015 Mar 17]. Available from: http://annkemery.com/

Evergreen Data [blog on the Internet]. Blog; c2014 [cited 2015 Mar 17]. Available from:
http://stephanieevergreen.com/blog/

Evergreen S. Presenting data effectively. 1st ed. Thousand Oaks, CA: Sage Publications; 2014.

Evergreen S, Emery A. Data visualization checklist; 2014 May [cited 2015 Mar 17] Available from: http://stephanieevergreen.com/wp-content/uploads/2014/05/DataVizChecklis...

A Tip-a-Day by and for Evaluators [blog on the Internet]. c2009- [cited 2015 Mar 17]. Available from:
http://aea365.org/blog/category/data-visualization-and-reporting/

Presentations

Duarte N. slideology: The Art and Science of Creating Great Presentations. North Sebastopol, CA: O'Reilly Media; 2008.

Duarte N. Slidedocs: spread ideas with effective visual documents. 2014 Feb 21 [cited 2015 Mar 17]. Available from http://www.slideshare.net/duarte/duarte-slidedocs-15mcafslideshare