1st Quarter Empowerment Evaluation Workshop

1^st Quarter Empowerment Evaluation Workshop.  The first evaluation workshop of the year was held on October 13, 2011at the Arkansas River Education Service Cooperative in Pine Bluff, Arkansas.

The empowerment evaluation training slides are available here.

Training focused on the:

1) Evaluation Dashboard - Actual Performance, Benchmarks, Goals, and Baseline

The dashboard consists of recording grantee baselines, goals, and benchmarks.  It also included recording actual performance to facilitate the comparison of their benchmarks with their actual performance.  This helps grantees monitor their performance throughout the year and make mid-course corrections (and ask for assistance) if they are not matching their benchmarks.

A hands-on exercise was used to ensure all grantees were able to complete the dashboard in a consistent and accurate manner.

2)  Empowerment Evaluation

 The group is responsible for monitoring and evaluating their own performance, with the assistance of their team of empowerment evaluators (Drs. Fetterman and Tremain and Ms. Delaney).  The process has 3 steps:  1) establishing a mission; 2) taking stock of current status; and 3) planning for the future.

The workshop focused on taking stock since the mission has been established as part of the MISRGO grant process.

Taking Stock is a group baseline exercise.  It has two steps: Prioritization and Rating.


Prioritization. We began by identifying the most important activities for the group to evaluate as a group. (The are the most important activities from the group's perspective, in order to accomplish their goals at that time).  It is impossible to evaluate everything.  Therefore everyone was given 5 dots to cast their vote on the most important activities to evaluate.  

  

The result of this exercise is presented below in a prioritization bar chart.

Bar Chart:  Prioritization

Rating.   We selected the most important activities and rated them.  Grantees rated each activity on a 1 (low) to 10 (high) scale and then discussed the reasons for the ratings

This exercise is designed to record the groups view of their efforts at the beginning of the year.  This exercise will be conducted again later in the year to determine if the group is making progress (as a group) in areas they determined to be critical to accomplishing their objectives.

A bar chart of the ratings is presented below.  Grantee efforts concerning smoke-free workplace policies was rated highly (8.1 on a 10 point scale) - based in large part on their successful efforts to establish smoke free policies, e.g. Act 811.

Bar Chart:  Rating

3) Social Media - YouTube, Twitter & Facebook

A number of grantees are using social media to get the tobacco prevention message across, particularly to youth.  One of the popular social media tools is YouTube.  A demonstration was provided with specific guidelines on how to produce a quality video. 

Example of YouTube Video 

of an 

Empowerment Evaluation in South Africa

The focus of the presentation was on planning and outlining the story before beginning the video editing and production stage.  Storyboarding was used as a tool to help “lay out the story”.  It was demonstrated using a row of individual pictures and a storyline that had a beginning, middle, and ending.

YouTube - Storyboarding with Individual Slides

YouTube:  User-friendly Video Uploading Tool

We produced two YouTube videos during the workshop.  The Coalition for Tobacco-Free Arkansas video been uploaded and is available on YouTube.

Coalition for Tobacco Free Arkansas YouTube Video

Katherine Donald

Grantee Collaboration

In addition, grantees discussed common problems and successful strategies.   Grantees with expertise in policy, such as the Coalition for Tobacco Free Arkansas, shared their tools with grantees with media expertise, specifically the Mississippi Coalition for Tobacco-Free Arkansas.

Katherine Donald
Coalition for Tobacco Free Arkansas

Elroy Brown
Mississippi Coalition for Tobacco-Free Arkansas

These two grantees served as models for other grantees, encouraging them to share what they learned with each other as well.

Evaluation Consultation:  Critical Friends

Grantees were reminded that they were responsible for their own individual evaluations.  However, they were encouraged to use their assigned evaluators, Dr. Beverly Tremain and Ms. Linda Delaney, to help them:

• review the logic of their program and corresponding evaluation
• make sure the activities contributed to their goals
• reassess their strategies or activities if they are not helping them accomplish their goals
• prepare their reports for MISRGO
• monitor their own performance throughout the year

Linda (right) consulting with
Imogene (left) about prioritizing

Bev recording ratings

Leroy, Bev & Linda
discussing reporting

MISRGO Reminders

MISRGO reminded the grantees to work with their evaluators and turn in their dashboards and quarterly reports on time to help facilitate analysis across grantees and to help MISRGO meet its reporting requirements.