Sunday, October 28, 2012

Tobacco Prevention Dissemination - MISRGO Empowerment Evaluation

Tobacco Prevention Dashboard.

Dr. Fetterman and his team, Dr. Tremain and Ms. Delaney, presented the Empowerment Evaluation Tobacco Prevention Dashboard at the American Evaluation Association annual meeting.  The presentation was held on October 27, 2012 in Minneapolis. 

The dashboard is used to facilitate the tobacco prevention program evaluation sponsored by the University of Arkansas at Pine Bluff’s Minority Sub-Recipient Grant Office (MISRGO) and the Arkansas Department of Health. 

The tool is used to monitor and evaluate program performance.

There are 18 tobacco prevention grantees ranging from hospitals to churches. The dashboard is a simple table that specifies the goal, such as 8 smoke free parks by the end of the year; benchmarks, such as 2 during the first quarter, 4 half way through the year, 6 smoke free parks three quarters through the year, and 8 by the end of the year. The findings are reported to the sponsor (MISRGO), the Arkansas Department of Health, Black Caucus, and the State legislature. This session highlights how dashboards are used by grantees to monitor their performance, help align overlapping goals (increasing their dose effect), and help aggregate the data to demonstrate program effectiveness across grantees.

Empowerment Evaluation Dashboard: An Overview

 We are facilitating the empowerment evaluation of these 18 tobacco prevention programs across the State. They have made many contributions to the State including, contributing to policy, referrals to the smoke Quitline, and using social media to reaching youth. The empowerment evaluation dashboard is used to help them monitor and evaluate their own performance. The dashboard has also been useful for external reporting purposes, such as presenting mid-year progress across grantees. This panel highlights how the dashboard has been used to help grantees monitor their own performance, align purposes or goals across grantees (to increase the “dose” effect), and how it helps us merge the data across grantees for legislative reporting purposes.

Empowerment Evaluation Dashboard: 
Monitoring Performance and Increasing the Dose Effect

The empowerment evaluation dashboard is a tool to help people monitor their own performance and see their role in relation to the larger group in terms of dose effect or overall impact. Once a grantee understands the value of the dashboard to monitor their own performance and how the empowerment evaluator is there to help them the relationship is transformed from resistance to one of cooperative constructive engagement. However, it is not a panacea. It is still important to highlights the importance of keeping the data format consistent. The dashboard format also reminds people of how much more effective they can be if they all have more in common to aggregate and report across programs (and grantees). It is the “dose effect”. This concept brings the group much closer together and the dashboard reporting format symbolically helps them to align their purpose.

Empowerment Evaluation Dashboard: 
A Learning Moment and Continued Funding

The dashboard helps to shape and stimulate a learning moment. Asking grantees to report their data in this format forces them to reflect on how realistic the goals are and provides a moment or a space in which to openly discuss how realistic their numbers and expectations are. It also opens the door to discussions about whether the activities are likely to contribute to the desired outcome. The dashboard becomes a tool in which to cultivate understanding, trust, and ownership. The empowerment evaluation dashboard is also a tool to help people see their role in relation to the larger group in terms of continued funding. Once a grantee “gets it” and understands their role in relation to the entire group - how if one group fails to report their accomplishments we all suffer in terms of providing justification for continued funding, their commitment to collect and transmit data is transformed.

Friday, October 5, 2012

MISRGO Tobacco Prevention Empowerment Evaluation Workshop Oct 4, 2012

 MISRGO Empowerment Evaluation Workshop.
Dr. Marian S. Evans-Lee helped to highlight critical points during the workshop, including a reminder about our LGBT tobacco prevention survey (which will go out in October 2012).  

Dr. Fetterman facilitated the workshop and technical assistance training (workshop slides available).  The topics included:  

1.  A summary of last year's annual report

2.  The use of the  evaluation dashboard (including how to make bar graphs from the table)

3.  Sharing what worked and what has not been working.  Sharing best practices and problems was the highlight of the workshop.  

Katherine highlighted her group's policy initiatives and forums planned for the month, as well as recent conferences.

We worked as a team.  One group explained how much trouble they were having reaching the LGBT population.   Another group provided a few recommendations concerning how to reach the LGBT community.

A break through moment occurred when one group stated how effective they had been reaching African Americans and how they have not been successful reaching out to the Hispanic and Latino population.  It was a break through moment because the Latino/Hispanic grantee group explained how that is all that they do.  

Suddenly it became clear how powerful we were when we operated as a group - networking and sharing resources.  The empowerment process was working.

Once we broke through the artificial boundaries associated with our individual grantee groups, we were able to more freely share our insights, understandings, and recommendations - cross cutting grantee concerns.


However, it was not all work.  We also had a few laughs along the way as we began to explore tech tools to enhance our tobacco prevention work.

The workshop concluded with a focus on social media and tech tools.

Dr. Fetterman and Ms. Linda Delaney demonstrated how to use both Skype and Google + Hangouts to videoconference over the internet for free.  These are important (free) tech tools to enhance communication during the year.  (As we make mid-course corrections in the middle of the year it is nice to be able to connect with colleagues and get tips on what to do to improve our performance - particularly when things are not going exactly according to plan.)

Dr. Marian S. Evans-Lee built on the tech section of the workshop day by creating and sharing a Smoke Free Parks Google Map for the group.  This helps us document our efforts and graphically represent the group's efforts in creating smoke free parks across the State.

At the end of the workshop, there was still some time to just share some notes and reflections about the day.  It was a very productive (and high tech) day.