Tuesday, December 4, 2012

1st Quarter MISRGO Empowerment Evaluation Report 2012


MISRGO Evaluation
1st Quarter Progress Report 2012
Prepared for:            MISRGO, Grantees & Legislative Offices
Prepared by:             Dr. Fetterman, Ms. Delaney, and Dr. Tremain
Date:                          November 30, 2012
                       






Introduction

Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, 2011).  The annual health care costs in Arkansas directly caused by smoking is $812 million, according to the Arkansas Department of Health (2009) and the Campaign for Tobacco Free Kids (2012).  Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.

 The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received Settlement funds, through the Arkansas Department of Health, to focus on tobacco use in minority communities. MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.

MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs.  MISRGO currently sponsors and provides technical assistance for 18 grantees across the State of Arkansas.

MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work.  (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
 
MISRGO grantees also have a strong track record of success based on past performance.  For example, last year, the majority of MISRGO grantees met or exceeded their annual performance goals (See MISRGO Evaluation Annual Report 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012).

This evaluation report highlights MISRGO tobacco prevention grantee progress during the first quarter of the year (July 1 – September 30, 2012).   This period represents the starting point for the year.  The fiscal and programmatic end of the year is June 30, 2013. 

This first quarter report suggests MISRGO grantees are well positioned to have another successful year.  (See Appendix A for a glossary of grantee names and abbreviations and a map of their locations throughout the State.)

MISRGO Grantee Intervention Areas

MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas.  They include:

Area 1:  Eliminate Exposure to Secondhand Smoke
Area 2:  Preventing Initiation Among Youth and Young Adults
Area 3:  Promoting Quitting Among Youth and Adults
Area 4:  Addressing Disparities

Grantee progress and activities are reported on a quarterly and annual basis, according to these CDC intervention areas. 
 
1st Quarter Progress

During the first quarter grantees established their goals, benchmarks, and baselines.   Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks.  Benchmarks are intermediate objectives or approximations of outcomes.  Baselines represent, for example, the number of smoke -free environments that existed before the grantee activity or intervention. 

This is also the first quarter actual performance is reported, e.g. the number of smoke-free parks established. 

The majority of grantees report low to zero benchmarks and actual performance figures because their outcomes are geared toward the end of the year.  In essence, the majority of grantees by design had not planned to have results by the end of the first quarter.  This time was set aside for planning and implementing initial steps and activities required to produce end-of-the-year outcomes.  However, MISRGO grantees are making progress toward annual goals.  On average, grantees have made progress in each CDC approved intervention area:



Grantees have made the most progress concerning Intervention Area 3:  Promoting Quitting Among Youth and Adults (on average 45% of annual goals).  Grantee activities focused on distributing information about the Quitline, referring community members to the Quitline, and enrolling community members in cessation programs.  In addition, they offered the “40 Days to Freedom” curriculum, and organized tobacco free rallies.

Grantees have also made progress concerning Intervention Area 2: Preventing Initiation Among Youth and Young Adults (on average 22% of annual goals).  Grantee activities focused on store front surveys, tobacco free campaigns and youth rallies, training youth in prevention strategies, securing tobacco free pledges, and conducting compliance checks on sales to minors.

An evaluation dashboard for each grantee’s 1st quarter performance is provided to facilitate communication and collaboration across grantees.  It is organized by CDC intervention area (see Appendix B). The cumulative MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix D).  

Technical Assistance

MISRGO grantee’s receive programmatic and evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.   Grantee’s participated in an evaluation workshop on October 4, 2012, highlighting the use of the evaluation dashboard, including baseline, goals, benchmarks, and actual performance. 


The evaluation dashboard is used to help grantees monitor their own progress throughout the year. MISRGO and evaluation team members also use it to enhance accountability, signaling precisely when quarterly and mid-year assistance is required to increase the probability of grantees accomplishing their end-of-year goals.  This self-evaluation approach complemented with external review is in accordance with an empowerment evaluation approach.  This approach is designed to build capacity and produce outcomes.  (Fetterman, 2013; Fetterman and Wandersman, 2005.)

Grantees are provided with additional tobacco prevention tools and updates to facilitate their programmatic and evaluative efforts  (See Appendix C for an example of a periodic technical assistance communication.) 

Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com/2012/10/misrgo-tobacco-prevention-evaluation.html.

Conclusion

MISRGO grantees have established baselines, goals, and benchmarks.  In addition, they are comparing actual performance with benchmarks and goals.  MISRGO and grantees are monitoring their performance.  This tool is used to alert MISRGO and grantees concerning the need for mid-course corrections as needed throughout the year.  This is in accordance with empowerment evaluation principles and guidelines, (Fetterman, 2013; Fetterman and Wandersman, 2005). Actual performance data indicates grantees are making progress toward annual goals.



References

Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
ARKANSAS STRATEGIC PLAN TO PREVENT AND REDUCE TOBACCO USE 2009 – 2014.  Little Rock, Arkansas:  Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf

Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2011). Tobacco Use.  Targeting the Nation’s Leading Killer
- At A Glance 2011.  Atlanta, Georgia:  Centers for Disease Control and Prevention.

Campaign for Tobacco Free Kids (2011). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

Fetterman, D.M. (2013).  Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice.  Stanford:  Stanford University Press.

Fetterman, D.M., Tremain, B., and Delaney, L. (2012).  MISRGO Evaluation Annual Report 2011-2-12.  July 12, 2012. San Jose: Fetterman & Associates.

Fetterman, D.M., Tremain, B., and Delaney, L. (2011).  MISRGO Empowerment Evaluation:  June 2011 Annual Report.  San Jose: Fetterman & Associates.

Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice.  New York: Guilford Publication.




Appendix A - Grantee Glossary of Names and Abbreviations



Map of Grantee Locations Throughout Arkansas


(Google Sites map courtesy of Dr. Marian Evans-Lee)



Appendix B  - Individual Grantee 1st Quarter Progress
Evaluation Dashboards

Area 1:  Eliminate Exposure to Secondhand Smoke







Area 2:  Preventing Initiation Among Youth and Young Adults







Area 3:  Promoting Quitting Among Youth and Young Adults







Area 4:  Addressing Disparities










Appendix C - Sample Technical Assistance Update Email


Hello Grantees:

Hope your holidays went well! Please see below for some resources from Linda and me that might be useful in your work. Please take a moment to visit these sites or read the article.

Also, I have reviewed all of your dashboards and quarterly reports and will continue to follow up with any of you for follow-up questions as needed. Otherwise, you will receive copies of the overall grantee report. Please do not hesitate to call me at XXX or through email if you have any questions about your work.

1. County Health Rankings. http://www.countyhealthrankings.org/#app/
The County Health Rankings rank the health of nearly every county in the nation and show that much of what affects health occurs outside of the doctor’s office. The County Health Rankingsconfirm the critical role that factors such as education, jobs, income, and environment play in how healthy people are and how long they live. Check out Arkansas County data now!

2. Beautiful Word Clouds - http://www.wordle.net/
Wordle is a toy for generating “word clouds” from text that you provide. The clouds give greater prominence to words that appear more frequently in the source text. Coalitions in the past have used wordle to create t-shirts with prevention messages and anti-smoking language on them. It's a great tool for social marketing and health messages or for your website.

3. Updated (as of November 9, 2012) stats on “The Toll of Tobacco in Arkansas” from the Campaign for Tobacco Free Kids:



Appendix D – Cumulative MISRGO Grantee Activity Log

A Quarterly Activity Log is an additional monitoring tool.  It is used to supplement the evaluation dashboards.  It enables grantees to record activities used to implement their programs and accomplish their objectives.  It also enables MISRGO and the empowerment evaluators to determine if grantees are on track and provide assistance as needed.