Friday, February 22, 2013

2nd Quarter MISRGO Tobacco Prevention Empowerment Evaluation Report



2nd Quarter MISRGO Tobacco Prevention 
Empowerment Evaluation
Progress Report 2013

Prepared for:            MISRGO, Grantees, and Legislative Offices
Prepared by:             Dr. Fetterman, Ms. Delaney, and Dr. Tremain
Date:                          February 22, 2013
                       









Introduction

Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, 2011).  The annual health care costs in Arkansas directly caused by smoking is $812 million, according to the Arkansas Department of Health (2009) and the Campaign for Tobacco Free Kids (2012).  Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.

 The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received Settlement funds, through the Arkansas Department of Health, to focus on tobacco use in minority communities. MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.

MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs.  MISRGO currently sponsors and provides technical assistance for 16 grantees across the State of Arkansas.

MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work.  (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
 
MISRGO grantees also have a strong track record of success based on past performance.  For example, last year, the majority of MISRGO grantees met or exceeded their annual performance goals (See MISRGO Evaluation Annual Report 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012).

This evaluation report highlights MISRGO tobacco prevention grantee progress during the second quarter of the year (October 1 – December 31, 2012).   This period represents the mid-way point of the year.  The fiscal and programmatic end of the year is June 30, 2013. 

This second quarter report suggests MISRGO grantees are well positioned to have another successful year.  (See Appendix A for a glossary of grantee names and abbreviations and a map of their locations throughout the State.)

MISRGO Grantee Intervention Areas

MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas.  They include:

Area 1:  Eliminate Exposure to Secondhand Smoke
Area 2:  Preventing Initiation Among Youth and Young Adults
Area 3:  Promoting Quitting Among Youth and Adults
Area 4:  Addressing Disparities

Grantee progress and activities are reported on a quarterly and annual basis, according to these CDC intervention areas. 
 
2nd Quarter Progress

During the first quarter, grantees established their goals, benchmarks, and baselines.   Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks.  Benchmarks are intermediate objectives or approximations of outcomes.  Baselines represent, for example, the number of smoke-free environments that existed before the grantee activity or intervention. 

During the second quarter, grantees report on their mid-year progress toward annual goals.  This provides grantees with an opportunity to reflect on their performance.  Based on these data, grantees continue as planned, make mid-course changes, and request assistance as needed.

MISRGO grantees are making progress toward annual goals.  On average, grantees have made progress in each CDC approved intervention area:

                                           Intervention Area     Percent Met/Exceeded
                                                                                       Annual Goals

                                              1.  Eliminating                          25%
                                              2.  Preventing                           19%
                                              3.  Promoting                           19%
                                              4.  Addressing                          44%




Highest Level of Progress Toward Annual Goals:  Addressing Disparities (Area 4)

Concerning CDC Intervention Areas, grantees have made the most progress at mid-point in the year toward their annual goals in Intervention Area 4:  Addressing Disparities (44% of annual goals). 

Grantee activities focused on implementing Project Alert Curriculums into school classes; educating a total of 39,487 underserved residents on the dangers of tobacco products through radio and television spots; launching media surveys with residents; and assessing community leaders/organizations/retail store owners' awareness of how tobacco industry targets minorities; and implementing a Smoke Free Basketball Campaign.

Second Highest Level of Progress Toward Annual Goals:  Eliminate Exposure to Secondhand Smoke (Area 1)

Grantees have also made notable progress concerning Intervention  Area 1:  Eliminating Exposure to Secondhand Smoke (25% of annual goals).

Grantee activities focused on surveying law enforcement officers to determine the most effective means of enforcing Act 811; conducting radio campaigns to emphasize the dangers of second hand smoke; and providing education to housing authority directors, childcare centers staff, Head start staff, Boys & Girls Club of Mississippi County staff, and teachers and school staff.   In addition, grantees pre/post-tested parents on Act 811.

Third Highest Level of Progress Toward Annual Goals:  Preventing Initiation Among Youth and Young Adults (Area 2)

Grantees are making progress in Area 2: Preventing Initiation Among Youth and Young Adults (19% of annual goals). 

Grantee activities focused on store-front surveys, tobacco free campaigns and youth rallies, training youth in prevention strategies, securing tobacco free pledges, and conducting compliance checks on sales to minors.

Tied for Third Highest Level of Progress Toward Annual Goals:  Promoting Quitting Among Youth and Adults (Area 3)

Grantees are also making progress in Area 3:  Promoting Quitting (19% of annual goals). 

Grantee activities focused on distributing information about Quitline, referring community members to Quitline, and enrolling community members in cessation programs.  In addition, they offered the “40 Days to Freedom” curriculum, and organized tobacco free rallies.

An evaluation dashboard for each grantee’s 2nd quarter performance is provided to facilitate communication and collaboration across grantees.  It is organized by CDC intervention area (see Appendix B). The cumulative MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix C).  

Technical Assistance

MISRGO grantees receive programmatic and evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.   Grantee’s participated in an evaluation workshop on October 4, 2012, highlighting the use of the evaluation dashboard, recording baseline, goals, benchmarks, and actual performance. 


The evaluation dashboard is used to help grantees monitor their own progress throughout the year. MISRGO and evaluation team members also use it to enhance accountability, signaling precisely when quarterly and mid-year assistance is required to increase the probability of grantees accomplishing their end-of-year goals.  This self-evaluation approach complemented with external review is in accordance with an empowerment evaluation approach.  This approach is designed to build capacity and produce outcomes.  (Fetterman, 2013; Fetterman and Wandersman, 2005.)

Grantees are provided with additional tobacco prevention tools and updates to facilitate their programmatic and evaluative efforts  (See Appendix D for an example of a periodic technical assistance communication.) 

Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com/2012/10/misrgo-tobacco-prevention-evaluation.html.

Conclusion

MISRGO grantees have established baselines, goals, and benchmarks during their first quarter.  During their 2nd quarter, they have compared actual performance with benchmarks and goals.  MISRGO and grantees are monitoring their performance.  This data dashboard was used to alert MISRGO and grantees concerning the need for mid-course corrections.  This is in accordance with empowerment evaluation principles and guidelines, (Fetterman, 2013; Fetterman and Wandersman, 2005). Actual performance data indicates grantees are making progress toward annual goals.





References

Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
ARKANSAS STRATEGIC PLAN TO PREVENT AND REDUCE TOBACCO USE 2009 – 2014.  Little Rock, Arkansas:  Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf

Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2011). Tobacco Use.  Targeting the Nation’s Leading Killer
- At A Glance 2011.  Atlanta, Georgia:  Centers for Disease Control and Prevention.

Campaign for Tobacco Free Kids (2011). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

Fetterman, D.M. (2013).  Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice.  Stanford:  Stanford University Press.

Fetterman, D.M., Tremain, B., and Delaney, L. (2012).  MISRGO Evaluation Annual Report 2011-2-12.  July 12, 2012. San Jose: Fetterman & Associates.

Fetterman, D.M., Tremain, B., and Delaney, L. (2011).  MISRGO Empowerment Evaluation:  June 2011 Annual Report.  San Jose: Fetterman & Associates.

Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice.  New York: Guilford Publication.




Appendix A - Grantee Glossary of Names and Abbreviations




Map of Grantee Locations Throughout Arkansas


(Google Sites map courtesy of Dr. Marian Evans-Lee)




Appendix B  - Individual Grantee 2nd Quarter Progress
Evaluation Dashboards









Intervention Area 2:  Preventing Initiation Among Youth & Young Adults









Area 3:  Promoting Quitting Among Youth and Young Adults









Intervention Area 4:  Addressing Disparities












Area 1:  Eliminate Exposure to Secondhand Smoke



Appendix C – Cumulative MISRGO Grantee Activity Log

A Quarterly Activity Log is an additional monitoring tool.  It is used to supplement the evaluation dashboards.  It enables grantees to record activities used to implement their programs and accomplish their objectives.  It also enables MISRGO and the empowerment evaluators to determine if grantees are on track and provide assistance as needed.

Minority Initiative Sub-Recipient Grant Office 2012-2013

Quarterly

QTR 1
July 1 - September 30
QTR II
Oct 1 - Dec 31
QTR III
Jan 1 - March 31
QTR IV
April 1 - June 30
Coalition members
2121
2153


Coalition meetings
32
71


Adult pledges
608
365


Youth pledges
637
995


Smoke-free home/car pledges
523
767


Promotional items distributed
2564
2753


Educational materials distributed
4019
9488


Newsletters distributed
0
13


Radio spot(s)
1097
402


Reach of radio spot(s)
1,785,714
19,562,886 (statewide)


Newspaper ad(s)
21
16


Reach of newspaper ad(s)
68,486
447,600


Television ad(s)
150
1


Reach of television ad(s)
23,840 *HHs
85 mile radius (Central AR)


Fax Back referrals
49
142


Individuals telephoned the Quitline
78
237 (some unknown)


Billboard(s)
4
4


Reach of billboard(s)
555,928
442,012


Website Hits
50,841
458


Facebook Impressions
1033
1111


Twitter Followers
487
542 (Some not captured)


TV Spots-Press Conference
23*
20*


TV Interviews
Katherine 3 & Stan Glantz 3
3 valued @ more than $33,000.00*


*Coalition for a Tobacco Free AR
FYEN: 19 initial phone contacts were made to coalition members in October, followed by an additional 19 calls - also in October.  Calls were made to remind them of the legislator's luncheon that was conducted for the Southeast Region last October in Pine Bluff.
2. Radio spots: 22 days x 835,098 listeners daily Q2 in Lee/Phillips and surrounding Delta area (Q1 correction=28 days x 835,098 KJIW listeners daily x 22 days = 23,382,744 + 740,000 KCET listeners daily x 30 days = 45,582,744 listeners during Q1).




Appendix D - Sample of Email Technical Assistance

Tobacco Prevention Sites

1. County Health Rankings: http://www.countyhealthrankings.org/#app/
The County Health Rankings rank the health of nearly every county in the nation and show that much of what affects health occurs outside of the doctor’s office. The County Health Rankings confirm the critical role that factors such as education, jobs, income, and environment play in how healthy people are and how long they live. Arkansas County data is available at this site.

2. Campaign for Tobacco Free Kids: “The Toll of Tobacco in Arkansas” has information concerning tobacco use and impacts in Arkansas:
http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

3. World Health Organization (WHO): Anatomy of a Smoke Free Ordinance. A common request of those developing smoke-free legislation is, “Do you have an example or model we can use?” They provide an example of an ordinance.

4County Health Calculator: The County Health Calculator is a tool for advocates and policy makers. Health is about more than healthcare. It is about education, income and the environment in which we live. Use the calculator to see how education and income affect health outcomes for Arkansas.http://countyhealthcalculator.org/location/105000/


Web Tools

1. Word Clouds - Wordle:  http://www.wordle.net/
Wordle is a web-based tool for generating “word clouds” from text that you provide. The images give greater prominence to words that appear more frequently in the source text. Coalitions in the past have used Wordle to create t-shirts with prevention messages and anti-smoking language on them. It's a great tool for social marketing and health messages or for your website.

2Online Surveys - Survey Monkey:  http://www.surveymonkey.com/.SurveyMonkey is the world's most popular online survey tool. It's easier than ever to send free surveys, polls, questionnaires, customer feedback and market research. Plus get access to survey questions and professional templates. Free accounts limit you to 10 questions and 100 reponses and paid accounts have more features.

3.  Online Surveys – Forms:  Google forms is an alternative online survey tool that is completely free, but not as fancy or user-friendly as Survey Monkey (www.formsite.com).