Saturday, March 1, 2014

Mid-Year MISRGO Tobacco Prevention Empowerment Evaluation Report 2013-2014



MISRGO Evaluation
2nd Quarter Report 2013-2014
Prepared for:   MISRGO, Grantees & Legislative Offices
Prepared by:   Dr. Fetterman, Ms. Delaney, and Dr. Tremain
Date:               February 27, 2014
                       







Introduction

Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, 2011).  The annual health care costs in Arkansas directly caused by smoking is $812 million, according to the Arkansas Department of Health (2009) and the Campaign for Tobacco Free Kids (2013).  Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.

 The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received Settlement funds through the Arkansas Department of Health to focus on tobacco use in minority communities. MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.

MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs.  MISRGO currently sponsors and provides technical assistance for 19 grantees across the State of Arkansas.

MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work.  (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
 
MISRGO grantees also have a strong track record of success based on past performance.  For example, last year, the majority of MISRGO grantees met or exceeded their annual performance goals (see MISRGO Evaluation Annual Report 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012 and MISRGO Evaluation Annual Report 2012-2013 Fetterman, Delaney, Tremain, July 18, 2013a).  See also Fetterman, Delaney, and Tremain 2013b and 2013c for interim reports on progress.

Purpose

The purpose of this report is to present MISRGO tobacco prevention grantee progress at the end of the 2nd Quarter 2013-2014.  This is a mid-year report on progress. The fiscal and programmatic end of the year is June 30, 2014.  (See Appendix A for a glossary of grantee names.)

MISRGO Grantee Intervention Areas

MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas.  They include:

Area 1:  Eliminate Exposure to Secondhand Smoke
Area 2:  Preventing Initiation Among Youth and Young Adults
Area 3:  Promoting Quitting Among Youth and Adults
Area 4:  Addressing Disparities

Grantee progress and activities are reported on both a quarterly and annual basis, according to these CDC intervention areas. 
 
Quarterly Progress

During the first quarter, MISRGO grantees established their goals, benchmarks, and baselines.   Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks.  Benchmarks are intermediate objectives or approximations of outcomes.  Baselines represent, for example, the number of smoke-free environments that existed before the grantee activity or intervention. 

During the second quarter (the focus of this report), grantees report on their mid-year progress toward annual goals.  This provides grantees with an opportunity to reflect on their performance.  Based on these data, grantees continue as planned, make mid-course changes, and request assistance as needed.

The third quarter focuses on any gaps between benchmarks and annual goals.  Grantees share techniques with each other, MISRGO staff provide additional recommendations, and Fetterman & Associates notify grantees about gaps and make specific recommendations to close the gap. 

The fourth quarter is the annual comparison of their goals with their actual performance throughout the year.

2nd Quarter Findings

The findings are mixed at mid-year.  Grantees have met or exceeded their 2nd Quarter evaluation benchmarks concerning promoting quitting among youth and adults and addressing disparities.  However, they have not met 2nd quarter benchmarks concerning the CDC intervention areas:  eliminating exposure to secondhand smoke and preventing initiation among youth and young adults:

Area 1:  Eliminate Exposure to Secondhand Smoke  (37% met or exceeded 2nd Qtr benchmarks)
Area 2:  Preventing Initiation Among Youth and Young Adults (42% met or exceeded 2nd Qtr benchmarks)
Area 3:  Promoting Quitting Among Youth and Adults (68% met or exceeded 2nd Qtr benchmarks)
Area 4:  Addressing Disparities (68% met or exceeded 2nd Qtr benchmarks)

Intervention Area
% Met-Exceeded 2nd Quarter Benchmarks

% Not Met 2nd Quarter Benchmarks



1.  Eliminating
37%
63%
2.  Preventing
42%
58%
3.  Promoting
68%
32%
4.  Addressing
68%
32%










CDC Area 1:  Eliminate Exposure to Secondhand Smoke

Approximately, 37% of the grantees have met or exceeded their 2nd Quarter evaluation benchmarks concerning Area 1:  Eliminate Exposure to Secondhand Smoke.  This suggests additional attention is needed in this area to met annual goals.

Grantee activities focused on:

·      Encouraging voluntary smoke-free perimeter policies
·      Educating parents and law enforcement officers about the benefits of smoke free environments in the car (Act 811)
·      Promoting Act 811
·      Gathering petition signatures for smoke free environments
·      Increase public support for smoke-free air policies
·      Promoting extracurricular activities designed to increase knowledge about smoking and how it affects nonsmoking individuals
·      Encouraging the adoption of tobacco-free policies
·      Planning to implement tobacco prevention training, such as Project Alert or Banner’s in the Schools
·      Planning to implement a plan to produce smoke free family cars
·      Establishing smoke-free parks
·      Increasing the number of community organizations that support smoking bans/policies
·      Increasing youth awareness about the dangers of tobacco products (surveys and outreach)

CDC Area 2:  Preventing Initiation Among Youth and Young Adults

42% of the grantees met or exceeded their benchmarks concerning Area 2: Preventing Initiation Among Youth and Young Adults.  This suggests additional attention is needed in this area to met annual goals.

Grantee activities focused on:

·      Conducting store-front surveys
·      Providing training in prevention strategies through media outreach, gorilla marketing campaigns, and youth rallies
·      Encouraging youth and young adults to pledge not to initiate tobacco use
·      Educating African American and Latino youth on the effects of smoking
·      Conducting compliance checks on retailer sales to minors
·      Increasing the use of the Quitline
·      Increasing the number of youth oriented tobacco prevention events
·      Educating youth and young adults about the dangers of tobacco

CDC Area 3:  Promoting Quitting Among Youth and Adults

The majority of grantees (68%) have met or exceeded their benchmarks concerning Area 3:  Promoting Quitting.   Grantees are on track concerning projected annual goals.

Grantee activities focused on:

·      Increasing use of the Arkansas Quitline
·      Encouraging African Americans and Latinos to participate in cessation programs
·      Encouraging African-American males to use the Arkansas Quitline
·      Helping to start a tobacco youth program where students learn about smoking
·      Helping to increase community partners
·      Increasing Hispanic calls to SOS Quitline
·      Decreasing chewing tobacco use
·      Increasing number participating in 40 Days to Freedom programs

CDC Area 4:  Addressing Disparities

68% of the grantees have met or exceeded their benchmarks concerning Area 4:  Addressing Disparities.  Grantees are on track concerning projected annual goals.

Grantee activities focused on:

·      Sharing information about current tobacco laws
·      Improving access to existing cessation resources for Hispanic males
·      Developing smoke free policies or point of purchase policy to reduce advertising
·      Creating diverse partnerships to address tobacco-related disparities
·      Increasing African-American and Latino post-test scores in awareness
·      Educating African American and Latino residents concerning tobacco related health disparities
·      Increasing the number of minorities pledging to keep themselves or their homes and vehicles tobacco free
·      Referring people to the SOS Quitline and/or other local cessation resources
·      Planning to organize Youth Extinguishing Smoking teams (Y.E.S. Teams)
·      Increasing the number of churches implementing the 40 Days to Freedom program
·      Increasing minorities’ awareness of tobacco related disparities

An evaluation dashboard for each grantee’s annual performance is provided to facilitate communication and collaboration among grantees, enhance MISRGO accountability, and build knowledge.  It is organized by CDC intervention area (see Appendix B). The MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix C).  

Technical Assistance

MISRGO grantees receive programmatic and evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.  

Primarily new grantees participated in an evaluation workshop on October 3, 2013.  At the workshop new grantees were re-introduced to the 3 steps of empowerment evaluation.  They openly discussed obstacles and generously shared advice and best practices, based on their own experience.

New grantees were also reacquainted with the evaluation dashboard, including recording baseline, goals, benchmarks, and actual performance.   As a result of budget constraints, grantees will probably only receive one evaluation technical assistance workshop this year.


Nevertheless, the evaluation dashboard has been used to help grantees monitor their own progress throughout the year. MISRGO and evaluation team members have also used it to enhance accountability, signaling precisely when assistance is required to increase the probability of grantees accomplishing their end-of-year goals. 


The evaluation dashboards are designed to build capacity and produce outcomes.  They are in accordance with an empowerment evaluation approach.  (Fetterman, 2013; Fetterman and Wandersman, 2005; Fetterman, Kaftarian, and Wandersman, in press.)

Grantees are provided with additional tobacco prevention tools and updates to facilitate their programmatic and evaluative efforts  (See Appendix D for an example of a periodic technical assistance communication.) 

Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com

Conclusion

MISRGO grantees established their goals, benchmarks, and baselines during the first quarter of the year.  In addition, they continue to document their actual performance.  The evaluation dashboards help grantees monitor their own performance.   They are being used to confirm effective performance and alert MISRGO and grantees concerning the need for corrections throughout the year. At present grantees are meeting or exceeding only half of the CDC intervention areas.  This signals the need to provide assistance to help grantees meet their annual goals.

MISRGO grantees are dedicated, committed, and hard working concerning tobacco prevention.  They also document progress toward annual programmatic and evaluation goals.  However, this is a time for mid-course corrections and assistance.




References

Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
Arkansas Strategic Plan to Prevent and Reduce Tobacco Use 2009 -2014.  Little Rock, Arkansas:  Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf

Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2011). Tobacco Use.  Targeting the Nation’s Leading Killer
- At A Glance 2011.  Atlanta, Georgia:  Centers for Disease Control and Prevention.

Campaign for Tobacco Free Kids (2013). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

Fetterman, D.M. (2013).  Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice.  Stanford:  Stanford University Press.

Fetterman,  D., Delaney, L., Tremain, B. (2013a).  MISRGO Evaluation Annual Report 2012-2013.  San Jose:  Fetterman & Associates (July 18, 2013).

Fetterman, D.M., Delaney, L., and Tremain, B. (2013b).  MISRGO Evaluation 2nd Quarter Progress Report 2013.  San Jose:  Fetterman & Associates.

Fetterman, D.M., Delaney, L., and Tremain, B. (2013c).  MISRGO Evaluation 3rd  Quarter Progress Report 2013.  San Jose:  Fetterman & Associates

Fetterman, D.M., Tremain, B., and Delaney, L. (2012).  MISRGO Evaluation Annual Report 2011-2-12.  July 12, 2012. San Jose: Fetterman & Associates.

Fetterman, D.M., Tremain, B., and Delaney, L. (2011).  MISRGO Empowerment Evaluation:  June 2011 Annual Report.  San Jose: Fetterman & Associates.

Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice.  New York: Guilford Publication.