Sunday, December 20, 2015

1st Quarter Tobacco Prevention Empowerment Evaluation Report 2015-2015


MISRGO Evaluation
1st Quarter Report 2015-2016
Prepared for:   MISRGO, Grantees & Legislative Offices
Prepared by:   Dr. Fetterman and Ms. Delaney
Date:               December 20, 2015
                       






 Introduction

Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, November 2014).  The annual health care costs in Arkansas directly caused by smoking is $1.21 billion, according to the Campaign for Tobacco Free Kids (2015).  The tobacco industry spends over $119 million marketing tobacco in Arkansas.  Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.  (See also Arkansas Department of Health (2009).)  However, “as of fiscal year 2010, Arkansas allocated 54.4% of the CDC-recommended funding for tobacco control ($19.8 million of $36.4 million).”[i]

 The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received a portion of the Settlement funds through the Arkansas Department of Health to focus on tobacco use in minority communities. MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.

MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs.  MISRGO currently sponsors and provides technical assistance for 14 grantees across the State of Arkansas.

MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work.  (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
 
MISRGO grantees also have a strong track record of success based on past performance.  For example, in the last 4 years, the majority of MISRGO grantees met or exceeded their annual performance goals. (See MISRGO Empowerment Evaluation Annual Report, 2014-2015, Fetterman and Delaney; MISRGO Evaluation Annual Report 2013-2014 Fetterman, Delaney, and Tremain, July 11, 2014; 2012-2013 Fetterman, Delaney, Tremain, July 18, 2013a; and 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012). 

Purpose

The purpose of this report is to present MISRGO tobacco prevention grantee progress at this point in the year - the end of the 1st Quarter 2015-2016.  The fiscal and programmatic end of the year is June 30, 2016.  (See Appendix A for a glossary of grantee names.)

MISRGO Grantee Intervention Areas

MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas.  They include:

Area 1:  Eliminate Exposure to Secondhand Smoke
Area 2:  Preventing Initiation Among Youth and Young Adults
Area 3:  Promoting Quitting Among Youth and Adults
Area 4:  Addressing Disparities

Grantee progress and activities are reported on both a quarterly and annual basis, according to these CDC intervention areas. 
 
Quarterly Progress

During the first quarter (the focus of this report), MISRGO grantees established their goals, benchmarks, and baselines.   Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks.  Benchmarks or milestones are intermediate objectives or approximations of outcomes.  Baselines represent, for example, the number of smoke-free environments that existed before the grantee activity or intervention. 

During the second quarter, grantees report on their mid-year progress toward annual goals.  This provides grantees with an opportunity to reflect on their performance.  Based on these data, grantees continue as planned, make mid-course changes, and/or request assistance as needed.

The third quarter focuses on any gaps between benchmarks or milestones and annual goals.  Grantees share techniques with each other, MISRGO staff provide additional recommendations, and Fetterman & Associates notify grantees about gaps and make specific recommendations to close the gap. 

The fourth quarter is the annual comparison of their goals with their actual performance throughout the year.

1st Quarter Findings

100% of MISRGO grantees have established their goals, benchmarks, and baselines for the year.   Between 21% and 43% of the grantees have met[ii] or exceeded their 1st Quarter evaluation benchmarks:


Area 1:  Eliminate Exposure to Secondhand Smoke  (21% met or exceeded 1st Qtr benchmarks)
Area 2:  Preventing Initiation Among Youth and Young Adults (21% met or exceeded 1st Qtr benchmarks)
Area 3:  Promoting Quitting Among Youth and Adults (43% met or exceeded 1st Qtr benchmarks)
Area 4:  Addressing Disparities (29% met or exceeded 1st Qtr benchmarks)



CDC Area 1:  Eliminate Exposure to Secondhand Smoke

21% of the grantees have met or exceeded their 1st Quarter evaluation benchmarks concerning Area 1:  Eliminate Exposure to Secondhand Smoke.

Grantee activities focused on:

·         Encourage voluntary smoke-free perimeter policies
·         Conduct secondhand smoke tobacco prevention messaging
·         Help establish smoke-free park policies
·         Encourage tobacco-free policies
·         Implement anti-smoking media campaigns (via radio, social media and television)
·         Promote smoke-free home and car campaign
·         Educate about the impact of marketing strategies
·         Educate about the benefits of smoke free environments in the car (ACT 811) and in the home
·         Conduct operation storefront/STARTS surveys
·         Organize tobacco free workshops/coalition meetings

CDC Area 2:  Preventing Initiation Among Youth and Young Adults

21% of the grantees have also met or exceeded their 1st Quarter evaluation benchmarks concerning Area 2: Preventing Initiation Among Youth and Young Adults.

Grantee activities focused on:

·         Conduct Operation Store Front/STARS surveys
·         Conduct retailer visits regarding sales to minors
·         Encourage smoke-free or tobacco-free church campus policy
·         Use media, include radio campaigns, social media and promotions, to inform youth about dangers of    smoking (focus on African Americans and Latina youth)
·         Educate youth about the effects of smoking
·         Help to implement point of purchase policy
·         Partner with organizations to prevent initiation of tobacco use by youth (presentations)
·        Encourage youth to pledge to be tobacco free
·         Train youth in prevention strategies through media outreach and youth rallies
·         Use media to educate about prevention strategies

CDC Area 3:  Promoting Quitting Among Youth and Adults

43% of the grantees have met or exceeded their 1st Quarter evaluation benchmarks concerning Area 3:  Promoting Quitting. 

Grantee activities focused on:

·         Encourage use of Arkansas Quit-line Fax Referral System
·         Encourage youth to pledge to quit
·         Work with churches, schools and peers to help friends quit smoking
·         Partner with community organizations to host or take part in events, presentations, and/or activities    to help people quit smoking
·         Help increase the number of participants in cessation programs
·         Educate through radio/social media and print about the dangers of tobacco products
·         Educate people about cessation services
·         Present information (tobacco facts) concerning the effects of tobacco
·         Encourage people to participate in the established cessation programs
·         Educate about the dangers of tobacco through evidence-based strategies
·         Educate about cessation services using surveys, education activities, and media outreach

CDC Area 4:  Addressing Disparities

29% of the grantees have also met or exceeded their 1st Quarter evaluation benchmarks concerning Area 4:  Addressing Disparities. 

Grantee activities focused on:

·         Provide information about dangers of tobacco use and secondhand smoke and the importance of     tobacco laws
·         Encourage people to call and enroll with the Arkansas Tobacco Quitline (focus on African American    and Hispanic women)
·         Help increase the number of voluntary smoke-free home policies
·         Encourage people to make smoke-free home pledges
·         Educate about tobacco disparities using focus groups and media campaigns (addressing tobacco-   related health disparities (focusing on African American and Latino tobacco users)
·         Help to increase knowledge about tobacco policies and regulations regarding youth
·         Conduct media campaigns to increase awareness of tobacco dangers and tobacco industry    advertising tactics using radio, social media, billboards, and flyers (disproportionately targeting    minorities)
·         Educate about the hazards of secondhand smoke through a radio, print, and social media campaign    (focusing on African American males)
·         Educate tobacco retailers about the laws associated with selling tobacco to underage minority youth
·         Encourage smoke-free policy or point of purchase policy to reduce advertising to youth and    disparity groups
·         Help increase the number of minorities who acknowledge tobacco related disparities and the    practices used by the tobacco industry to create these disparities
·         Educate underserved about the ills of tobacco products through surveys and media outreach

An evaluation dashboard for each grantee’s annual performance is provided to facilitate communication and collaboration among grantees, enhance MISRGO accountability, and build knowledge.  It is organized by CDC intervention area (see Appendix B). The MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix C).  

Technical Assistance

MISRGO grantees receive evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.  

New and long-term grantees participated in an evaluation workshop on September 11, 2015, highlighting:  1) tobacco prevention resources; 2) establishing financial sustainability; 3) e-cigarettes; and 4) evaluation dashboards.  (See Fetterman, 2015 for details concerning approaches to establishing  financial stability and sustainability.)



Tobacco Prevention Resources

The evaluation resources shared with the grantees included: Campaign for Tobacco-Free Kids; County Health Ranking ; County Health Calculator; Arkansas Department of Health; CDC State Tobacco Activities, Tracking, and Evaluation System; as well as the MISRGO Evaluation blog.


In addition, a chapter about 10 years of MISRGO tobacco prevention evaluation was published in October 2014.  The title is “Empowerment Evaluation and Evaluation Capacity building in a 10-Year Tobacco Prevention Initiative.” The chapter was highlighted during the workmshop in part to applaud the works of the group.  It was published in: Fetterman, D.M., Kaftarian, S. and Wandersman, A. (2014) Empowerment Evaluation:  Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability.  Thousand Oaks, CA: Sage.

  
Evaluation Dashboards

Grantees participated in another exercise to ensure the appropriate use of their evaluation dashboards, specifically to monitor their own progress throughout the year. The exercise highlighted the value of establishing goals, benchmarks or milestones, and baselines.  Once these were established, only the actual performance measures will be needed throughout the year – comparing actual performance with benchmarks or milestones and goals to determine if adequate progress has been made.


MISRGO and evaluation team members also use it to enhance accountability, signaling precisely when assistance is required to increase the probability of grantees accomplishing their end-of-year goals. 

The evaluation dashboards are designed to build capacity and produce outcomes.  They are in accordance with an empowerment evaluation approach.  (Fetterman, 2013; Fetterman and Wandersman, 2005; Fetterman, Kaftarian, and Wandersman, 2015.)

e-cigarette

The e-cigarette was discussed as well, highlighting the tobacco industry’s financial backing for the device.  In addition, the FDA and World Health Organization have called for stricter regulation of e-cigarettes.  Additional observations were made abut the marketing of e-cigarettes during the workshop, including that it is no accident that the e-cigarette looks like a cigarette and that this is part of the tobacco industry’s efforts to “normalize” the use of cigarettes in the public again. Recent statistics also highlight the dramatic increase in the use of e-cigarettes by youth.  The CDC has found that:  E-cigarette use tripled among middle and high school students in just one year” (CDC, 2015)


Dialogue

In addition, grantees openly discussed obstacles and generously shared advice and best practices, based on their own experience.



Grantees are provided with additional tobacco prevention tools and updates to facilitate their programmatic and evaluative efforts.  (See Appendix D for an example of a periodic technical assistance communication.) 

Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com

Conclusion

MISRGO grantees have established their goals, benchmarks or milestones, and baselines.  In addition, they have documented their actual performance.  Between 21% and 43% of the grantees have met[iii] or exceeded their 1st Quarter evaluation benchmarks/milestones in each of the CDC intervention areas.  The evaluation dashboards help grantees monitor their own performance.   They are being used to confirm effective performance and alert MISRGO and grantees concerning the need for corrections throughout the year.

The evaluation findings reported in this 1st Quarter evaluation report provide evidence of MISRGO grantee dedication and commitment, concerning tobacco prevention.  They also document progress toward annual programmatic and evaluation goals.





References

Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
ARKANSAS STRATEGIC PLAN TO PREVENT AND REDUCE TOBACCO USE 2009 – 2014.  Little Rock, Arkansas:  Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf

Centers for Disease Control and Prevention (2015).  E-cigarette use triples among middle and high school students in just one year


Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2014). Tobacco Use.  Targeting the Nation’s Leading Killer
- At A Glance 2011.  Atlanta, Georgia:  Centers for Disease Control and Prevention.

Centers for Disease Control and Prevention.  Prevention Status Report, 2013.  http://www.cdc.gov/psr/tobacco/2013/ar-tobacco.pdf

Campaign for Tobacco Free Kids (2013). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

Fetterman, D.M. (2013).  Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice.  Stanford:  Stanford University Press.

Fetterman, D.M. (2015).  MISRGO:  Sustainability Report (Empowerment Evaluation Workshop).  San Jose: Fetterman & Associates.

Fetterman, D.M. and Delaney, L. (2015).  MISRGO Empowerment Evaluation Annual Report 2014-2015.  San Jose:  Fetterman & Associates.

Fetterman, D.M., Delaney, L., Triana-Tremain, B., and Evans-Lee, M. (2014). Empowerment Evaluation and Capacity Building in a 10-Year Tobacco Prevention Initiative.  In Fetterman, D.M., Kaftarian, S., and Wandersman, A. (eds). Empowerment Evaluation:  Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability.  Thousand Oaks, CA: Sage.

Fetterman, D.M., Delaney, L., and Tremain, B. MISRGO Evaluation Annual Report 2013-2014.  San Jose:  Fetterman & Associates (July 11, 2014).

Fetterman,  D., Delaney, L., Tremain, B. (2013a).  MISRGO Evaluation Annual Report 2012-2013.  San Jose:  Fetterman & Associates (July 18, 2013).

Fetterman, D.M., Delaney, L., and Tremain, B. (2013b).  MISRGO Evaluation 2nd Quarter Progress Report 2013.  San Jose:  Fetterman & Associates.

Fetterman, D.M., Delaney, L., and Tremain, B. (2013c).  MISRGO Evaluation 3rd  Quarter Progress Report 2013.  San Jose:  Fetterman & Associates.

Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2015).  Empowerment Evaluation:  Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability.  Thousand Oaks, CA: Sage.

Fetterman, D.M., Tremain, B., and Delaney, L. (2012).  MISRGO Evaluation Annual Report 2011-2-12.  July 12, 2012. San Jose: Fetterman & Associates.

Fetterman, D.M., Tremain, B., and Delaney, L. (2011).  MISRGO Empowerment Evaluation:  June 2011 Annual Report.  San Jose: Fetterman & Associates.

Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice.  New York: Guilford Publication.





[i]  See CDC site:  http://www.cdc.gov/psr/tobacco/2013/ar-tobacco.pdf
[ii] Grantees either met or closely approximated their benchmarks.
[iii] Grantees either met or closely approximated their benchmarks.