Friday, May 9, 2014

3rd Quarter Tobacco Prevention Progress Report 2013-2014

3rd Quarter Comparison with Annual Goals 

MISRGO Evaluation
3rd Quarter Report 2013-2014
Prepared for:            MISRGO, Grantees & Legislative Offices
Prepared by:             Dr. Fetterman, Ms. Delaney, and Dr. Tremain
Date:                            May 1, 2014
                       





Introduction

Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, 2011).  The annual health care costs in Arkansas directly caused by smoking is $812 million, according to the Arkansas Department of Health (2009) and the Campaign for Tobacco Free Kids (2013).  Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.

 The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received Settlement funds through the Arkansas Department of Health to focus on tobacco use in minority communities.  MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.

MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs.  MISRGO currently sponsors and provides technical assistance for 19 grantees across the State of Arkansas.

MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work.  (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
 
MISRGO grantees also have a strong track record of success based on past performance.  For example, last year, the majority of MISRGO grantees met or exceeded their annual performance goals (see MISRGO Evaluation Annual Report 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012 and MISRGO Evaluation Annual Report 2012-2013 Fetterman, Delaney, Tremain, July 18, 2013a).  See also Fetterman, Delaney, and Tremain 2013b and 2013c for interim reports on progress.

Purpose

The purpose of this report is to present MISRGO tobacco prevention grantee progress at the end of the 3rd Quarter 2013-2014.  The fiscal and programmatic end of the year is June 30, 2014.  (See Appendix A for a glossary of grantee names.)

MISRGO Grantee Intervention Areas

MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas.  They include:

Area 1:  Eliminate Exposure to Secondhand Smoke
Area 2:  Preventing Initiation Among Youth and Young Adults
Area 3:  Promoting Quitting Among Youth and Adults
Area 4:  Addressing Disparities

Grantee progress and activities are reported on both a quarterly and annual basis, according to these CDC intervention areas. 
 
Quarterly Progress

During the first quarter, MISRGO grantees established their goals, benchmarks, and baselines.   Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks.  Benchmarks are intermediate objectives or approximations of outcomes.  Baselines represent, for example, the number of smoke-free environments that existed before the grantee activity or intervention. 

During the second quarter, grantees reported on their mid-year progress toward annual goals.  This provided grantees with an opportunity to reflect on their performance.  Based on these data, grantees continued as planned, make mid-course changes, and/or requested assistance as needed.

The third quarter focuses on any gaps between benchmarks and annual goals.  Grantees share techniques with each other, MISRGO staff provide additional recommendations, and Fetterman & Associates notify grantees about gaps and make specific recommendations to close the gap. 

The fourth quarter is the annual comparison of their goals with their actual performance throughout the year.

3rd Quarter Findings

The findings are promising but mixed.  Grantees have met or exceeded their annual goals in the 3rd Quarter, concerning preventing initiation among youth and young adults.  However, they have not met annual goals concerning the CDC intervention areas:  eliminating exposure to secondhand smoke and promoting quitting among youth and young adults, and addressing disparities:

Area 1:  Eliminate Exposure to Secondhand Smoke  (38% met or exceeded annual goals in 3rd Qtr )
Area 2:  Preventing Initiation Among Youth and Young Adults (57% met or exceeded annual goals in 3rd Qtr)
Area 3:  Promoting Quitting Among Youth and Adults (26% met or exceeded annual goals in 3rd Qtr)
Area 4:  Addressing Disparities (37% met or exceeded annual goals in 3rd Qtr)

Intervention Area
% Met-Exceeded  Annual Goals

% Not Met Annual Goals



1.  Eliminating
38%
62%
2.  Preventing
57%
43%
3.  Promoting
26%
74%
4.  Addressing
37%
63%



CDC Area 1:  Eliminate Exposure to Secondhand Smoke

Approximately, 38% of the grantees have met or exceeded their annual goals in the 3rd Quarter concerning Area 1:  Eliminate Exposure to Secondhand Smoke.  This suggests additional attention is needed in this area to met annual goals.

Grantee activities focused on:

·      Encouraging voluntary smoke-free perimeter policies
·      Educating parents and law enforcement officers about the benefits of smoke free environments in the car (Act 811)
·      Promoting Act 811
·      Gathering petition signatures for smoke free environments
·      Increase public support for smoke-free air policies
·      Promoting extracurricular activities designed to increase knowledge about smoking and how it affects nonsmoking individuals
·      Encouraging the adoption of tobacco-free policies
·      Planning to implement tobacco prevention training, such as Project Alert or Banner’s in the Schools
·      Planning to implement a plan to produce smoke free family cars
·      Establishing smoke-free parks
·      Increasing the number of community organizations that support smoking bans/policies
·      Increasing youth awareness about the dangers of tobacco products (surveys and outreach)
  
CDC Area 2:  Preventing Initiation Among Youth and Young Adults

57% of the grantees met or exceeded their annual goals concerning Area 2: Preventing Initiation Among Youth and Young Adults.  Grantees are on track concerning projected annual goals.

Grantee activities focused on:

·      Conducting store-front surveys
·      Providing training in prevention strategies through media outreach, gorilla marketing campaigns, and youth rallies
·      Encouraging youth and young adults to pledge not to initiate tobacco use
·      Educating African American and Latino youth on the effects of smoking
·      Conducting compliance checks on retailer sales to minors
·      Increasing the use of the Quitline
·      Increasing the number of youth oriented tobacco prevention events
·      Educating youth and young adults about the dangers of tobacco

CDC Area 3:  Promoting Quitting Among Youth and Adults

The majority of grantees (26%) have met or exceeded their annual goals concerning Area 3:  Promoting Quitting.   This suggests additional attention is needed in this area to met annual goals.

Grantee activities focused on:

·      Increasing use of the Arkansas Quitline
·      Encouraging African Americans and Latinos to participate in cessation programs
·      Encouraging African-American males to use the Arkansas Quitline
·      Helping to start a tobacco youth program where students learn about smoking
·      Helping to increase community partners
·      Increasing Hispanic calls to SOS Quitline
·      Decreasing chewing tobacco use
·      Increasing number participating in 40 Days to Freedom programs

CDC Area 4:  Addressing Disparities

37% of the grantees have met or exceeded their annual goals concerning Area 4:  Addressing Disparities.  This suggests additional attention is needed in this area to met annual goals.

Grantee activities focused on:

·      Sharing information about current tobacco laws
·      Improving access to existing cessation resources for Hispanic males
·      Developing smoke free policies or point of purchase policy to reduce advertising
·      Creating diverse partnerships to address tobacco-related disparities
·      Increasing African-American and Latino post-test scores in awareness
·      Educating African American and Latino residents concerning tobacco related health disparities
·      Increasing the number of minorities pledging to keep themselves or their homes and vehicles tobacco free
·      Referring people to the SOS Quitline and/or other local cessation resources
·      Planning to organize Youth Extinguishing Smoking teams (Y.E.S. Teams)
·      Increasing the number of churches implementing the 40 Days to Freedom program
·      Increasing minorities’ awareness of tobacco related disparities

An evaluation dashboard for each grantee’s annual performance is provided to facilitate communication and collaboration among grantees, enhance MISRGO accountability, and build knowledge.  It is organized by CDC intervention area (see Appendix B). The MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix C).  

Technical Assistance

MISRGO grantees receive programmatic and evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.  

Primarily new grantees participated in an evaluation workshop on October 3, 2013.  At the workshop new grantees were re-introduced to the 3 steps of empowerment evaluation.  They openly discussed obstacles and generously shared advice and best practices, based on their own experience.





New grantees were also reacquainted with the evaluation dashboard, including recording baseline, goals, benchmarks, and actual performance.   The evaluation dashboard has been used to help grantees monitor their own progress throughout the year. MISRGO and evaluation team members have also used it to enhance accountability, signaling precisely when assistance is required to increase the probability of grantees accomplishing their end-of-year goals. 




The evaluation dashboards are designed to build capacity and produce outcomes.  They are in accordance with an empowerment evaluation approach.  (Fetterman, 2013; Fetterman and Wandersman, 2005; Fetterman, Kaftarian, and Wandersman, 2014.)

A second evaluation technical assistance workshop was conducted on April 17, 2014.  The workshop introduced a number of new technological tools including Google Glass.  Google Glass is a pair of glasses that allow you to take a picture by voice command and with a swipe of the finger emailed it and posted it to a site on the web.  The group immediately recognized the potential for evaluation use.  They could see how it could be used to document site visits, share preliminary findings, and even stream video for spontaneous training opportunities.  (For more information about Google Glass in evaluation see:  http://aea365.org/blog/david-fetterman-on-google-glass-part-ii-using-glass-as-an-evaluation-tool/)

The focus of the workshop, however, was on closing the gap between our 3rd quarter performance and our end of the year (June) goals.

We discussed our individual 3rd quarter performance, noting where some of us had already exceeded our annual goals, including establishing smoke free policies and creating smoke free parks.  We also examined evaluation dashboard case examples documenting where we were not meeting our benchmarks.

We also used art as a way to talk about why we were not meeting our benchmarks, the obstacles we faced, and generally what was not working and impeding our work.  We broke into small groups and drew pictures of our problems. 

One work of art depicted a “tale of two cities.” It highlighted one side of the street with ample resources and the other side with few resources and opportunities.  The side of the street with inadequate resources is where one of the groups worked.  The picture captured complex social conditions.  It explained why they had to focus on building capacity while working to accomplish specific outcomes.  This safe way of discussing the social context helped them launch into ad discussion about solutions that matched the community needs and capacity, before launching monumental initiatives.





Other artwork depicted more practical problems such as the need for translation (Spanish and Marshallese), convincing pastors to establish smoke free churches and parking lots (especially if they are smokers), problems implementing the 40 days program – a faith based approach to smoke cessation, problems tracking fax referral forms for the Quitline, convincing store managers to comply with signage and perimeter laws, and not having enough time in the day to accomplish all of our objectives.

We also shared conventional high-powered tools and resources, including:


Center for Disease Control and Prevention monographs (such as Key Outcome Indicators, Developing an Effective Evaluation Plan,  and Introduction to Process Evaluation)
Center for Disease Control and Prevention sites (such as the States Tobacco Activities Tracking and Evaluation System)





Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com

Conclusion

MISRGO grantees established their goals, benchmarks, and baselines during the first quarter of the year.  In addition, they continue to document their actual performance.  The evaluation dashboards help grantees monitor their own performance.   They are being used to confirm effective performance and alert MISRGO and grantees concerning the need for corrections throughout the year. At present grantees are meeting or exceeding only one of their annual goals concerning the four CDC intervention areas.  This signals the need to provide assistance to help grantees meet their annual goals.

MISRGO grantees are dedicated, committed, and hard working concerning tobacco prevention.  They also document progress toward annual programmatic and evaluation goals.  However, this is a time to close the gap between actual performance and end-of-the year annual goals.


References

Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
Arkansas Strategic Plan to Prevent and Reduce Tobacco Use 2009 -2014.  Little Rock, Arkansas:  Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf

Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2011). Tobacco Use.  Targeting the Nation’s Leading Killer
- At A Glance 2011.  Atlanta, Georgia:  Centers for Disease Control and Prevention.

Campaign for Tobacco Free Kids (2013). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas

Fetterman, D.M. (2013).  Empowerment Evaluation in the Digital Villages:  Hewlett-Packard’s $15 Million Race Toward Social Justice.  Stanford:  Stanford University Press.









Fetterman,  D., Delaney, L., Tremain, B. (2013a).  MISRGO Evaluation Annual Report 2012-2013.  San Jose:  Fetterman & Associates (July 18, 2013).

Fetterman, D.M., Delaney, L., and Tremain, B. (2013b).  MISRGO Evaluation 2nd Quarter Progress Report 2013.  San Jose:  Fetterman & Associates.

Fetterman, D.M., Delaney, L., and Tremain, B. (2013c).  MISRGO Evaluation 3rd  Quarter Progress Report 2013.  San Jose:  Fetterman & Associates

Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2014).  Empowerment Evaluation:  Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability.  Thousand Oaks, CA: Sage.

Fetterman, D.M., Tremain, B., and Delaney, L. (2012).  MISRGO Evaluation Annual Report 2011-2-12.  July 12, 2012. San Jose: Fetterman & Associates.









Fetterman, D.M., Tremain, B., and Delaney, L. (2011).  MISRGO Empowerment Evaluation:  June 2011 Annual Report.  San Jose: Fetterman & Associates.


Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice.  New York: Guilford Publication.

Detailed Findings Available Upon Request