Prepared for: MISRGO, Grantees & Legislative Offices
Prepared by: Dr. Fetterman, Ms. Delaney, and Dr. Tremain
Date: July 11, 2014
“Tobacco use is the single most preventable cause of disease, disability, and death in the United States” (Center for Disease Control and Prevention, 2011). The annual health care costs in Arkansas directly caused by smoking is $812 million, according to the Arkansas Department of Health (2009) and the Campaign for Tobacco Free Kids (2013). Arkansas’ Master Tobacco Settlement revenue is being used in part to address this serious health issue.
The University of Arkansas at Pine Bluff’s Minority Initiative Sub-Recipient Grant Office (MISRGO) has received Settlement funds through the Arkansas Department of Health to focus on tobacco use in minority communities. MISRGO’s mission is specifically to prevent and reduce tobacco use in minority communities.
MISRGO has awarded more than 50 Arkansas organizations with funding for tobacco prevention and cessation programs. MISRGO currently sponsors and provides technical assistance for 19 grantees across the State of Arkansas.
MISRGO grantees have broad-based support across the State for their tobacco prevention and cessation work. (See MISRGO Empowerment Evaluation: June 2011 Annual Report, Fetterman, Tremain, and Delaney, 2011.)
MISRGO grantees also have a strong track record of success based on past performance. For example, the majority of MISRGO grantees have met or exceeded their annual performance goals (see MISRGO Evaluation Annual Report 2011-12 Fetterman, Tremain, and Delaney, July 12, 2012 and MISRGO Evaluation Annual Report 2012-2013 Fetterman, Delaney, Tremain, July 18, 2013a). See also Fetterman, Delaney, and Tremain 2013b and 2013c for interim progress reports.
The purpose of this report is to present MISRGO tobacco prevention grantee outcomes at the end of year 2013-2014. The fiscal and programmatic end of the year is June 30, 2014. (See Appendix A for a glossary of grantee names.)
MISRGO Grantee Intervention Areas
MISRGO grantees use the Centers for Disease Control and Prevention’s (CDC) approved and recommended intervention areas. They include:
Area 1: Eliminate Exposure to Secondhand Smoke
Area 2: Preventing Initiation Among Youth and Young Adults
Area 3: Promoting Quitting Among Youth and Adults
Area 4: Addressing Disparities
Grantee progress and activities are reported on both a quarterly and annual basis, according to these CDC intervention areas.
During the first quarter, MISRGO grantees established their goals, benchmarks, and baselines. Goals are the planned outcomes or results at the end of the year, e.g. number of smoke-free parks. Benchmarks are intermediate objectives or approximations of outcomes. Baselines represent, for example, the number of smoke-free environments that existed before the grantee activity or intervention.
During the second quarter, grantees reported on their mid-year progress toward annual goals. This provided grantees with an opportunity to reflect on their performance. Based on these data, grantees continued as planned, make mid-course changes, and/or requested assistance as needed.
The third quarter focused on any gaps between benchmarks and annual goals. Grantees shared techniques with each other, MISRGO staff provided additional recommendations, and Fetterman & Associates notified grantees about gaps and made specific recommendations to close the gap.
The fourth quarter is the annual comparison of their goals with their actual performance throughout the year.
Grantees have met or exceeded the majority of their annual goals in each intervention area.
Area 1: Eliminate Exposure to Secondhand Smoke (80% met or exceeded annual goals)
Area 2: Preventing Initiation Among Youth and Young Adults (74% met or exceeded annual goals)
Area 3: Promoting Quitting Among Youth and Adults (80% met or exceeded annual goals)
Area 4: Addressing Disparities (65% met or exceeded annual goals)
CDC Area 1: Eliminate Exposure to Secondhand Smoke
Approximately, 80% of the grantees have met or exceeded their annual goals concerning Area 1: Eliminate Exposure to Secondhand Smoke.
Grantee activities focused on:
· Encouraging voluntary smoke-free perimeter policies
· Educating parents and law enforcement officers about the benefits of smoke free environments in the car (Act 811)
· Promoting Act 811
· Gathering petition signatures for smoke free environments
· Increase public support for smoke-free air policies
· Promoting extracurricular activities designed to increase knowledge about smoking and how it affects nonsmoking individuals
· Encouraging the adoption of tobacco-free policies
· Planning to implement tobacco prevention training, such as Project Alert or Banner’s in the Schools
· Planning to implement a plan to produce smoke free family cars
· Establishing smoke-free parks
· Increasing the number of community organizations that support smoking bans/policies
· Increasing youth awareness about the dangers of tobacco products (surveys and outreach)
CDC Area 2: Preventing Initiation Among Youth and Young Adults
74% of the grantees met or exceeded their annual goals concerning Area 2: Preventing Initiation Among Youth and Young Adults.
Grantee activities focused on:
· Conducting store-front surveys
· Providing training in prevention strategies through media outreach, gorilla marketing campaigns, and youth rallies
· Encouraging youth and young adults to pledge not to initiate tobacco use
· Educating African American and Latino youth on the effects of smoking
· Conducting compliance checks on retailer sales to minors
· Increasing the use of the Quitline
· Increasing the number of youth oriented tobacco prevention events
· Educating youth and young adults about the dangers of tobacco
CDC Area 3: Promoting Quitting Among Youth and Adults
The majority of grantees (80%) have met or exceeded their annual goals concerning Area 3: Promoting Quitting.
Grantee activities focused on:
· Increasing use of the Arkansas Quitline
· Encouraging African Americans and Latinos to participate in cessation programs
· Encouraging African-American males to use the Arkansas Quitline
· Helping to start a tobacco youth program where students learn about smoking
· Helping to increase community partners
· Increasing Hispanic calls to SOS Quitline
· Decreasing chewing tobacco use
· Increasing number participating in 40 Days to Freedom programs
CDC Area 4: Addressing Disparities
65% of the grantees have met or exceeded their annual goals concerning Area 4: Addressing Disparities.
Grantee activities focused on:
· Sharing information about current tobacco laws
· Improving access to existing cessation resources for Hispanic males
· Developing smoke free policies or point of purchase policy to reduce advertising
· Creating diverse partnerships to address tobacco-related disparities
· Increasing African-American and Latino post-test scores in awareness
· Educating African American and Latino residents concerning tobacco related health disparities
· Increasing the number of minorities pledging to keep themselves or their homes and vehicles tobacco free
· Referring people to the SOS Quitline and/or other local cessation resources
· Planning to organize Youth Extinguishing Smoking teams (Y.E.S. Teams)
· Increasing the number of churches implementing the 40 Days to Freedom program
· Increasing minorities’ awareness of tobacco related disparities
An evaluation dashboard for each grantee’s annual performance is provided to facilitate communication and collaboration among grantees, enhance MISRGO accountability, and build knowledge. It is organized by CDC intervention area (see Appendix B). The MISRGO Grantee Activity Log provides an additional insight into the list of grantee activities across sites and intervention areas (see Appendix C).
MISRGO grantees receive programmatic and evaluation technical assistance throughout the year, including assistance monitoring and evaluating their progress.
Primarily new grantees participated in an evaluation workshop on October 3, 2013. At the workshop new grantees were re-introduced to the 3 steps of empowerment evaluation. They openly discussed obstacles and generously shared advice and best practices, based on their own experience.
New grantees were also reacquainted with the evaluation dashboard, including recording baseline, goals, benchmarks, and actual performance. The evaluation dashboard has been used to help grantees monitor their own progress throughout the year. MISRGO and evaluation team members have also used it to enhance accountability, signaling precisely when assistance is required to increase the probability of grantees accomplishing their end-of-year goals.
The evaluation dashboards are designed to build capacity and produce outcomes. They are in accordance with an empowerment evaluation approach. (Fetterman, 2013; Fetterman and Wandersman, 2005; Fetterman, Kaftarian, and Wandersman, 2014.)
A second evaluation technical assistance workshop was conducted on April 17, 2014. The workshop introduced a number of new technological tools, including Google Glass. Google Glass is a pair of glasses that allow you to take a picture by voice command and with a swipe of the finger email it and posted it to a site on the web. The group immediately recognized the potential for evaluation use. They could see how it could be used to document site visits, share preliminary findings, and even stream video for spontaneous training opportunities. (For more information about Google Glass in evaluation see: http://aea365.org/blog/david-fetterman-on-google-glass-part-ii-using-glass-as-an-evaluation-tool/)
The focus of the workshop, however, was on closing the gap between our 3rd quarter performance and MISRGO grantee end of the year (June) goals.
We discussed individual grantee 3rd quarter performance, noting where some grantees had already exceeded their annual goals, including establishing smoke free policies and creating smoke free parks. We also examined evaluation dashboard case examples, documenting where grantees were not meeting their benchmarks.
We also used art as a way to talk about the obstacles grantees faced, why some grantees were not meeting their benchmarks, and generally what was not working and impeding progress. Grantees broke into small groups and drew pictures of their program implementation problems.
One work of art depicted a “tale of two cities.” It highlighted one side of the street with ample resources and the other side with few resources and opportunities. The side of the street with inadequate resources is where one of the groups worked. The picture captured complex social conditions. It explained why they had to focus on building capacity, while working to accomplish specific outcomes. This safe way of discussing the social context helped them launch into a discussion about solutions that matched the community needs and capacity, before launching large-scale initiatives.
Other artwork depicted more practical problems, such as the need for translation (Spanish and Marshallese), convincing pastors to establish smoke free churches and parking lots (especially if they are smokers), problems implementing the 40 Days to Freedom program – a faith based approach to smoke cessation, problems tracking fax referral forms for the Quitline, convincing store managers to comply with signage and perimeter laws, and not having enough time in the day to accomplish all of the grantees’ objectives.
Conventional high-powered tools and resources were also shared, including:
Arkansas Department of Health web site
Center for Disease Control and Prevention monographs (such as Key Outcome Indicators, Developing an Effective Evaluation Plan, and Introduction to Process Evaluation)
Center for Disease Control and Prevention sites (such as the States Tobacco Activities Tracking and Evaluation System)
Grantees are provided with additional tobacco prevention tools and updates to facilitate their programmatic and evaluative efforts. (See Appendix D for an example of a technical assistance communication, Appendix E concerning grantee appreciation of evaluation workshops, and Appendix F concerning grantee evaluation of technical assistance workshop.)
Details concerning evaluation technical assistance are provided on the MISRGO tobacco prevention blog at: http://tobaccoprevention.blogspot.com
MISRGO grantees established their goals, benchmarks, and baselines during the first quarter of the year. In addition, they continued to document their actual performance each quarter. The evaluation dashboards help grantees monitor their own performance. They were used to confirm effective performance and alert MISRGO and grantees concerning the need for corrections throughout the year.
MISRGO grantees are dedicated, committed, and hard working, concerning tobacco prevention. They took the required set of actions to close the gap between actual performance and end-of-the year annual goals. The result is another year of accomplishments in which MISRGO grantees have met or exceeded the majority of their annual goals, concerning the four CDC intervention areas.
Arkansas Department of Health (2009). Tobacco Prevention and Cessation Program
Arkansas Strategic Plan to Prevent and Reduce Tobacco Use 2009 -2014. Little Rock, Arkansas: Arkansas Department of Health, p. 3. http://www.healthy.arkansas.gov/programsServices/tobaccoprevent/Documents/TPCPStrategicPlan.pdf
Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion (2011). Tobacco Use. Targeting the Nation’s Leading Killer - At A Glance 2011. Atlanta, Georgia: Centers for Disease Control and Prevention.
http://www.cdc.gov/chronicdisease/resources/publications/aag/osh.htm. See also: http://www.cdc.gov/chronicdisease/resources/publications/aag/pdf/2011/Tobacco_AAG_2011_508.pdf
Campaign for Tobacco Free Kids (2013). Annual health care costs in Arkansas directly caused by smoking. http://www.tobaccofreekids.org/facts_issues/toll_us/arkansas
Fetterman, D.M. (2013). Empowerment Evaluation in the Digital Villages: Hewlett-Packard’s $15 Million Race Toward Social Justice. Stanford: Stanford University Press.
Fetterman, D., Delaney, L., Tremain, B. (2013a). MISRGO Evaluation Annual Report 2012-2013. San Jose: Fetterman & Associates (July 18, 2013).
Fetterman, D.M., Delaney, L., and Tremain, B. (2013b). MISRGO Evaluation 2nd Quarter Progress Report 2013. San Jose: Fetterman & Associates.
Fetterman, D.M., Delaney, L., and Tremain, B. (2013c). MISRGO Evaluation 3rd Quarter Progress Report 2013. San Jose: Fetterman & Associates
Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2014). Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability. Thousand Oaks, CA: Sage.
Fetterman, D.M., Tremain, B., and Delaney, L. (2012). MISRGO Evaluation Annual Report 2011-2-12. July 12, 2012. San Jose: Fetterman & Associates.
Fetterman, D.M., Tremain, B., and Delaney, L. (2011). MISRGO Empowerment Evaluation: June 2011 Annual Report. San Jose: Fetterman & Associates.
Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice. New York: Guilford Publication.
(Download full report for appendices.)