Wednesday, September 23, 2015

Empowerment Evaluation Tobacco Prevention Workshop

MISRGO 1st Quarter Empowerment Evaluation Workshop
(2015-2016)

This is the beginning of the annual cycle for the MISRGO Tobacco Prevention grantees.  We assembled on September 11th, 2015 at the University of Arkansas, Pine Bluff.  The beginning of the workshop providing basic training and technical assistance for the grantees. It also provided the team with an opportunity to chart their own course for the year.


It included a review of the end-of-the-year evaluation findings, in which between 73% and 100% of the grantees met or exceeded their annual goals.


We also discussed how we were building both programmatic and evaluative capacity.


Then we launched into resources ranging from the Campaign for Tobacco-Free Kids to the Centers for Disease Control and Prevention.




We provided up dates on the E-cigarette phenomenon.
"E-cigarette use tripled among middle and high school students in just one year."




Evaluation Dashboard

The heart of the training focused on the evaluation dashboard to monitor quarterly and annual program performance.

It consisted of the following data sources:
  • Actual Performance - how many things were accomplished each quarter and annually (as the data comes in)
  • Milestones - what is expected each quarter and annually (concerning outcomes or results)
  • Goals - the number of things that should happen at the end of the year
  • Baseline - where things were before the beginning of the year.

The grantees agreed on the baseline, goals, and milestones.  From this point on all that is required of the grantees is recording the actual performance each quarter.   The dashboard allows grantees to monitor their own progress and performance.  They can compare their actual performance with their quarterly milestones and annual goals.  It also allows the MISRGO office and evaluation team to determine if additional intervention and assistance are needed.  A sample dashboard is provided below:


Empowerment Evaluation

The team carved out time in the schedule to conduct an abbreviated empowerment evaluation exercise, focusing on funding and sustainability.  It resulted in concrete steps to retain existing sources of funding and an action plan to generate new funding streams for tobacco prevention work. 

The Sustainability Report and action plan generated from that exercise is available online.

Technological Tools in Evaluation

The workshop concluded with a variety of technological tools used in evaluation to facilitate communication and reporting, ranging from online surveys to word clouds and infographics.  

Click here for a copy of the workshop slides .




Sunday, September 13, 2015

Empowerment Evaluation Sustainability Report



MISRGO Sustainability Report
Empowerment Evaluation Workshop

Prepared for:  MISRGO & MISRGO Grantees     
Prepared by:   Dr. David Fetterman        
Date:                September 11, 2015
                       




Fetterman & Associates - www.davidfetterman.com - 650-269-5689

Introduction

This report summarizes MISRGO grantee plans to enhance their program stability and sustainability.  Their efforts are the result of their participation in an empowerment evaluation workshop held at the STEM Center at UAPB on September 11, 2015.  The report is organized into 3 sections associated with the steps of empowerment evaluation, including:  1) re-establishing mission; 2) taking stock of or rating the current status of their work; and 3) planning for the future.

Mission

MISRGO grantees re-established their mission as a group.  It is highlighted in bullet format below:

Mission:  MISRGO Tobacco Prevention Grantees

  •      Create a tobacco free world.
  •      Create an end game or plan to eliminate tobacco consumption.
  •      Improve the health of Arkansas residents
  •      Create an awareness of the dangers of tobacco use
  •      Promote positive healthy role models for next generation, concerning tobacco use
  •      Eliminate exposure to second hand smoke
  •      Promote cessation among youth and young adults
  •      Address disparities
  •      Prevent initiation
  •      Decrease health costs


Taking Stock

The grantees took stock of their performance after generating their bullet pointed mission statement.  Their taking stock self-assessment relied on a two-step approach:  prioritization and ratings.

Prioritization.  First grantees generated a list of critical activities that are required to accomplish their individual program and larger group mission.  They prioritized this list to prepare for the group’s assessment of their work.


  
The list of activities, from most important to least important, were as follows: securing new funding, keeping existing funding, policy, enforce policies, education, engaging youth, building coalitions, networking, media, evaluation, planning and development, communication, program sustainability, and needs assessment.


Ratings.  They used this list of prioritized activities to assess their own performance. They rated each activity on a 1 (low) to 10 (high) scale.   The highest ratings were given to:  networking, policy, building coalitions, and evaluation.  The lowest ratings were given to:  keeping existing funding and securing new funding.




MISRGO grantees engaged in a dialogue about their ratings.  They were required to provide evidence to support their ratings. The discussion helped to reveal the “elephant in the room” – serious concerns about future funding and sustainability.


Planning for the Future

The mission and taking stock steps were used to prepare grantees for their next step:  planning for the future.  Planning for the future consists of establishing goals, generating strategies to accomplish goals, and specifying credible evidence to confirm strategies are implemented and effective.

The evidence used to support their ratings during the taking stock step, were used to facilitate the plan for the future exercise.  For example, the evidence used to support a low rating concerning new funding was the absence of new proposals. The grantees used this information to develop a strategy to solicit new funding during the session.  This included researching funding options, drafting proposals, and submitting proposals.

The most significant goals for their plans for the future exercise included:  securing new funding and keeping existing funding.

The goals, strategies and evidence associated with these significant tasks are listed below.  They are followed by evaluation dashboards.  They are used to monitor or track progress toward specified goals.

_/_/_/_/_/ _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

A) Goal 1:  Get New Dollars $ (proposal focus)

B) Strategies:

  1. Partner with other agencies
  2. Diversify funding streams
  3. Fund raising efforts
  4. Call colleagues
  5. Review Foundation Directories (funding sources)
  6. Write proposals
  7. Solicit in-kind donations
C) Evidence

  1. Award notification letter
  2. Check in the mail (and in account)
D) Evaluation Dashboard to Monitor Progress (proposal tracking)

Writing Proposals to Secure New Funding

Comparisons
1st Qtr
2nd Qtr
3rd Qtr
4th Qtr





Actual
0 but conducting research on sources of funding
0 proposals
written
2 proposals written and submitted
4 written and submitted proposals
Benchmarks
0
0
2
4
Goals
4
4
4
4
Baseline
0
0
0
0

In this case, the goal is to submit 4 proposals by the end of the year. 

The first quarter is devoted to researching sources of funding.  The second quarter is devoted to writing proposals (and potentially securing a grant writer to assist in the effort).  The third quarter benchmark is to submit 2 proposals.  The fourth quarter benchmark is to submit 2 more proposals, reaching the annual total of 4 proposals written and submitted.


 _/_/_/_/_/ _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/



A) Goal 2:  Get New Dollars $ (partnering with similar agencies focus)

B) Strategy: 

Partnering with Other Agencies to Support Tobacco Prevention Program Activities

C) Evidence:

  1. Completed MOUs
  2. Secure support for 75% of program activities
D) Evaluation Dashboard to Monitor Progress Toward Goals



This evaluation dashboard outlines partnering strategies to secure funding to support program activities.

The goal is to secure support for 75% of grantee program activities.  Sub-goals or steps that help grantees accomplish there goals include:

  1. Calling and meeting with 5 agencies
  2. Drafting 4 MOUs

The evaluation dashboard provides a tool to track progress toward these goals (and subgoals).  It includes the following benchmarks:

Quarter 1:  

  1. Call and meet 2 similar agencies

Quarter 2: 

  1. Call and meet 3 similar agencies
  2. Draft 2 MOUs

Quarter 3: 

  1. Call and meet 4 similar agencies
  2. Draft 3 MOUs
  3. Secure support for 60% of program activities

Quarter 4:

  1. Call and meet 5 similar agencies
  2. Draft 4 MOUs
  3. Secure support for 75% of program activities

Once these goals and benchmarks are entered into the evaluation dashboard, the only additional data needed is the actual performance data (or what grantees actually accomplished).  In this case example, grantees may have called 2 agencies and met with 1 agency (out of 5) by the end of the first quarter.  During the second quarter grantees called and met with 3 out of 3 similar agencies.  In addition, they drafted 1 out of 2 memorandum of agreement (MOUs).

During the third quarter, grantees called and met with 4 out of 4 similar agencies, drafted 2 out of 3 MOUs, and secured support for 50% of their program activities (from a benchmark of 60%).  Finally, grantees ended the year with the following accomplishments:  1) called and met with 5 out of 5 agencies; 2) drafted 4 MOUs out of 4; and 3) secured support for 60% of program activities (from a goal of 75%).

This evaluation dashboard allows grantees to track their performance throughout the year.  This enables grantees to determine if they are on course, need assistance, and/or make mid-course corrections before the end of the year.  It does not guarantee but enhances the probability of accomplishing grantee goals and objectives.


_/_/_/_/_/ _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

A) Goal 3:  Keeping Existing Funding (getting paid for agreed upon work)

B) Strategies:

Contact representatives in each county served by MISRGO program
Highlight accomplishments (include evaluation data)
Thank them for past support
Invite representatives to next event (including news coverage and photo opportunity)
Express appreciation for continued support in the future
Contact and re-enlist support from past-legislative supportersContact friends, alumni, and the Black Caucus (past MISRGO grantee recipients)
Continue to showcase MISRGO grantee work
C) Evidence of Accomplishment:

  1. Receive sign off letters
  2. Get paid
D) Evaluation Dashboard to Monitor Progress Toward Goals (contacting representatives)

Evaluation Dashboard:  Contact 40 Representatives

Comparisons
1st Qtr
2nd Qtr
3rd Qtr
4th Qtr





Actual
20
30
35
40
Benchmarks
10
20
30
40
Goals
40
40
40
40
Baseline
0
0
0
0


Ideally grantees contact 40 representatives in the first quarter, instead of at the end of the 4th quarter.