Tuesday, October 26, 2010

Online Survey Training.  Dr. Beverly Tremain provided an online survey training webinar for the grantees on October 26, 2010.

She provided an overview of the free version of SurveyMonkey - a popular and user-friendly online survey program.

We started with creating a survey, including yes/no, multiple choice, and open-ended questions.
Then she explained how to use the Question Logic option or skipping option - to jump from one question to another string of questions (depending on your response to the initial question).

Bev continued the demonstration with a focus on data collection, primarily by sending the survey URL to everyone in the community or placing the URL on Facebook.  She concluded the demonstration with suggestions for sharing the findings, including saving the bar charts as an html or pdf file. One member of the webinar suggested taking a computer screen snapshot and emailing that out to everyone.  (It can be edited with PhotoShop if needed.)

Bev highly recommended piloting the survey before launching.  She also reminded everyone that this tool is designed to create surveys and help with data collection and display.  It still requires a separate and concerted effort to construct quality survey questions. 

An added bonus of the demonstration is that it was conducted using free DimDim software, which facilitates group online demonstrations and webinars.  It allows the presenter to share their desktop with all of the webinar participants. This allows the presenter to demonstrate the software, in this case SurveyMonkey, on her screen and simultaneously project it to the computer screens of every webinar participant.

Bev also mentioned another resource during the demonstration, which is called  FreeConferenceCall.com.  This enables you to conduct a conference call for fee for up to 20 people at a time. 

Dr. Tremain plans on providing monthly training sessions to promote capacity building this year.  Please contact her for upcoming training events for MISRGO grantees.  Her email is:  btremain@publichealthconsulting.net

Thursday, October 21, 2010

“Using Evaluation to Inform Policy” - Dr. David M. Fetterman

Dr. Fetterman, Fetterman & Associates, was our MISRGO University of Arkansas at Pine Bluff guest speaker on October 14, 2010.  The presentation was made at the American Cancer Society, 901 N. University Avenue, Little, Rock, Arkansas.

Dr. Fetterman presented 5 case examples to illustrate how ethnographic and empowerment evaluations have been used to influence policy and policy decision-making.  The case examples included:
  1. Arkansas Department of Education:  Schools in Academic Distress
  2. Hewlett-Packard $15 Million Digital Village Project
  3. MISRGO:  Tobacco Prevention
  4. Stanford University School of Medicine
  5. U.S. Department of Education:  Dropout and Gifted Programs
The examples demonstrated how to use evaluation to inform policy on the local, state, and national levels.  

Small groups were organized to: 

1) discuss what they had heard during the presentation
2) share their own experiences linking evaluation to policy; and
3) apply what they had learned to their own experience and generate new ideas for future work.  
The presentation concluded with a demonstration of useful technological tools to facilitate evaluations, specifically designed to inform policy decision-making.
Dr. Fetterman’s slide presentation is available here.

October 13, 2010 MISRGO Empowerment Evaluation Workshop

Empowerment Evaluation Workshop. This MISRGO empowerment evaluation workshop focused on:  documenting actual performance in comparison to the baseline, goals, and benchmarks.  We critiqued each other’s five year plans, using this model.  Then we applied the same approach to quarterly reporting for this year.   By comparing actual performance with the benchmarks we were able to determine if we were meeting our goals or if we needed to fortify our efforts.  This approach helps us monitor change over time. (Slides from the presentation are available here.)

Quarterly Measurements of Performance:  
Benchmark and Actual Performance Comparisons

Our evaluation team, Dr. Beverley Tremain, Ms. Linda Delaney, and Dr. Fetterman provided the grantees with individualized assistance as they refined their first quarterly report.   
Bev (above) and Linda (below) facilitating 
the discussion and critique

As we volunteered our work for a group critique, it became clear that this was going to be a learning process for all of us.
 Grantees Actively Listening

Key lessons learned during the workshop included:
  • separating out goals, instead of clustering them in one sentence, in order to track them more effectively
  • ensuring that activities are related to the goal
  • remembering to record actual performance data
  • stating benchmarks for each year and each quarter; (as opposed to only one year and making the reader assume that the same benchmarks apply to each quarter or year)
  • using actual numbers verses percentages; stating the goal in terms of a number
  • completing the calculations for the reader; (instead of stating that the goal is to improve by 20% and making the reader go back to the baseline in a previous report to calculate 20% of the baseline figure)
  • similarly, avoiding ambiguous statements, e.g. will increase, but not specifying by how much or from what baseline; will improve, without stating how they will improve, etc.
  • using the matrix instrument to facilitate record keeping, e.g. baseline, goals, benchmarks, and actual performance
  • creating bar charts that match the matrix data or figures
Our discussions were animated and productive.  We shared strategies to accomplish goals we had in common.  

Light Moments and Serious Moments 
During Our Workshop Exchange and Critique Session

We also made a “good faith” commitment to focus on fewer things that we all had in common in order to have a greater “dose” effect.  Even the breaks were productive, sharing insights into what works and what has not worked with each other.  MISRGO staff were viewed as supportive and helpful.
Bev, Ruthie, Linda, and Marian 
at the Table During a Break

We invited the Arkansas Department of Health to share their knowledge and resources with us.  One of their valuable tobacco related data and report web pages is available here.
Sherry Johnson and 
ADH Colleagues

They recommended Center for Disease Control reports including:
They also shared their Tobacco Data Deck 2010, providing useful State-wide data to compare with local grantee efforts.  The Data Deck is available on the ADH web site and will be available on the GEMS web-site. (GEMS is the online place for grantees to enter their data.)

We concluded the workshop with our own local “best practices” exchange.  We mixed evidenced-based practices with practice-based evidence.
Dr. Fetterman 
Concluding the Workshop