Baseline, Benchmark, and Goals Workshop

Baseline, Benchmark, and Goals Workshop. Our September 24, 2009 evaluation workshop received rave reviews. A few grantee and evaluator comments included:

"It wet our appetite for more".
"This is precisely what we needed."
"Timely."
"Clear."

MISRGO staff also valued the day: "We've gotten good reviews about the workshop. Thanks so much for all you do!!!"

Dean Johnson

Dean Johnson took time out of his busy schedule to join us and help strategize about next steps in the evaluation.

Dean Calvin Johnson

Three Evaluation Speakers/Facilitators

We had three speakers/facilitators for the day. The day began with Dean Stewart Donaldson from Claremont Graduate University. He facilitated a session entitled:

Practical Program Evaluation: A Program Theory Approach

He highlighted the role of program theory. It is used to make sure everyone agrees with the logic of their program in an evaluation.For example, if you did X (connected them with the Quitline and heard back from them) it would be likely to produce Z (help people stop smoking).
He conducted an exercise where everyone was asked to evaluate the room. It was practical and down-to-earth and emphasized the role of an individual's perspective in evaluation.

Grantee and Evaluator Presenting Insights Based on Dr. Donaldson's Exercise

The second facilitator was Dr. Tarek Azzam, also from Claremont. He presented the tech session. It was titled:

Using Technology to Enhance Applied Research & Evaluation

He highlighted the use of technology to conduct evaluations and to more effectively communicate about evaluation findings.
Tarek covered the use of: GIS, online surveys, and various qualitative data display tools.

Use of GIS to Identify Problem Issues and Areas: Tobacco Billboards & Income

Qualitative Data Display: Frequency of Words Used Visually Represented in a Word Cloud

The third speaker was Dr. David Fetterman, from Fetterman & Associates. He built on the first two speakers ideas and techniques. Dr. Donaldson focused on program theory - does the program make sense as designed. Dr. Tarek highlighted the use of technology to display evaluation findings.

Dr. Fetterman highlighted a visually powerful way of highlighting grantee program baselines, benchmarks, and goals. This will be the focus of next year's reporting - to help us all keep track of our progress throughout the year and for the next 5 years. The title of his session was:

Baseline, Benchmarks, and the Visual Alert System

The MCCTFA Baseline and Quarterly Evaluation Assessment Form was used as a case example. Dr. Fetterman explained how to pull the baseline figure from the narrative. Then he demonstrated how to identify the goal, for example, to increase the number of public buildings with smoke free policies by 15%. From there it was simply a matter of pulling the current year objective, 5%, and establishing benchmarks for the next 5 years. The slides for his presentation are available here online. (Select Baseline.)

Dr. Fetterman presented a translation of the evaluation language we are using at this stage of the empowerment evaluation to help make the process jargon free:

Normal Question (Evaluation Terms)

• Where are you now? (Baseline)
• Where do you want to go? (Objective or Goal)
• How do you plan to get there? (Activities or Strategies)
• How do you know where you are along the way? Are we there yet? (Benchmarks)
• How do you know when you have arrived at your destination? (Comparison of baseline with long-term objective or outcome)

Here is a bar chart that the group used as an example of how to visually report their own work.

Bar Chart: Baseline, Benchmarks, and Goal

Dr. Fetterman providing one-on-one technical assistance

Dr. Fetterman, Ms. Evans, and Ms. Delaney went from table to table, helping grantees and evaluators link activities to their baselines and benchmarks. It was a rewarding and productive day.