Establishing a strategy for evaluation
Every evaluation undertaken by a grantmaker should be unique, and needs to be designed accordingly. Evaluation itself is easy - it is the design of the evaluation strategy that is difficult.
- What is it that we want to evaluate? Are we interested in the outcomes that we are trying to achieve, or are we interested in testing a method of delivery, or are we interested in the type of organisation delivering the program?
- What is the context for our evaluation? Is this a routine review of an existing program? Is there a lot of media, academic or political interest in the program?
- What is special about our program design? What are its key characteristics? Is it community development, is it simply delivering an output? Has it got any special features we are interested in?
- What is special about the delivery method? Is it being delivered by established organisations, or is it being delivered by new and emerging or small organisations?
- Who wants us to evaluate? Who are the key stakeholders? Does the bureaucracy want us to evaluate? Has a councillor or a council as a whole got a particular interest in the program or in an aspect of the program?
- When do we want to evaluate? Are we interested in the results at the end, or are we interested in the program's progress, or are there critical milestones that are important to us?
- Who do we want to conduct the evaluation? Is this something that we want to do in-house because we have the skills or because of its sensitivity? Or do we want external credibility for it and therefore should we engage a trusted contact or an academic organisation to carry it out?
- How do we want to evaluate? Are we interested in discrete numbers? Are we interested in the colour and the movement in the stories that people have to tell us? What do we intend to do with the evaluation results?Is it going to be a public document, or an internal document? Are we going to use it to feed into a budget process or to change the policy or the program or both?
- When do we intend to use the results? Before the next round? Before the next budget is finalised? When hell freezes over?