Independent evaluation

What is independent evaluation?

Independent evaluation is about testing the ideas, theories, methods, direction and ultimately the success of your programs. It's not (in the sense that we are using the phrase here) about acquittals of grants, although it will likely involve input from grantees.

If yours is a large organisation you might have some of your own staff dedicated entirely to program evaluation. In a small organisation, you might have a staff member responsible for evaluations as part of a broader role (one that does not involve administering the grants program) or you might employ someone from outside your organisation to carry out evaluations.

How does independent evaluation work?

Evaluators will review organisational goals and then consider the strategies proposed to address them. The design of an evaluation will vary with each organisation, but typically might involve assessing the situation at the beginning, when strategy is decided; in the middle, halfway through the execution of the strategy; and at the end of a grant program. At each stage, the evaluators consider possible interventions in light of the results they would expect to see at that stage.

As a strategy reaches its end-point, there are four useful questions to ask:

• Were the strategy's goals achieved?
• If not, why not? If they were, what role did grantees play?
• What can you learn from the strategy? Can the lessons inform other grantmaking?
• Is there still work to be done?

Where do program staff fit in?

Evaluators work closely with program staff. Evaluators need to be diplomatic. They need to be aware of and utilise the wealth of knowledge and experience that program staff have, but also maintain their independence. It is hard for program staff to step back and apply a critical eye to their work and to the grantees they work with day-to-day.

In a healthy evaluation model, the aim is not to "catch anybody out" or slap anybody on the wrist. Quite the opposite: the aim is to help program staff, grantees, and everybody else involved to ensure the success of the program by providing constructive feedback.

When program staff and evaluation staff disagree, it's an opportunity for constructive tension. And so long as program staff have been consulted in the design of the evaluation, nothing should come as a nasty surprise.

Someone - your chief executive officer, for example - should be appointed to make executive decisions on any issues that evaluators and program staff don't manage to resolve. Senior leadership and board support for the process is essential.

Keys to a successful independent evaluation

• Establish a reasonable evaluation budget. It might be two cents to every funding dollar, or five cents to every dollar.
• Be clear about the theory of intervention, what you hope to gain.
• Have someone from outside the organisation take a look at the evaluation at critical points.
• Ensure that grantees who are consulted realise that the work of program staff is also being assessed.
• Consider each evaluation as actionable, something that can inform your work in future.
• Be open to new lessons, possibly hard lessons.
• Be prepared to cease programs on the strength of an evaluation.
• And equally, be prepared to invest further where an evaluation suggests that doing so is necessary or would be useful.