Program evaluation from design to implementation
Program evaluation is a systematic, evidence-based review process that assesses how well a grants program is doing. Like every element of a grants program, it needs to be carefully planned.
Tips on evaluating programs
- What is your program objective? Be clear about the program's stated purpose, and evaluate outcomes directly in relation to this.
- Examine both direct and indirect achievements when evaluating outcomes.
- Use triple-bottom-line reporting. Look at the economic, social and environmental program outcomes.
- Define your key performance indicators (KPIs) carefully. Apply the SMART test - are your KPIs Specific, Measurable, Achievable, Realistic, and Time-bound?
- Ask questions. When seeking feedback, look for both qualitative and quantitative information.
- Know your audiences and address them all - community, government and other stakeholders, for example.
- Examine project acquittals. Draw information from interviews, questionnaires and reports.
- Evaluate your evaluation.
What does a good evaluation look like?
A good evaluation is:
- able to demonstrate that the dollars are achieving the program's purpose (or not)
- set up at the start of the grants or funding program
- simple and easy to implement
- thoughtfully planned and structured; it asks questions that will elicit useful answers in a useful format
- based on easy data collection and collation
- based on easy analysis, which clearly shows trends and allows relevant comparisons
- repeatable and consistent, to promote a continuous improvement loop
- time-efficient and cost-effective
- able to clearly communicate findings to key stakeholders.
Implementing the findings
To make the most of an evaluation, funders need to act on its findings. Some useful questions to ask during this process are:
- What are the key achievements?
- What are the KPIs, and what do they tell you?
- What are the challenges that are indicated, and what are the disappointments (the aims not achieved)?
- What will you do next year, or as part of your new program?
These evaluation tips were presented by the then manager of the NSW heritage grants program, Miriam Stacy, to a past SmartyGrants grants management conference