Planning an assessment always also involves defining a standard of comparison to evaluate the subject under study. How the standard of comparison is defined determines to a great degree how the measurements should be approached. The following dimensions must be kept in mind when doing this:
The measurements can be taken before the project, at the end, or after completion. If the aim is to investigate what happens over the course of the project, then additional measurements need to be made during the project.
- Reference values
Actual performance can be compared with the objectives, the initial situation or with a control group.
The following basic models can be used to measure the achievement of objectives, changes in the target group or the project’s influence.
A rigorous impact assessment can in principle only be carried out with a before-and-after comparison combined with a control group. This makes it possible to clearly attribute observed changes to the project and to exclude external influences. In practice, institutional donors increasingly demand this kind of comparative model. However, they are hard work; at least four measurements are needed. The choice of the control group and the taking of samples are no laughing matter. Smaller organisations in particular will hardly have the necessary resources to carry out such rigorous impact assessments, and even larger organisations have to consider when and where they make sense.
We would like to argue here that simpler models may sometimes be appropriate. They are, for example, suited for learning within organisations. However, to enable before-and-after comparisons a measurement must be made at the beginning of the project (baseline study); to enable target/performance comparisons clear objectives must be set.
It requires rigorous impact assessment methods to clearly attribute a result to an intervention. Simpler approaches are sufficient to make a plausible case that an intervention has made a contribution to achieving overarching goals. Whatever the case, it must be clear how the impact assessment has been conducted and what conclusions can be derived from it.
It is frequently the case that, in practice, only the final situation of a project is described. Yet, for an impact assessment, a simple description of the target group with no link to the objectives, the initial situation or a control group is not sufficient.
More complex models that allow for firmer assertions can be created by expanding and combining basic models.