Even if an effect has been observed and measured, one should not deduce from this that the result came about through the project alone. And even if the direct effect (outcome) can be clearly attributed to an intervention, this does not prove that this contributes to an overarching goal (impact). This state of affairs is known as the attribution gap.
However, it is not always thoroughly necessary to prove that a particular input has had a specific development impact. Depending on what the impact assessment is to be used for, it is often sufficient to make a case for the fact that the project has contributed to the observed change. A good results model plays an important part in this.
Attribution vs. Contribution Analysis
A distinction should be made between whether an effect needs to be proved and attributed to a development measure for purposes of legitimation (attribution), or whether a case needs to be made for a project or programme having contributed to improving a situation (contribution), e.g. for purposes of organisational learning or steering within the organisation.
Rigorous impact assessment methods are needed in order to attribute an outcome to an intervention. There must be a comparison with a control group to record what would have happened without the project. The analysis requires scientific, statistical methods. This makes it possible to exclude external factors and to clearly attribute the findings to a specific intervention. However, such comparisons are expensive.
Contribution analysis can be made using simpler approaches. It is already a good approach if a baseline study is carried out to enable before & after comparisons to be made.
It is only possible to say whether an intervention has had an impact at a higher level if the links and effects at this level have been proved through rigorous impact assessment methods. That is barely possible in practice. It is therefore all the more important to make a plausible case for further impact hypotheses.
Examples of how to formulate effects
The report should disclose which assertions can be made on the basis of the completed impact assessment.
- Clearly attributable effects
The project or programme has effects in a x% fall in child mortality in Region A from … to …
- Plausible effects
The project or programme has contributed to a x% fall in child mortality in Region A from … to …
It is expensive to close the attribution gap using rigorous impact assessment methods and, notwithstanding enormous effort, it is often impossible. Furthermore, the simplification to a link between cause and effect on which this method is based comes in for a great deal of criticism.
Participatory methods offer an alternative or complementary approach to this and can provide qualitative information about the effect of a development project. These centre primarily on asking the target groups what has changed for them as well as to which influences and to which actual project they attribute this result.
This approach takes the opposite perspective to a strict input/output model. First of all, it records the total change (gross impact), which encompasses external factors and side effects. Next, the causes of these changes are investigated using participatory methods. The aim is to ascertain the net effect that a specific project has had on the target group. Some of the models for this context-specific approach include MAPP (Method for Impact Assessment of Programmes and Projects) and Most Significance Change. These methods focus primarily on the changes in the target groups and/or their experiences. Alternative approaches are often rather controversial in practice, but they can be used in combination with the Logic Model. In particular, they allow one to complete quantitatively proven changes (What has changed?) with qualitative information (Why did it change?).
Due to the pressure to legitimise development projects, impact assessments are often carried out too early and in too much detail. Often, expectations of impact assessments are too high – and cannot be fulfilled.
If funders have unrealistic ideas about proving results, or if different funders have different conceptions, then the organisation should seek to agree on an appropriate impact assessment system with them. Aid agencies that have implemented an impact assessment system are better placed than organisations that have not yet developed any ideas of their own on the subject.