In 2004, Fritz Institute launched its Humanitarian Impact Initiative with a conference that included participants from donor governments, humanitarian organizations, universities and foundations. The participants of the conference emphatically agreed that demonstrating impact was crucial to the sector. They also agreed that individual efforts to understand impact needed to be better linked to each other and the results accumulated and disseminated in order for the sector to benefit. Thus, a loosely knit network of interest around humanitarian impact was created. The network will be coordinated by Fritz Institute, and the next conference will be held in June 2006.
A significant theme at the 2004 Humanitarian Impact Conference was the use of participative approaches. In the absence of metrics about the effectiveness of humanitarian relief, the conference participants felt that asking the recipients of aid about the adequacy, quality and process of aid delivery would be useful.
The Initiative was launched at the Humanitarian Impact Conference in Washington, DC on June 28-29, 2004. In preparation for this event, the Humanitarian Policy Group of the Overseas Development Institute prepared a comprehensive literature review. The first of its kind, Measuring the Impact of Humanitarian Aid, assesses the current state of humanitarian impact.
One of the main obstacles to measuring the impact of humanitarian assistance is its contested definition. Without a clear definition, it is difficult to have a standard against which to compare outcomes. Thus, the conference began with a session on definitions. In defining the purpose of humanitarian assistance, the entire group agreed on the basic precept of saving lives and alleviating suffering, which can be measured by the reduction of excess mortality and morbidity. Others argued for a broader definition which included security and protection, the reduced dependence on international aid - including the restoration of livelihoods; local capacity building; disaster preparedness and mitigation - as well as the incorporation of human rights and dignity.
It was clear that the diversity of participants and approaches in the sector contributes to the lack of common definitions. Further, the discussion highlighted the fact that there are few commonly accepted measurement protocols and baselines, which make the aggregation of data and comparison of results across events exceedingly difficult. Furthermore, since agencies tend to evaluate their own projects, and since these evaluations influence funding volumes, it was generally acknowledged that poor results were inevitably sanitized.
When participants were asked individually about whether they thought the humanitarian sector was performing well, the group was almost evenly divided. Approximately one-third said a lack of evidence on the effectiveness of humanitarian interventions made it impossible to know. For example, there has only been one system-wide evaluation, the conclusions from which have still not been fully incorporated back into the system. Another third suggested that while the sector did appear to be doing well, it could be functioning much better. Getting more timely data, applying more rigorous standards in data collection and documentation and implementing learning from one intervention to the next were some areas that were suggested for improvement. The final third felt that the sector was not doing a good job and had little interest in its impact. This was suggested to have arisen as a result of the fact that organizations evaluate themselves - an inherent conflict of interest - and as a result, beneficiary perspectives are rarely incorporated and monitoring systems are, at best, patchy.
Theories of Change
Organizational policies are ideally derived from a theoretical model of the social change that they wish to affect. These theories, referred to as 'theories of change', produced a lively discussion amongst conference participants. However, it became clear that their practical application is very limited. Indeed, the instances in which a theory of change is clearly articulated at the beginning of an intervention, and where the staff implementing the program understood this theory and were able to connect what was done with the desired outcome were very rare (with the exception of health care interventions). Considering that the articulation of causal theory is critical to the identification and definition of indicators to assess humanitarian impact, this is a central area for future work of the Humanitarian Impact Initiative.
The Role of the Media
The discussion topic for dinner focused on the role of the media in influencing perceptions of impact. Why do certain humanitarian stories receive more coverage while others are forgotten? The results of the recent media study sponsored by Fritz Institute Toward New Understandings: Journalists and Humanitarian Coverage, were presented. This study, the largest and most comprehensive of its kind, illustrated that humanitarian organizations could strengthen their communication capabilities with greater press training of their field staff and greater use of technology in their interactions with journalists and the media.
Where Do We Go From Here
The final session of the conference focused on a discussion of what donors, agencies and academics could do to move the agenda of demonstrating humanitarian impact forward. Donors have already begun reflecting on impact through the Good Humanitarian Donorship initiative; however, the donor representatives suggested that many more efforts could be made. For example, why not establish clearly articulated targets with specific reference to humanitarian impact, similar to the field of development's Millennium Development Goals (MDG's)? Similarly, why not pool resources in humanitarian relief, as is currently the case with development? Why not have more system-wide evaluations to enable more focused improvements? Finally, the donors expressed a strong preference to have independent evaluators to assess performance, which does not originate from within the agencies themselves.
Agencies need to have greater clarification of the objectives of humanitarian assistance. These objectives must be reflected in the strategies that the agencies pursue as well as the evaluations that they undertake. The resources and institutions required to make this connection between objectives, strategies and outcomes must be strengthened, validated and systematized. There is a large role for academics, donors and independent evaluators to achieve this.
Academics also have a significant role to play in improving the assessment of humanitarian impact. By engaging in research around these issues academics can facilitate the creation of common indicators with definitions, protocols and procedures for their measurement. They can also work with agencies to build stronger links between theory and practice and, over time, facilitate the tools and methods needed to make data aggregation and comparison possible.
The need to define and measure humanitarian impact in a systematic and exhaustive way is essential to providing operational agencies with the tools appropriate to construct theories and methods to support humanitarian interventions. Similarly, communicating the effectiveness of impact will allow the humanitarian sector to respond to the increasing pressure of donors and the general public to demonstrate the results of its efforts.
Conclusion and Next Steps for the Humanitarian Impact Initiative
The Humanitarian Impact Conference confirmed the will and capacity shared by representatives of the community's different actors to collaborate on a community-wide effort to measure impact. At the request of the group, Fritz Institute has agreed to play a leadership and coordinating role in a series of projects and activities to move the agenda of demonstrating impact forward. This will involve:
- Mobilizing members of the donor, agency, academic and foundation communities into a formal network of interest around the subject of humanitarian impact. This may include the creation of sub-groups, a quarterly newsletter, and a website to coordinate existing and new efforts.
- Mapping and documenting what already exists in terms of research, indicators and databases around the world.
- Identifying resources to be systematically applied to the development of definitions, indicators, measurement and evaluation of humanitarian impact.