Ex post evaluations are a key element of the AF-TERG FY21-FY23 strategy and work programme, originating from the request of the Adaptation Fund Board to develop post-implementation learning for Fund projects and programmes and provide accountability of results financed by the Fund.
They intend to evaluate aspects of both sustainability of outcomes and climate resilience, and over time feed into ex-post-evaluation-informed adjustments within the Fund’s Monitoring Evaluation and Learning (MEL) processes.
The framework developed by the AF-TERG is an innovative and Fund-specific framework to conduct ex post evaluations. Given the relative novelty of climate change adaptation portfolios and the limited body of work on ex post evaluation for adaptation, it presents possible methods that will be piloted in field-tested ex post evaluations in fiscal year 2022 (FY22).
Progress Update to the Board
A progress update on ex post evaluations (AFB/EFC.28/Inf.4) was presented to the twenty-eighth Ethics and Finance Committee (EFC) meeting in October 2021, to update the Board on progress in the AF-TERG ex post evaluations and the future piloting of such evaluations.
The document concludes that following the shortlisting of five completed projects as potential candidates for the pilots, two projects were selected for ex post evaluations:
– Enhancing Resilience of Samoa’s Coastal Communities to Climate Change (Samoa, UNDP)
– Enhancing resilience of communities to the adverse effects of climate change on food security, in Pichincha Province and the Jubones River basin (Ecuador, WFP)
The next step will be to train evaluators for the piloting of ex-post evaluation methods. This phase will enable the AF-TERG to revisit and shape methods with national partners, in line with its co-creation principle. This will allow evaluators to adapt methods to assess sustainability and resilience of project outcomes in the field.
Phase one report
The Phase one report on ex post project sustainability evaluation provides an overview of the first stage of ex post evaluations. Building on preliminary studies, the report presents the methods that will be piloted in two ex post evaluations in FY22. It also identifies a list of potential projects for ex post evaluation pilots from the Fund’s 17 completed and evaluated projects.
The framework presented in the report introduces possible methods to evaluate the sustainability of project outcomes, considering the characteristics, strengths and weaknesses of the Fund portfolio. It also presents an analysis tool to assess climate resilience, bearing in mind that this area is pivotal to climate change adaptation yet has rarely been measured.
The resilience analysis tool covers five components:
- The climate disturbances
- The human and natural systems (and their nexus) affected by and affecting project outcomes
- The characteristics of resilience in the outcomes
- means and actions supporting outcomes (exemplifying characteristics of resilience), and
- A typology of resistance-resilience-transformation (RRT) into which the overall project can be mapped based on how actions are designed to maintain or change existing structures and functions.
Training material for ex post pilots
The training material for ex-post pilots will be used for the training of evaluators and national partners before the field work. The training will focus on evaluating projects at ex-post and emerging sustainability and resilience, as well as presenting and adapting methods to country and project realities.
The training will be divided into three sessions:
Part A: Understanding ex-post & resilience evaluations. Introduce and understand ex-post evaluations of sustainability and resilience, especially in the field of climate change adaptation
Part B: Discussing country-specific outcome priorities and co-creating learning with stakeholders. Discuss the project and its data more in-depth to understand and select what outcome(s) will be evaluated at ex-post
Part C: Developing country-specific methods and approaches. Discuss range of methods with the national evaluator and M&E experts to best evaluate the selected outcome(s) and impact(s)