Evaluation Policy Guidance Development
The Board approved a new Evaluation Policy (EP) for the Fund at its thirty-eighth meeting in April 2022. The new EP will become operational in October 2023. During the transition between April 2022 and September 2023, the AF-TERG is working, in collaboration with the secretariat and the Evaluation Policy Guidance Advisory Group (EPG AG), to develop a series of evaluation policy guidance (EPG) to support operationalization of the new EP. This work supports evaluation capacity development (ECD) at the Fund that is aligned with its EP for a reliable, useful, and ethical evaluation function that supports the pursuit of mission and vision.
A dedicated page on evaluation policy guidance development keeps you up-to-date on how this work is progressing.
Review and revision of the Evaluation Framework
The Evaluation Framework is a key document supporting the implementation of the evaluation function in the Fund and guiding its evaluation practice. The review and revision of the Fund’s Evaluation Framework and associated tools and guideline documents is as such a core work element of the FY21-FY23 strategy and work programme of the AF-TERG.
The review and revision of the Evaluation Framework’s inception report provides the findings from the inception phase. During the inception phase, the team reviewed Board decisions and policies, strategies, and processes approved since the current evaluation framework was put in place. After reviewing different options for the types of instruments and those in use by similar climate funds and multilateral agencies, the inception phase found that an evaluation policy instrument might be best suited to the evaluation function of the Fund. A recommendation by the EFC to the Board was taken over in Decision B.36/32.
The Draft Evaluation Policy of the Adaptation Fund was presented and approved at the 38th Board meeting (Decision B.38/48).
The inception report, progress updates and unedited and graphically edited versions of the Evaluation Policy of the Adaptation Fund can be found on the dedicated evaluation framework revision page.
Thematic evaluations of Fund performance will provide perspectives on core features of the Fund, such as the country driven and innovative character of Fund operations with a view to assessing the potential for scale up and longer-term impact. Based on extensive consultation processes the currently commissioned thematic evaluations are:
- The experience of the Fund with innovation;
- Scalability in the Fund, and;
- Accreditation and related readiness in the Fund.
Other topics identified are knowledge management and learning, and the efficiency of Fund processes. More information can be found on the dedicated page on Thematic Evaluations.
Ex post evaluations
Ex post evaluations are a key element of the AF-TERG FY21-FY23 strategy and work programme that originated from the request of the Adaptation Fund Board to develop post-implementation learning for Fund projects and programmes and provide accountability of results financed by the Fund.
The work on ex post evaluations builds on a multi-phase process:
- Phase one, to develop a framework to conduct ex post evaluations and a shortlist of completed projects as pilots for ex post evaluation.
- Phase two, to test methods in at least two pilots.
- Phase three, to continue ex post evaluations over time, and related capacity building of evaluators, feeding into ex post evaluation informed adjustments within the Fund.
The Phase one report on ex post project sustainability evaluation, training materials for ex post pilots, as well as a progress update on ex post evaluations, to be presented at the twenty-eighth EFC meeting in October 2021, can be found on the dedicated ex-post evaluations page.
Mid-Term Review of the Medium-Term Strategy
The mid-term review of the Fund’s Medium-Term Strategy was a core work element of the FY21-FY23 strategy and work programme of the AF-TERG. The mid-term review aimed to track implementation of the Medium-Term Strategy, identifying progress and suggesting course correction as needed.
The mid-term review is the first step in the process to evaluate the Medium-Term Strategy of the Fund, to be carried out in fiscal year 2023 (FY23) after completion of the five-year strategy period. It is also a crucial input into the Overall Evaluation of the Fund planned for FY23 and provides feedback to the development of any future Medium-Term Strategy developed by the Board.
The final report was presented at the twenty-eight Ethics and Finance Committee (EFC) meeting in October 2021 and can be found on the dedicated mid-term review page, which also presents the inception report as well as an information document on emerging findings and areas for discussion presented at the twenty-seventh EFC meeting in March 2021.
Synthesis of Adaptation Fund Final Evaluations
The first synthesis of final evaluations covered all 17 final evaluations of projects/programmes received from 2015 to June 2020. It was based on the first cohort of completed projects in the Fund portfolio. The main findings of the review and synthesis were:
- Compliance with the Fund guidelines for final evaluations was high, with an average compliance level of 70 per cent
- Most reports (>70 per cent) were found to be easy to understand, comprehensive, and concise, yet about 50 per cent did not have a clear presentation of results
- Reports were based on evidence (82 per cent), described applied methodologies (65 per cent) and made use of evidence or monitoring and evaluation (M&E) data (82 per cent). However, evaluations seldom assessed the quality of data (29 per cent)
- Performance ratings were well substantiated, apart from M&E ratings
- No significant difference was found between the evaluation and the review ratings at the portfolio level, but the small size of the portfolio (n=17) prevented any meaningful comparative analysis.
In its first year of operation the AF-TERG commissioned three studies that explored discrete aspects of Adaptation Fund evaluation practice, and of the evaluation of climate change adaption more broadly.
While these studies were quite distinct, some common challenges faced by the Adaptation Fund and adaptation evaluation practitioners were identified. This briefing summarizes common problems that were apparent across all three studies, outlining the implications these challenges have for both Adaptation Fund stakeholders, and for the broader constituency of individuals and groups involved in the management and delivery of adaptation-focused monitoring, evaluation and learning (MEL).
The comprehensive evaluation phase 1 and phase 2 were completed before the AF-TERG’s establishment.
Comprehensive Evaluation of the Adaptation Fund – Phase 2
The purpose of this Phase 2 evaluation was to evaluate the long-term outcomes, impacts and sustainability of Adaptation Fund interventions, focusing on its portfolio of funded projects. Its objective was to examine and assess progress of the Adaptation Fund portfolio toward financing concrete adaptation projects and programs in developing countries Parties to the Kyoto Protocol. It assesses the portfolio’s relevance, efficiency, effectiveness and potential impacts and the sustainability of technical, institutional and financial results.
The evaluation, which builds on the Phase 1 evaluation and two reviews conducted by the CMP in 2011-2012 and 2014, also compiles lessons learned for the future of the Adaptation Fund portfolio. The management response to the evaluation can be accessed here.
Comprehensive Evaluation of the Adaptation Fund – Phase 1
The Adaptation Fund was established in 2001 to finance concrete adaptation projects and programs in developing country Parties to the Kyoto Protocol and those that “are particularly vulnerable to the adverse effects of climate change.” This Independent Evaluation is the first of two phases in a comprehensive evaluation of the Adaptation Fund.
This report presents the findings and analysis for the First Phase evaluation, which is a process evaluation that focuses on the Adaptation Fund’s operational performance from its establishment through June 2015. The management response to the evaluation can be accessed here.