Operational-level evaluations refer to mandatory and non-mandatory evaluations focused on Fund-supported projects and programmes. There are several types of operational-level evaluations as outlined in the figure below. Evaluation guidance resources have been developed by the AF-TERG to support the implementation of these evaluations.
As stated in the Evaluation Policy, Implementing Entities (IEs), with the input of Designated Authorities, are responsible for commissioning or conducting evaluations for each project/programme. Mandatory evaluations for IEs include project baseline, mid-term reviews (MTR), and final project and programme evaluations. Non-mandatory operational-level evaluations include real time evaluations and ex post evaluations.
As required by the Board, the AF-TERG will conduct ex post evaluations three to five years after closure of selected Fund-financed projects to assess and inform learning from longer-term impact and sustainability.
Operational-level evaluation definitions
The section below outlines definitions and requirements of operational-level evaluations, as stated in the Evaluation Policy. This information is provided in part for reference for Implementing Entities (IEs), which are primarily responsible for executing baseline data reports, MTRs, and Final project and programme evaluations. Within operational-level evaluations, AF-TERG has the real-time evaluation and ex post evaluations within its purview.
Baseline data report
IEs shall prepare and submit a project baseline report based on primary data collection and/or relevant and reliable secondary data, per the Fund’s “Results Framework and baseline guidance.” Baseline data are to be used for designing the project, setting targets, and monitoring implementation progress, and assessing performance and outcomes. Baseline data shall be submitted to the secretariat by no later than the submission of the first PPR. The baseline data report may be conducted independently or semi-independently, or self-conducted.
Mid-term Review (MTR)
IEs are required to conduct MTRs of any project with four or more years of implementation, and to submit the MTR report to the secretariat no later than six months after the project midpoint. MTRs are formative evaluations to assess project performance and context to inform project management decision-making and course correction during the remaining implementation period. The IE is to submit a management response to the MTR report to the secretariat within six months of receiving the MTR report, describing what, why, and how MTR learning will be utilized. The MTR may be conducted independently or semi-independently. MTRs are optional for projects less than three years in duration; these may be self-conducted. All MTRs are publicly available in the respective project pages on the Adaptation Fund website.
Final project and programme evaluation
All Fund-supported projects and programmes that complete implementation should conduct a final evaluation to assess project/programme performance and impact to support learning and accountability, and inform future CCA interventions. The audience for final evaluation is broad, including the IE and the Fund, the intended beneficiaries and project participants, Fund partners, and other relevant stakeholders.
- Projects: All IEs are required to commission an independent final evaluation of their projects, submitted to the secretariat and the DA within nine months of project completion.
- Programmes: Programme evaluations approved and implemented under a single IE agreement are to be planned and budgeted as final project evaluations. Programme evaluations for projects involving multiple IEs are to be selected, planned, budgeted, and managed as thematic evaluations. Programme evaluations seek to generate lessons and insights across projects, countries, regions, and/or IEs. For the programme evaluations, IEs will be notified within three months of approval of the Fund’s evaluation budget whether their project has been selected for the programme evaluation. IEs are to submit to the secretariat and the DA or DAs a management response to the final evaluation report within six months of receiving the evaluation report. This should describe what, why, and how final evaluation learning will be incorporated into current or future AF Fund work.
All final project and programme evaluations are publicly available in the respective project pages on the Adaptation Fund website.
Real-time evaluation (RTE)
The nature of unpredictable environmental and social dynamics, (e.g., disruptions such as natural disaster, economic recession, pandemics, or social conflict), require projects to be nimble and course correct according to contextual changes and emergent learning during project implementation. RTE is “a range of evaluative approaches, reviews, and assessments with the purpose of understanding and articulating issues that need to be addressed in an ongoing development or humanitarian response, that can be fed back immediately into programming, decisionmaking and management processes with the overall aim of improving the response” (Buchanan-Smith and Morrison-Métois, 2021). RTEs are encouraged but not mandatory, but when planned they must be incorporated into the project budget. They may be self-conducted, independent, or semi-independent evaluations.
Ex post evaluations
As required by the Board, the AF-TERG will conduct ex post evaluations three to five years after closure of selected Fund-financed projects to assess and inform learning from longer-term impact and sustainability. Ex post evaluations will be useful entry points to support longitudinal learning, assess the Fund’s contribution to the wider CCA community, and to assess and track the Fund’s contribution to the GGA. They are not required of all projects but will be determined by the AF-TERG in consultation with the secretariat. Like thematic evaluations, ex post evaluations generate learning that contributes to achieving the Fund’s longerterm mission and goal, as well as the longer-term interests and needs of IEs. Ex post evaluations are to be budgeted by the AF-TERG under the Fund’s evaluation function. IEs whose projects will be selected for ex post evaluations will be informed within three months of approval of the selection.