Terms of Reference for external evaluation

Context and Background

The project “Ecosystem-based adaptation, climate-resilience measures and institutional development in the Lower Dniester area” is implemented by consortia of partners: BIOTICA Ecological Society and Public Association EcoContact, with funding from Austrian Development Cooperation (ADC). The duration of the project is 48 months, December 2017 – November 2021.

The project area is the Lower Dniester Ramsar Site (LDRS) in the Republic of Moldova – one of the poorest European countries with rich, yet damaged nature and heavily transformed landscapes.

The project aims to continue developing the facilities and protected area management started in the previous projects supported by the Austrian Development Agency (ADA) (2012-2017), inter alia, for climate adaptation in Lower Dniester Ramsar Site (LDRS). Within the activities planned in the project are feasibility studies of supply/ sewage systems, developing the Engineering Design Documents and rehabilitating Water Supply System (WSS), including avoiding water losses and providing equal access for vulnerable groups, planting new forest areas with resilient species of trees. All above mentioned activities will contribute in a long term to the climate change mitigation and adaptation in the region. The project is expected to impact approximately 8204 direct rural beneficiaries (out of which about 4293 women, 99 social vulnerable persons, 378 representatives of authorities) with specific need for safe water and 40,000 indirect beneficiaries (21,000 women), providing equal access of women and men to improved services and information.

Defined by law as Natural Protected Area, and maintaining the highest biodiversity among the Moldovan Protected Areas (PAs), LDRS has no appropriate management in place. Despite the fact that local authorities have very limited capacity to manage the site, they support the building of a better management on participatory basis.

The LDRS Administrative Council will be promoted for a better management and documents for establishment of a National Park will be produced in favor of climate adaptation and resilience strengthening. Awareness raising/ education campaign (on protected area management, climate adaptation and linked economy issues), study tours, in-formation days, launching the LDRS 3-languages barrier-free web site and small grant program component for women will serve for LDRS promotion and development.

 The project outcome:

The long-term functionality of the Lower Dniester Ramsar Site is secured through improved conditions for its protection, enhanced ecosystem services and climate resilient livelihoods of the population.

The project outputs are as follows:

Output 1. Implemented climate adaptation measures for water supply systems in villages from the LDRS, ensuring equal and affordable access to water for women, men and social vulnerable groups.

Output 2. Legal and institutional framework for the creation of a Lower Dniester National Park is developed.

Output 3. Developed capacities of local communities from target area of the Lower Dniester for climate adaptation, nature conservation and management.

Output 4. Improved management of the Lower Dniester Ramsar Site through cost-based evaluation, biodiversity conservation and climate adaptation measures.

Output 5. Selected areas of the LDRS are rehabilitated through forestation on both river banks. Ecosystem services and climate mitigation are enhanced.

The Austrian Development Agency (ADA) is supporting the project, with funding from ADC (EUR 750,000), with co-funding in the amount of 83,300 Euro coming from ES BIOTICA, PA EcoContact, Sweden, as well as Cioburciu, Copanca, Popeasca and Talmaza mayoralties.

Purpose and Objectives

This partner-led evaluation is focused primarily to supply credible and reliable results for evidence-based decision making on strategic and operational levels and support the identification of why particular development interventions have worked or not, thus addressing two-fold evaluation purpose (steering and learning functions) as per ADA Evaluation Guidelines.

The purpose of this assignment is to conduct an evaluation of the overall project progress against the outcome, outputs and indicators of achievement as mandated by the donor ADA and stipulated in the project document / logical framework matrix.

Objectives:

1. To assess the effectiveness of the adaptation and climate resilience measures (water supply systems, forestation works etc.) in villages from the LDRS conducted in the frame of the project.
2. To assess the level of development of the legal and institutional framework for the creation of the Lower Dniester National Park.

Intended users:

• Primary users: Project stakeholders, in particular project implementing partners, ADA and other co-donors.
• Secondary users: Local public authorities from LDRS, Ministry of Agriculture, Regional Development and Environment.

3. Scope

The evaluation will cover the assessment of the key activities that have been implemented in the frame of the project.

Timing: The evaluation will take place in the period July-November 2021.

Geographical area to be covered: The assessment must cover: Project target villages from Stefan-Voda and Causeni districts.

Thematic focus to be covered by the evaluation: climate change adaptation measures and institutional development aspects in the region.

Evaluation questions

To address the scope of the evaluation, following OECD DAC primarily two criteria should be addressed:

• Relevance: Is the intervention doing the right things?
• Effectiveness: Is the intervention achieving its objectives?

The evaluation should be able to answer (but not limited to) the following questions:

Relevance:

• To what extent was the project design aligned with national environment planning priorities?
• To what extent are project activities and outputs aligned with the current needs and demands of project target groups?
• Were the objectives of the project clear, realistic and likely to be achieved within the established time schedule and with the allocated resources?

Effectiveness:

• To what extent were the planned activities and outputs of the project achieved and what were the factors that contribute to the achievement or non-achievement?
• How did factors outside of the control of the project affect project implementation and achievement of the project outcome / outputs and how did the project deal with these external factors?
• What are the reasons for deviations from the project implementation plan / time schedule? Were the steering measures set by the project (if any) effective and appropriate?
• To what extent were the legal and institutional foundations for the creation of the Lower Dniester National Park laid? What further steps would be required towards promotion / establishment of the National Park?
• To what extent were the recommendations from the gender and social standards appraisal included in the project?

Design and Approach

The evaluator(s) should address steering and learning aspects that can be drawn from the project but addressing the evaluation criteria stipulated in the previous section. It is expected that evaluator(s) will apply triangulation method in the process to ensure that multiply sources are used to enhance the credibility of findings. The data collection and analysis methods will be sufficiently rigorous to allow for a complete, fair and unbiased assessment. All data collected needs to disaggregated by sex.

The evaluator(s) should use various evaluation techniques and research methods. Methods should include quantitative and qualitative approached that will increase variety of information, reflections and insights and be a basis for data triangulation.

The list of evaluation methods, as well as approach and adjusted timeframe should be included into the Inception report. The evaluator(s) should consult ADA Evaluation Policy and ensure that proposed methodologies and approaches are in line with the donor requirements. The evaluation should follow ADC and OECD/DAC norms and standards as well as technical guidelines for evaluations.

The suggested methodology will include:

1. Contract and Kick-off meeting: The contract is signed and a discussion about the assignment takes place at BIOTICA office.
2. Desk study: The evaluator(s) studies all relevant materials and resources, including:

• Project proposal, including logical framework, timetable and budget;
• ADA´s gender and social standards appraisals including any recommendation thereof;
• Narrative progress reports presented to ADA;
• Studies and reports produced by the project;
• Guidelines for Programme and Project Evaluations of ADA;
• Other documents as necessary.

3. Inception-Phase: Submission of the inception report, containing the evaluation framework, detailed evaluation methodology, work plan and logistical arrangements.
4. Field-phase: visits to meet with the local stakeholders, beneficiaries and partners involved in the implementation of the Projects, including interviews with direct beneficiaries, and to determine the relevance and effectiveness on the ground.
5. Final Draft Report: Submission and presentation of final draft report, inclusion of comments from partners and contracting authority.
6. Final Report: Submission of final report including all annexes. The report must be written in English. The structure of the report should follow the structure provided in the technical guidelines for evaluations. The annexes must include these Terms of Reference and the Results Assessment Form completed by the evaluator (Results Assessment Form (RAF) Template 15 KB | XLSX).

ADA’s cross-cutting issues (Social inclusion/promotion of social protection and gender equality) have to be taken into consideration throughout the evaluation and reporting process.

The evaluator(s) should incorporate in the evaluation design, approach and methods the human rights based approach (HRBA), as well as the basic principles and quality standards applying to ADA’s programme and project design. The evaluation will follow ADC and OECD/DAC norms and standards as well as ethical guidelines for evaluations.

Workplan

Evaluation Management Arrangements

The project manager of BIOTICA Ecological Society will be in charge of communication with the evaluation expert(s). The expert(s) reports directly to the project manager. The project team will provide information for the evaluation process and will support the Evaluator(s) in conducting the evaluation. The project manager will circulate the draft report to key stakeholders and will collect comments. Based on that the evaluator(s) will finalize the report.

Evaluation management needs to respect the ethical standards and guiding principles for evaluation, including impartiality and independence.

Requirements for the Evaluator

8.1. General

– The evaluation will be done by an Evaluator or Evaluation team;

– Evaluator(s) must not have been involved in the design or implementation of the project being evaluated.

8.2. Qualification and skills

– University degree or equivalent background in relevant disciplines, with specialized training in areas such evaluation, project management, environment;

– Relevant professional experience in design and management of evaluation process, project planning, monitoring and management;

– At least 5 years of experience in the evaluation of the projects and proven experience in qualitative data collection and analysis (esp. planning and conducting semi-structured interviews and focus group discussions;

– Experience in evaluating at least 3 development projects in the Republic of Moldova;

– Fluency in Romanian and English.

The evaluator and evaluation team should respect the applicable COVID-19 restrictions.

Specifications for the Submission of Offers

The offer should include:

• Technical proposal (description of approach and methodology – up to 4 pages);
• CVs;
• Links to the evaluation reports in English from previous assignments if available.
• Financial proposal (in Euro, for the whole implementation of assignment) including travel, communication, health insurance and other relevant costs;

The award criteria for the evaluation of the best bidder are:

Quality (70%):

Price (30%)

The maximum available budget is 12 000 Euro. This budget includes the evaluator’s / evaluators’ fee and all expenses related to the contract, such as travel costs and eventual translation services.

The offer should be sent via e-mail to: biotica.md@gmail.com. Deadline is 30 June 2021, 16.00

Annexes

Project document “Ecosystem-based adaptation, climate-resilience measures and institutional development in the Lower Dniester” with all annexes (will be provided only to the selected bidder);

ADA Guidelines for Programme and Project Evaluations:

https://www.entwicklung.at/fileadmin/user_upload/Dokumente/Evaluierung/Evaluierungs_Leitfaeden/Guidelines_for_Programme_and_Project_Evaluations_ADA_2020.pdf.