Post title - Project Evaluator

Duty Station -  Ukraine, home-based

Duration of Initial Contract - December 2021- February 2022

Expected Duration of Assignment - 25 working days within the timeframe December 2021 – February 2022

Candidates should apply via https://jobs.undp.org/cj_view_job.cfm?cur_job_id=102390

Governance Beyond Recovery. Responding to Strengthen Ukraine’s Resili-ence in the Face of COVID-19 Epidemic (RESURFACE)

Millions of people continue experiencing untold misery and suffering, as the COVID-19 pandemic over-whelms health, education, and economic systems and takes human development gains back, for the first time since 1990. While the summer of 2021 brought some relief, and the international pharmaceuti-cal scientific community has worked miracles by offering first COVID-19 vaccines in less than a year for (uneven) rollout worldwide, the pandemic is very far from over, the next wave is expected late 2021. As such, humanity is experiencing only the pandemic’s initial impacts, since the consequences of the measures we are taking in response will be far-reaching for years to come. Economies and social ties will face aftershocks for decades, as the world only begins to return to “normalcy” in 2021. The trajecto-ry of that return is obscure, shadowed by slower-than-anticipated vaccine rollout, new virus strains, economic troubles and political turmoil worldwide.

Against this backdrop, Ukraine has benefitted from a holistic UN assistance package from spring to fall of 2020. Yet, with the numbers growing drastically in the autumn-winter season of 2020 – 2021, on some days registering over 16 000 new cases, absence of robust forecasting, the growing threat of new COVID-19 strains (variants), widespread societal fatigue over quarantine measures, uncertainty over vaccination rollout plans, continued macroeconomic challenges and lack of significant progress with resolving the conflict in the east of the country or regaining control over the occupied Autonomous Re-public of Crimea, Ukraine is still far from being on the trajectory for surfacing from the country-wide epidemic or stabilizing the environment in which the country’s governance machinery currently oper-ates.

The RESURFACE Project is a mix of rapid response to the highly volatile situation and a longer-term resilience-driven transformation of Ukraine’s crisis decision-making and adaptation to extraordinary circumstances. The project is implemented for 12 months starting from March 2021 and has incorporated lessons learned from its predecessor Crisis Coordination and Management in Ukraine (CCMU) Project. Apart from practical results achieved between April and October 2020 under the small-scale and agile CCMU intervention, UNDP has taken note of ideas and proposals mentioned throughout the high-level policy dialogue meeting between the UNDP Administrator, Achim Steiner, and Minister of Foreign Affairs of Ukraine, Dmytro Kuleba. As the CCMU initiative drew to a close, Ukrainian government counterparts reiterated to UNDP Ukraine their satisfaction with results of the smaller engagement and contributed to the design of the RESURFACE Project by proposing activities, areas for collaboration and making sure that the submission is fully aligned with national priorities and needs including those, based on Ukraine’s Socio-Economic Impact of COVID-19 Assessment (SEIA) and the resulting national COVID-recovery plans.

Given the above, the proposed RESURFACE Project takes catalytic assistance to Ukraine’s government authorities to a new level than CCMU by deploying a mechanism that aims at not only delivering on the urgent needs of today but programming for more resilient Ukraine’s governance systems in two aspects (1) coordination and (2) institutional adaptability, as the country starts to emerge from the COVID-related crisis. Key beneficiearies of the project are Secretariat of the Cabinet of Ministers, Ministry of Foreign Affairs and Ministry of Culture and Information Policy. Desired Outcome is if the Government can increase the resilience of its internal (PMO / SCMU) and external (MFA) crisis coordination mechanisms, and if its ministries and agencies replicate good institutional coping strategies tested with one of them (MCIP), then Ukraine’s government institutions would be able to react more effectively to large-scale crises (not only COVID-related) and continue having in mind the larger 2030 Agenda and the principle of leaving no one behind, because a combination of the top-down (PMO, SCMU, MFA) and lateral (MCIP, MDT) experience-sharing and tangible positive results from more coherent crisis-response will encourage adoption of identified resilience-building strategies for better governance. Outcome of the project is supported by the RESURFACE Project’s Output: “The Government of Ukraine and the target Ministries have better capacity to provide a more coordinated, efficient, and inclusive response to the COVID-19 national crisis”.

SCOPE AND OBJECTIVE OF THE ASSIGNMENT

The main objective of the assignment is to conduct the forward-looking Final Evaluation of the Project “Governance Beyond Recovery. Responding to Strengthen Ukraine’s Resilience in the Face of COVID-19 Epidemic (RESURFACE)”. The purpose of the evaluation is to analyse the implementation of the project in 2021 and formulate lessons learned; and provide recommendations for scale-up and future initiatives in challenge-based partnerships.

The key product expected is a comprehensive evaluation report (up to 30 pages without annexes, single spacing, Myriad Pro font, size 11), which includes, but is not limited to, the following components:

• Executive summary (up to 3 pages);
• Introduction;
• Assessment of scope and objectives;
• Assessment approach and methods;
• Development context and project background;
• Data analysis and key findings and conclusions;
• Lessons learned and up to 7 recommendations for similar interventions (including viable ideas on areas which could be sharpened and further optimized in future interventions);
• Annexes;
• List of people interviewed; interview questions, etc.

This final evaluation will assess project’s performances against the review criteria: relevance, effectiveness, efficiency, sustainability and impact, in line with UNDP Evaluation Guidelines. Human Rights and Gender Equality will be added as cross-cutting criteria. More specifically, it will cover, but not be limited to, the following areas and preliminary questions:

 A.       RELEVANCE

The report will examine the extent to which the project is relevant to the:

• Country context: How relevant was the project to the interventions target groups, including Government’s needs and priorities? To what extent was the project aligned with the policies and strategies of the Government, SDGs as well as UNDP Country Programme Document/United Nations Partnerships Framework?
• Target groups: To what extent was the project relevant to address the needs of vulnerable groups and gender issues (both at project and stakeholder’s level)? To what extent did the initial theory of change for the project take those groups into consideration?
• Does the project remain relevant taking into account the changing environment while taking into consideration the risks/challenges mitigation strategy? Was there a need to reformulate the project design and the project results framework given changes in the country and operational context.
• What measures can be taken to improve the relevance of the project?

B.       EFFECTIVENESS

• Assess the overall performance of the project with reference to its respective project document agreement, strategy, objectives and indicators, and identify key issues and constraints that affected the achievement of project objectives. Were the planned objectives and outcomes achieved in the framework of the key project components?
• What are the results achieved beyond the logical framework? What were the supporting factors? What are the main lessons learned from the partnership strategies and what are the possibilities of replication and scaling-up? How can the Project build on or expand the achievements?

• How have stakeholders been involved in project implementation?
• What measures can be taken to improve the effectiveness of the project?

• Assess the project effectiveness at addressing the challenges around which the partnerships were formed?

C.       EFFICIENCY

The extent that to which:

• Was the project cost-efficient? Was the project using the least costly options? Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve the relevant outputs and outcomes?
• Has the project produced results (outputs and outcomes) within the expected time frame? Was project implementation delayed, and, if it was, did that affect cost effectiveness or results?
• Are the project’s activities in line with the schedule of activities as defined by the project team and annual work plans? Are the disbursements and project expenditures in line with budgets?
• Was the project management, coordination and monitoring efficient and appropriate?
• Assess the criteria of select project partners’ selection.
• What can additionally be done to improve the efficiency of the project?

 D.      SUSTAINABILITY

Sustainability is understood as the likelihood of continued benefits after the project ends. Assessment of the sustainability of project results will be given special attention.

• Is there sufficient public/stakeholder awareness in support of the project’s long-term objectives?
• Is the projects activity likely to continue, be scaled up, replicated and increasingly contribute to the development after the project?  Define which of the platforms, networks, relationships development in the framework of the Project that have the highest potential for further scaling up and/or replication.
• Are there any social or political risks that may jeopardize the sustainability of project results?
• Do the legal frameworks, policies, and governance structures and processes within which the project operates pose risks that may jeopardize the sustainability of project benefits?
• To what extent were capacity-building initiatives for partner organizations adequate to ensure sustainability? What could be done to strengthen exit strategies and sustainability?
• Identifying possible priority areas of engagement, offer recommendations for the next phase of the Project.

• To what extent do mechanisms, procedures and policies exist to allow primary stakeholders to carry forward the results attained on gender equality, empowerment of women, human rights and human development?

 E.        IMPACT

• Has the Project contributed or is likely to contribute to long-term social, economic, technical changes for individuals, communities, local governance self-bodies and other institutions related to the project?
• What difference has the project made to the direct beneficiaries, involved in the implementation of the initiatives, as well as indirect beneficiaries (target communities)?

• To what extent has the project promoted positive changes in gender equality and the empowerment of women? Were there any unintended effects?

The final list of evaluation questions and tools to be proposed by the evaluator and agreed with UNDP.

EVALUATION APPROACH AND METHODOLOGY

Methodology

The scope of the final evaluation will cover all activities undertaken in the framework of the Project. Given the forward-looking nature of the evaluation, the Evaluator will: a) compare planned outputs of the project to actual outputs and assess the actual results to determine their contribution to the attainment of the project’s objectives, as well as b) provide clear recommendations to UNDP, based on identified lessons learned in key areas of project implementation.

The evaluator will be required to use different methods to ensure that data gathering and analysis deliver evidence-based qualitative and quantitative information, based on diverse sources: desk studies and literature review, statistical analysis, individual interviews, focus group meetings, surveys and direct observation. This approach will not only enable the final evaluation to assess causality through quantitative means but also to provide reasons for why certain results were achieved or not and to triangulate information for higher reliability of findings. The concrete mixed methodological approach will be detailed in the inception report and stated in the final report.   All data provided in the report should be disaggregated by gender and vulnerability.

The evaluator is expected to follow a participatory and consultative approach ensuring close engagement with UNDP Country Office (CO), project team, government counterparts, international partner organisations all stages of the evaluation planning and implementation. The evaluation will assess the extent to which the project was successfully mainstreamed with UNDP strategic priorities, including eradicating poverty, accelerating structural transformations for sustainable development and building resilience to crises and shocks.

The evaluation of project performance will be carried out against the expectations set out in the Project Logical Framework/Results Framework, which provides performance and impact indicators for project implementation along with their corresponding means of verification. All indicators in the Logical Framework need to be assessed individually, with final achievements noted. An assessment of the project M&E design, implementation and overall quality should be undertaken. The evaluation will assess the key financial aspects of the project, including project budget revisions. Project cost and funding data will be required from the project, including annual expenditures. Variances between planned and actual expenditures will need to be assessed and explained.

The conclusions related to the implementation of the project should be comprehensive and balanced, and highlight the strengths, challenges and outcomes of the project. They should be also well-substantiated by the evidence and logically linked to the assessment findings. In addition, they should also provide insights into the identification of and/or solutions to important problems or issues pertinent to project beneficiaries and UNDP.

The recommendations for the project should identify how best practices and achievements of the project can be scaled up or proliferated to increase the positive impact of similar intervention countrywide. The recommendations need to be supported by an evidential basis, be credible, practical, action-oriented, and define who is responsible for the action - to have potential to be used in decision-making.

The evaluator should provide a proposed design, methodology of evaluation (methods, approaches to be used, evaluation criterion for assessment of each component to be proposed), detailed work plan and report structure to UNDP prior to the start of fieldwork; these documents and the list of partners and other stakeholders to visit should be agreed with UNDP. While proposing the methodology, the Consultant should be guided by UNDP approach to project evaluations.

The evaluator is expected to develop and present a detailed statement of evaluations methods/approaches in the inception report to show how each objective and evaluation criterion will be assessed.

The methodology will be based on the following:

1. Desk review of the documents listed below (but not limited to):

a)     The original project documents, action plans, M&E frameworks, and financial documents (such as the cost-sharing agreement with donor);

b)     Notes from meetings involved in the project (such as Board meeting minutes, National Steering Committee meeting minutes);

c)     Other project-related material produced by the project (such as datasets, publications, audio-visual materials and consultancies reports).

1. Interviews with the relevant UNDP Country Office and the project’s management and staff, and national authorities dealing with projects’ activities as necessary, to provide in-depth briefing on the projects, results, context of partnerships with different stakeholders etc. as well as vision for future.

3. Interviews and focus groups discussions with projects partners at the national level.

De-briefing session will be arranged for discussing the evaluation findings, results and recommendations.

Deliverables

The Consultant should provide the following deliverables for the evaluation of the project “Governance Beyond Recovery. Responding to Strengthen Ukraine’s Resili-ence in the Face of COVID-19 Epidemic (RESURFACE)”:

The detailed structure of the final report should be agreed with UNDP and reflect all key aspects in focus. Payment will be based upon satisfactory completion of deliverables. 100% of the total amount shall be paid upon completion of the Deliverables 1-3.

 MONITORING/REPORTING REQUIREMENTS, MANAGEMENT ARRANGEMENTS

The consultant will interact with UNDP project and CO staff to receive any clarifications and guidance that may be needed. He/she will also receive all necessary informational and logistical support from UNDP CO and the Project. On a day-to-day basis, consultant’s work will be coordinated with UNDP Project Manager. The satisfactory completion of each of the deliverables shall be subject to the endorsement of the UNDP CO Partnership and Coordination Officer (Evaluation Manager).

The consultant will inform UNDP of any problems, issues or delays arising during the implementation of the assignment and take necessary steps to address them.

The key product expected is comprehensive evaluation report (with parameters indicated above in section 2)

The report must be as free as possible of technical jargon in order to ensure accessibility to its wide and diverse audience. The Report should be prepared in English.

All reports and results are to be submitted to the UNDP in electronic form (*.docx, *.xlsx, *.pptx, and *.pdf or other formats accepted by UNDP).

EVALUATION ETHICS, IMPARTIALITY AND CONFIDENTIALITY

Evaluation consultants will be held to the highest ethical standards and are required to sign a code of conduct upon acceptance of the assignment. UNDP evaluations are conducted in accordance with the principles outlined in the United Nations Evaluation Group (UNEG) ‘Ethical Guidelines for Evaluations’. Evaluators need to sign the Pledge of Ethical Conduct.

The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses with the express authorization of UNDP and partners.

The evaluator selected should not have participated in the project preparation and/or implementation and should not have conflict of interest with project related activities.

The consultants must also agree to hold in trust and confidence any information or documents (“confidential information”) disclosed to them or discovered by them or prepared by them in the course of or as a result of the evaluation and agree that it shall be only used for the purposes of this evaluation and shall not be disclosed to any party without UNDP approval.

EXPERIENCE AND QUALIFICATIONS REQUIREMENTS

• Education: Advanced University degree (Master’s or PhD) in Economics, Management, Mathematics, Social Sciences, Public Administration, Business Administration or other relevant area;
• Relevant professional experience: At least 3 years of work experience in the area of public policy, sustainable development, private sector development, monitoring and evaluation;
• Experience in evaluation:At least, 3 accomplished complex evaluations projects where the candidate was the author or co-author understanding of gender aspects (a reference to or copies of previously developed knowledge materials including analytical reports, research papers, case studies materials, etc. to be provided);
• Proven knowledge of monitoring and evaluation methodologies, a summary of a proposed evaluation methodology is to be provided (up to 2 pages);
• Languages proficiency: Excellent knowledge of Ukrainian and/or Russian, English.

 DOCUMENTS TO BE INCLUDED WHEN SUBMITTING THE PROPOSALS

FINANCIAL PROPOSAL

Lump sum contract

The financial proposal shall specify a total lump sum amount, and payment terms around specific and measurable (qualitative and quantitative) deliverables (i.e. whether payments fall in instalments or upon completion of the entire contract). Payments are based upon output, i.e. upon delivery of the services specified in the TOR. In order to assist the requesting unit in the comparison of financial proposals, the financial proposal will include a breakdown of this lump sum amount (including travel, per diems, and number of anticipated working days).

The travel is not envisaged for this assignment. In the case of unforeseeable travel, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon, between the respective business unit and Individual Consultant, prior to travel and will be reimbursed. BSAFE course must be successfully complet-ed before the commencement of travel. Individual Consultant is responsible for ensuring he/she has vaccina-tions/inoculations when travelling to certain countries, as designated by the UN Medical Director. A consultant is required to comply with the UN security directives set forth under https://dss.un.org/dssweb/

EVALUATION CRITERIA

• Education: Master’s/Specialist’s degree or equivalent in Economics, Management, Mathematics, Social Sciences, Public Administration, Business Administration or other relevant area –  10 points max
  • PhD degree – 10 pts;
  • Master’s degree - 8 pts;
• Relevant professional experience: At least 3 years of work in the area of public policy, sustainable development, private sector development, monitoring and evaluation –
  15 points max
  • 6 and more years - 15 pts;
  • 4-5 years - 12 pts;
  • 3 years – 10 pts;
• Experience in conducting complex evaluations: At least 1 accomplished complex evaluation where the candidate was the author or co-author, with the understanding of gender aspects (a reference to or copies of previously developed knowledge materials including analytical reports, research papers, case studies materials, etc. to be provided) – 15 points max
  • 3 and more relevant evaluation projects -  15 pts;
  • 2 relevant evaluation projects – 12 pts;
  • 1 highly relevant evaluation projects -  10 pts
• Knowledge of monitoring and evaluation methodologies proved by summary of a proposed evaluation methodology - 25 points max
  • highly relevant methodology, based on previous successful experience with the examples of its use for such tasks, adapted to the needs of the target audience and TOR - 25 pts;
  • intermediate level of quality and relevance, methodology based on previous successful experience with the following examples of its use for such tasks - 20 pts;
  • acceptable quality and relevance of the methodology (the methodology is based on the information, provided in TOR) - 16 pts;
• Languages proficiency – 5 points max
  • Excellent knowledge of English and Ukrainian/Russian - 5 pts;

Maximum available technical score - 70 points.

 EVALUATION METHOD

 Cumulative analysis

Contract award shall be made to the incumbent whose offer has been evaluated and determined as:

a) responsive/compliant/acceptable, and

b) having received the cumulative highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.

* Technical Criteria weight: 70%

* Financial Criteria weight: 30%

Only candidates obtaining a minimum 70% from the maximum available technical score (70 points) would be considered for the Financial Evaluation

The maximum number of points assigned to the financial proposal is allocated to the lowest price proposal and will equal to 30. All other price proposals will be evaluated and assigned points, as per below formula:

30 points [max points available for financial part] x [lowest of all evaluated offered prices among responsive offers] / [evaluated price].

The proposal obtaining the overall cumulatively highest score after adding the score of the technical proposal and the financial proposal will be considered as the most compliant offer and will be awarded a contract.