Consultancy for Final External Evaluation for project in South Sudan

GENERAL INFORMATION

Project background and Context

South Sudan, since its independence in 2011, faces extreme challenges in meeting basic needs due to violent conflicts, political instability, and socioeconomic underdevelopment. Over 80% of the population lives in poverty[1], with a fragility index just below Somalia and Sudan. While the country has experienced relative calm at national level since the signing of the revitalised Agreement on the Resolution of the Conflict in the Republic of South Sudan (R-ARCSS) and formation of the Transitional Government of National Unity in February 2020, there has been a spike in conflicts at sub-national level which have resulted in displacement of thousands of communities in many areas. An estimated 9 million people, including refugees in South Sudan, will experience critical needs in 2024[2]. Despite a 2018 peace deal, conflicts over resources and inter-community clashes persist. More than half a million Sudanese refugees and returnees from South Sudan have registered in South Sudan since the start of the Sudan crisis last April 2023. The COVID-19 pandemic has further exacerbated vulnerabilities. Food security situation has deteriorated due to multiple shocks, including flooding, ongoing conflict, displacement, and a high cost of living. The livelihoods situation in South Sudan is extremely challenging, shaped by ongoing conflict, economic instability, and environmental factors. The persistent violence and political turmoil have displaced millions, disrupting agricultural activities and livestock management, which are the primary sources of livelihood for most of the population. As per the Integrated Food Security Phase Classification (IPC) of October 2023, approximately 7.1 million individuals (56 percent of the total population of 12.4 million) are projected to face crisis-level or more severe acute food insecurity during the lean season from April to July 2024[3]. Many households have lost their productive assets, leading them to adopt negative coping strategies, such as reducing meal frequency, selling essential possessions, or engaging in unsafe practices. The lack of access to education and technical skills further limits opportunities for sustainable livelihoods, trapping many in a cycle of poverty and vulnerability. Women and girls are particularly affected, often bearing the brunt of food scarcity and being forced into early marriages as a survival strategy. The education sector is severely impacted, with more than 2.8 million children, or over 70 per cent, are out of school in South Sudan[4]. The majority of out-of-school children are girls, encountering significant obstacles in accessing education, including poverty, early marriage and GBV, especially for girls.

Oxfam and partners proposed a project to build resilience through education and sustainable livelihoods in Juba, Pibor, and Rumbek, the regions most affected by conflict and poverty. In June 2021, the Swedish International Development Cooperation Agency (Sida) approved an Oxfam proposal titled “Building resilience through gender and conflict sensitive approaches to education, skills development and sustainable livelihoods in South Sudan”. The project aims to improve resilience through gender and conflict sensitive education and skill-based solutions for sustainable livelihoods among the target girls, boys, women, and men. This is achieved through:

1. Ensuring that conflict-affected adolescents, youth, women and men have improved knowledge and skills through safe, quality and gender responsive education and skills development and
2. Strengthening inclusive participation and gender responsive local leadership to ensure resilient education systems and sustainable livelihoods.

The project’s outcomes and outputs and target groups are presented under Annex B. The intervention logic or theory of change of the intervention may be further elaborated by the evaluator in the inception report, if deemed necessary.

The project is implemented in Juba, Rumbek, Awerial, and Pibor, initially over a period of July 2021 to May 2025. The programme is funded under Sida’s Development Cooperation with South Sudan and had an initial total budget of 90,000,000 SEK subject to Swedish Parliamentary appropriations. In December 2023, the Swedish government took a decision to phase out Swedish development cooperation in South Sudan during 2024 resulting in a reduced budget to 78,000,000 SEK and project timeframe to December 2024. Oxfam Denmark is the project’s Cooperation Partner and contract holder (management and financial). Oxfam South Sudan is the responsible Implementing Partner in coordination with local partners: The National Education Coalition (NEC), Serving and Learning Together (SALT) in Pibor, Disabled Association for Rehabilitation and Development (DARD) in Rumbek, Women Aid Vision (WAV) in Awerial, and Support for Peace and Education Development (SPEDP) in Juba.

EVALUATION

Scope of the Evaluation

Purpose of the final evaluation

Oxfam has planned a final external evaluation to be undertaken as part of a culture of learning and accountability, in coordination between Oxfam Denmark, Oxfam South Sudan, and in partnership with the project local partners. The final evaluation will assess the project’s performance and achievements vis-à-vis the project’s overall objectives and impact, will generate lessons learned from the implementation of the project’s activities for future programming, and will develop specific recommendations with and for major stakeholders. The evaluation will use the OECD Development Assistance Committee (DAC) international evaluation principles, aspects and criteria as its base.

The primary intended users of the evaluation are: Oxfam, project local partners, and Sida. Other stakeholders that should be kept informed about the evaluation include community members, Relief and Rehabilitation Commission (RRC), Ministry of Education, Ministry of Agriculture and Food Security, Ministry of Gender Child and Social Welfare, Ministry of Animal Livestock and Fisheries, interested donors, and local authorities in state, country and Payam levels in the project locations.

The evaluation will cover activities carried out over the project implementation period (July 2021-December 2024). The evaluation will assess the actual results achieved. It will evaluate the sustainability, impact, effectiveness, efficiency, relevance and coherence. The evaluation will extract lessons learned, diagnose and critically analyse issues and formulate a concrete and viable set of recommendations.

The substantive work coverage will include a review of background activities towards field interventions, and of the actual measures of interventions in the four project areas, namely: Juba, Awerial, Rumbek, Pibor.

The review will cover both Oxfam South Sudan and partner-led components: Serving and Learning Together (SALT), Disabled Association for Rehabilitation and Development (DARD), Women Aid Vision (WAV), and Support for Peace and Education Development (SPEDP). It will be important for the review to assess the impact of the implemented partnership approach, its value add, and complementarity. It will also aim to review the complementarity between the education and livelihoods outcomes, but also more broadly the complementary approach in respect to other interventions (Oxfam and other organizations) in the locations of implementation. Additionally, the review will specifically assess the extent to which National Education Coalition (NEC) contributed to strengthening the inclusive citizens voices in decision-making process and management at different level of education sector in South Sudan through 1) strengthened NEC internal governance and organizational capacity, 2) NEC’s meaningful engagement with stakeholders to monitor and influence education policies, education management, access and quality of education in South Sudan.

Evaluation objective: Questions and criteria

A team of independent, external consultants will be contracted to assess the performance and results against the mandate that was set in the project design, and to determine the reasons for success or lack thereof, draw lessons and recommendations for improved performance in future similar interventions.

The evaluation shall conform to OECD/DAC (2010) “Quality Standards for Development Evaluation” and OECD/DAC (2014) “Glossary of Key Terms in Evaluation”, as well as the OECD/DAC (2021) “Applying Evaluation Criteria Thoughtfully“. Key guiding questions are suggested below and the evaluation team is not limited to them. The refining and further elaboration of the questions should be done by the evaluation team:

Relevance and coherence:

1. To what extent has the project objectives and design responded to: Target beneficiaries/communities’ needs and priorities; The strategy for Sweden’s development cooperation with South Sudan 2018–2022; The national development priorities of South Sudan, as well as to Oxfam’s and partner’s needs, policies and priorities and have they continued to do so if/when circumstances have changed?

• Gender perspective: What is the extent to which the project has accounted for gender differences and female/empowerment when developing and implementing project interventions? How are gender considerations mainstreamed into project interventions, especially promotion of leadership and at decision-making levels? Suggest measures to strengthen the project’s gender approach.
• Assessment of programmatic approaches outlined in the proposal and their interlinkages (education, skills based diversified livelihoods, resilience, gender justice and Gender Action Learning System – GALS- methodology, conflict sensitive and safe programming, climate and environmental adaptation) and how they were operationalized in the project implementation. For example, how has Oxfam operationalized the one program approach, integrating education, livelihoods and resilience activities?

2. To what extent have lessons learned from what works well and less well been used to adapt appropriately intervention implementation?

3. How compatible/synergetic has the intervention been with other interventions in the country, sector or organisation where it is being implemented?

• To what extent has the project strengthened resilience and addressed the root causes of humanitarian challenges through a triple nexus approach in the different project locations, while pointing to possible remaining gaps across co-ordination, programming, and financing?

Effectiveness:

1. To what extent has the intervention achieved, or is expected to achieve, its expected outcomes and objectives, including any differential results across groups?
2. To what extent is the project design logical and coherent?
3. Given that the crisis modifier implemented under this project is a new modality for Sida, what were its main results to mitigate the impact and strengthen resilience in communities and the learning (if any) from its implementation?

Efficiency:

1. To what extent has the intervention delivered results efficiently and cost effectively, in a timely way and been able to adapt to any changing conditions to date?

• Adequacy of management arrangements and capacities (including staffing structure) put in place in support of the achievements of results. With focus on the support provided by Oxfam (South Sudan, Denmark and GB), reviewers are to assess the role of Oxfam against the requirements set out (for example, field visits/monitoring and support, Project Oversight Group, reporting, planning and budgeting, technical guidance and learning, communication and advocacy etc.), including the quality assurance elements. Assess the contribution to the project from Oxfam assistance on advocacy, communication, and coordination.
• Partnerships: Assess how implementing partners are involved in the project implementation and management and are performing on the project, as well as the extent of their capacity strengthening, interaction and cooperation together for greater impact. Do the implementing partners recognize themselves as active partners in a joint initiative? Do the implementing partners take advantage of their individual capacities to reach optimized results?
• Assess how local stakeholders and community groups participate in the project (example the Parent Teacher Associations as part of school governance etc.) and include an analysis of the strengths and weaknesses of the approach adopted by the project and suggestions for improvement if necessary. Identify opportunities for stronger substantive partnerships between the key stakeholders of the project.
• Adequacy of monitoring and accountability systems: Assess the monitoring tools being used to determine whether they provide the necessary information, involve implementing partners, are efficient or whether additional tools are required to adequately track the project. Assess if the project has a well-established and functional accountability mechanism (this includes information sharing, transparency, complaints and feedback management).

2. To what extent have project-level monitoring and evaluation systems, reporting, and project communications supported the progress towards outcomes, contributed to learning, contribute to decision-making related to adapting aspects of the project based on the changing context?

Impact:

To what extent has the project generated significant positive or negative, intended or unintended, high-level effects (examples, effects for individuals, families, schools, communities, implementing partners and other organizations, the education system at payam, state, and national level)?

Sustainability:

To what extent will the net benefits of the intervention continue, or are likely to continue? What are the major contributing factors that influence the achievement/non-achievement of the sustainability of the project (for example, through the creation of structures, ownership, linkages and synergies with other interventions, and capacity building of project implementing partners and stakeholders at the local and national levels, etc.)?

Methodology

It is expected that the evaluator will describe and justify an appropriate evaluation approach/methodology and methods for data collection in the tender. The evaluation design, methodology and methods for data collection and analysis are expected to be fully developed and presented in the inception report. The evaluator is to present an approach/methodology that provides evidence to the evaluation questions. Limitations to the chosen approach/methodology and methods shall be made explicit by the evaluator and the consequences of these limitations and mitigation measures to address them presented in the tender. A gender-responsive approach/methodology, methods, tools and data analysis techniques should be used. Oxfam would like the evaluation to be utilization-focused and it is expected that the evaluation will advise how intended users are to participate in and contribute to the evaluation process and make use of the evaluation. The evaluation process shall be designed, conducted and reported to meet the needs of the intended users. It is expected that the evaluation will use both qualitative and quantitative data to support all findings and recommendations. Wherever feasible, the evaluator will gather success stories documenting perceptions of change and document them in the report.

Evaluation Management

This evaluation is commissioned by Oxfam Denmark (Senior Humanitarian Advisor) and will be coordinated with Oxfam South Sudan’s Programme Quality Coordinator, jointly the main Focal Points for the Final Evaluation. A reference group will be formed, and which will approve the inception report and the final report of the evaluation. The reference group will participate in the start-up meeting of the evaluation, as well as in the debriefing/validation workshop where preliminary findings and conclusions are discussed. A representative of the local partners will be presented on the reference group.

Evaluation quality

All Sida’s evaluations shall conform to OECD/DAC’s Quality Standards for Development Evaluation. The evaluators shall use the Sida OECD/DAC Glossary of Key Terms in Evaluation and the OECD/DAC Better Criteria for Better Evaluation. The evaluators shall have an independent quality assurance during the evaluation process to ensure that the evaluation meets the quality expectations that are outlined in these TOR. The evaluators shall specify how quality assurance will be handled by them during the evaluation process.

Expected Timeframe and deliverables

It is expected that a time and work plan is presented in the tender and further detailed in the inception report. Given the local insecurity situation, the time and work plan must allow flexibility in implementation. The evaluation shall be carried out [30 September – 6 December 2024] and most of the work must be completed before 31 December 2024. The timing of any field visits, surveys and interviews needs to be agreed with Oxfam during the inception phase. The table below lists key deliverables for the evaluation process. Alternative deadlines for deliverables may be suggested by the consultant and negotiated during the inception phase.

Deliverables and deadlines

Start-up meeting/s – virtual – 30 September 2024

Participants: Meeting with Oxfam Denmark, Oxfam South Sudan, evaluation team

Draft inception report – Tentative 7 October 2024

This report will include in detail: the key scope of the work and detailed study methodology, the data collection tools, a work plan/schedule of tasks designating a team member with the lead responsibility for each task and deliverable (output), sources of data, and a data reporting plan.

Responsible: Evaluators

Inception meeting to address Oxfam comments – virtual – Tentative 14 October 2024

• Detailed methodology and data collection tools to be used
• Roles and responsibilities of the team members and description of their respective roles
• A complete work plan for the data collection and review period
• Agree on the final report layout

Participants: Evaluators, Oxfam Denmark, Oxfam South Sudan, NEC, SALT, DARD, WAV, SPEDP, Sida

Data collection (Juba, Rumbek and Pibor), analysis, report writing and quality assurance- Tentative 18 October – 9 November 2024

Responsible: Evaluators

Debriefing/validation workshop (online) – Power-point presentation of the key preliminary findings of the evaluation and recommendations- Tentative 11 November 2024

Participants: Evaluators, Oxfam South Sudan, partners, Oxfam Denmark.

Draft evaluation report- Tentative 18 November 2024

The draft report shall be written in English and should be no more than a maximum of 40 pages excluding annexes. The final report should have clear structure and follow the layout format of the reporting template provided (see Annex C). The executive summary should be maximum 3-5 pages understandable as a stand-alone document.

Responsible: Evaluators

Final evaluation report revised according to Oxfam comments- 2 December 2024

Responsible: Evaluators

Evaluation presentation – online virtual presentation- 6 December 2024

Participants: Evaluators, Oxfam South Sudan, NEC, SALT, DARD, WAV, SPEDP, Oxfam Denmark, Oxfam Great Britain, and Sida.

EVALUATION TEAM SPECIFICATION

The evaluation team shall include the following competencies:

Education

• A Master’s degree or higher in a relevant field in Public Policy, International Development, Development Economics/Planning, Economic, Public Administration and Management, or in any other related university degree.

Experience

• A minimum of ten years’ experience, expertise and knowledge in the field of evaluation of development programmes related to sectors, quantitative and qualitative research and analysis
• Extensive experience in working with international organizations and donors, relevant expertise in education and livelihoods programming and practices
• Experience of programme formulation, monitoring and evaluation of education and livelihoods programming
• Strong analytical and conceptual skills to clearly synthesize and present findings, draw practical conclusions, make recommendations
• Experience in assessing organizational capacity and gaps and ability to recommend the corrective measures
• Demonstrated excellent written and spoken communication skills in English
• Good understanding and knowledge of the South Sudan context and crises

The proposals will be evaluated according to the following criteria;

a) Technical and financial proposal, with technical proposal having more weight (30%)

b) Proposed personnel for the assignment (30%)

c) Consultancy firm/organization capacity (10%)

d) Demonstrated experience with contactable references in evaluation of education and livelihoods projects/programmes (skills and experience at Masters’ Level or higher) (30%)

e) Experience in actual on ground programming work with NGOs, UN, or donors will be an added advantage.

FINANCIAL AND HUMAN RESOURCES

The maximum budget amount available for the evaluation is 60,000 USD (all inclusive). The consultant/consulting firm will submit a financial proposal that indicates all-inclusive costs for conducting the evaluation. The consulting firm shall bear all costs associated with the preparation and submission of the proposal. All costs should be quoted in USD and will remain valid up to sixty days (60) from the day of proposal submission.

Invoicing and payment shall be managed according to the following:

1. 25% upon approval of the inception report, submitted after signing of the contract and a preparatory meeting. This report will outline in detail the key scope of the work and detailed study methodology and data collection tools; a work plan/schedule of tasks designating a team member with the lead responsibility for each task and deliverable (output); sources of data; and a data reporting plan.
2. 50% upon approval of the draft final evaluation report by Oxfam Denmark.
3. 25% upon finalization of the final evaluation report approved by Oxfam Denmark capturing all findings, recommendations and lessons learned from the final evaluation (in English – using guidelines on content outlined in Annex C). Issues requiring management response should be clearly outlined. The Report length should not exceed 40 pages in total (excluding annexes). The executive summary should be maximum 3-5 pages understandable as a stand-alone document.

All the outputs – reports, data base, etc, produced under this assignment will not be disseminated in part or whole without authority from Oxfam Denmark. Thus, the consultant firm shall not produce these materials in any form (electronic, hard copies, etc) to a third party without written permission from Oxfam Denmark. The consultancy firm shall handover to Oxfam Denmark a clean data set and transcriptions of the data gathered.

Relevant Sida project documentation will be provided by Oxfam Denmark and Oxfam South Sudan in the inception phase.

The evaluation team will be responsible for its own travel itinerary to South Sudan and necessary security arrangements in coordination with Oxfam South Sudan. Oxfam Denmark will be contracting the evaluation team and coordinating on travel arrangements with Oxfam South Sudan for country travel. Oxfam South Sudan will support and facilitate the data collection and help in coordination with the different stakeholders. If required, Oxfam South Sudan will assist in arranging stakeholder interviews, support identifying beneficiaries for different interviews, in-country transportation arrangements including the booking of UNHAS flights, identifying enumerators and translators in the data collection locations, availing training facilities for enumerators/translators, organizing of the cost of translators and enumerators related to the data collection, coordinating meetings with relevant government counterparts or other stakeholders, and obtaining the needed approvals. Some of these costs will be agreed with the consultants and deducted from the final evaluation budget (specified above) and consultancy payments.

For the TOR and annexes, please download from the following: https://oxfam-dk.career.emply.com/da/ad/consultancy-for-final-external-evaluation-for-project-in-south-sudan/5l60w1

——-

[1] Global_POVEQ_SSD.pdf (worldbank.org)

[2] South Sudan: Humanitarian Needs and Response Plan 2024 (Issued November 2023) – South Sudan ReliefWeb

[3] South Sudan: Acute Food Insecurity Situation for September – November 2023 and Projections for December 2023 – March 2024 and for April – July 2024 IPC – Integrated Food Security Phase Classification (ipcinfo.org)

[4] Education UNICEF South Sudan

How to apply

Interested consultants must submit:

1. A cover letter including an expression of interest and related experience.
2. Technical proposal (maximum eight pages), including consultants understanding of the TOR, clear and elaborate workplan, CVs for all the consultant(s) proposed and proposed capacity of any complementary staff, and organizational capacity statement, experience and activities related to project/programme evaluations.
3. Financial proposal and including a suggested schedule of payments.
4. Applications should include 3 organizations that will act as professional references.
5. Two copies of similar samples of written work and relevant evaluations undertaken in the last 3 years.

Interested applicants must submit their proposal with all required documents merged in one standalone file including all them by 6 September 2024 to the following email: [email protected]

Incomplete applications or applications received after the closing date will not be given consideration. Please note that only applicants who are shorted-listed will be contacted.