Result of Service
The evaluation will be conducted in accordance with the United Nations Evaluation Group (UNEG) norms and standards. It will serve a dual purpose of accountability and learning. The evaluation will assess the Medical Hardship Fund programme efficiency and its impact on beneficiaries’ lives, and how effectively it has helped reduce the medical financial burden of accessing health services for people with chronic illnesses. Additionally, the evaluation will examine the programme design to determine its relevance, efficiency, effectiveness, coherence, coverage, sustainability and impact within the context of UNRWA. It will identify any gaps or limitations and recommend areas for improvement. The study will evaluate the overall MHF programme design and all related policies with focus on specific MONACO MHF fund over the specific years (from 2017—2020 & 2021—2023). The evaluation should cover all beneficiary target groups covered under the MHF programme in the five operational areas of UNRWA in Lebanon. key evaluation questions The evaluation should follow the standard OECD-DAC criteria. The evaluation should comprehensively incorporate gender, human rights, and disability inclusion aspects into its examination, with a focus on humanitarian principles, protection, and accountability to affected populations. An inclusive gender and human rights analysis should be consistently applied across each criterion to thoroughly assess how the diverse needs and vulnerabilities of women, men, boys, girls, individuals with disabilities, and the elderly have been accounted for in the planning, execution, and oversight of the MHF project. The evaluation should answer the questions set out in the below section. The evaluation team is expected to update the evaluation questions and formulate sub-questions in the inception report focusing on the MHF target groups. These questions are designed to ascertain the sustainability of fund in the medium to long term. The consultant/team is also expected to develop the evaluation matrix and the evaluation methodology. RELEVANCE – To what extent did the design of the MHF respond to the needs of the different stakeholder groups, including men, women, and people with disabilities? – How has the MHF policy adapted to recent changes in the national landscape? COVERAGE – To what extent has the MHF benefited the most vulnerable of the different population groups? COHERENCE – How well aligned are MHF activities to UNRWA strategic priorities and regular programme activities, including strategies on cross-cutting issues? – To what extent is the MHF complementary to the work of other relevant humanitarian actors? EFFECTIVENESS – To what extent were the objectives of the project achieved? – What were the major factors that influenced achievement of results? – To what extent the results framework of the intervention integrated Human Rights and Gender Equity dimensions? EFFICIENCY – To what extent has the intervention been implemented in a timely and cost-effective manner? – How efficient were the processes for managing and delivering project activities, including results monitoring activities? – To what extent does the allocation of resources to targeted groups prioritize the most marginalized? IMPACT – How has the MHF contributed to improving accessibility to necessary care for patients with catastrophic health conditions? – What impact has the MHF had on improving access to tertiary hospitalization support for extremely poor patients? – In what ways has the MHF decreased the financial burden on patients suffering from catastrophic health conditions or extreme poverty? – Are there other unexpected effects of the MHF? – What is the impact of the evolving scope of MHF on number of beneficiaries covered? SUSTAINABILITY – Is the MHF integrated into broader health systems strengthening efforts to ensure sustainability beyond the project duration? – How sustainable are the outcomes achieved through the MHF in terms of enhancing access to tertiary care, medication, and support for chronic diseases? Aapproach and methodology 8. The evaluation will be guided by the UNRWA Evaluation Policy and Standards and Procedures for Quality Assurance in Evaluation, which are based on UNEG Norms and Standards for Evaluation. 9. The evaluation will adopt a mixed-method approach to answer the research questions outlined above, including both qualitative and quantitative methodologies. Information collected will be triangulated to ensure soundness of the analysis. 10. Data and information collection methods will include, but are not limited to: – Literature Review: This includes reviewing all project related documents, historical reaching to the existing MHF concept note to identify best practices, history of MHF, changes and progress of provided services. – Survey: Surveying the current MHF recipients to gather data their experiences, including the type and amount of medical expenses covered along with demographic and other figures that may serve area of interest for donors, the ease of the application and implications of the process assessing gaps or limitations. – semi-structured interviews and focus group discussions with relevant stakeholders groups (e.g. Palestinian refugees, health programme and project Staff, health care providers and other relevant parties) to gather additional insights into the scope and coverage of the fund. – Data analysis of the existing programme database of different components to identify trends, gaps and areas of improvement. – Impact assessment: using qualitative and quantitative data to evaluate the economic impact of the MHF programme on people. – Develop actionable recommendations that are logically derived from findings of the study. The analysis and the presentation of data and information will, to the extent possible, be gender-disaggregated and take into consideration the needs of vulnerable groups. Data collection methods, such as Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs), will be conductedwith the logistical support of the Evaluation Manager (see 6.21.). The evaluator must assess potential risks and limitations associated with the data collection methods and propose mitigation strategies to ensure robust evidence triangulation. In the event that activities are hindered by security restrictions post-commissioning, the evaluator will collaborate with UNRWA and the donor to explore innovative and alternative data collection tools. Ethical considerations associated with these approaches will be thoroughly detailed for approval. -roles and responsibilities The Evaluator will report to the UNRWA Evaluation Manager. The Evaluation Manager will provide all documents and information required to the Evaluator, organize meetings/interviews with relevant UNRWA staff, provide backstopping and liaise regularly on the progress of the evaluation with internal UNRWA management. The evaluation deliverables will be quality-assured by the Evaluation Manager. Evaluation ToRs, inception report and draft evaluation reports are also reviewed by the Evaluation Reference Group. The Evaluator is expected to undertake the evaluation in consultation with UNRWA, in full accordance with the terms of references outlined herewith and in full compliance with the UNEG’s norms and standards for evaluation. Qualifications, experience and special skills The consultant should not have been involved in the design or implementation of the Medical Hardship Fund Project or have any other conflict of interest. S/he will act impartially and conform to UNEG ethical standards and norms in all parts of the evaluation process. – An advanced university degree (Masters or equivalent) from accredited educational institution in Public Health, evaluation, or another field of health relevance. – A minimum of 3 years of evaluation experience in evaluating health programming, and the use of health information/data to analyse project results. – Demonstrates a solid knowledge and familiarity with the region and UNRWA fields of operation, direct experience and knowledge of the Lebanese context is preferable. – Proven experience in the application of quantitative and qualitative evaluation methods and impact assessments, including consulting local communities, preferably in humanitarian contexts. – Expertise in applying gender, disability, and human rights dimensions in evaluation. – Expertise in the application of principles for ethical research, including research involving vulnerable populations. – Strong analytical skills coupled with a proven capacity to synthesize data and other information and report writing. – Fluency in oral and written English and Arabic. – Regional or national consultants are strongly encouraged.
Duties and Responsibilities
The Evaluator will report to the UNRWA Evaluation Manager. The Evaluation Manager will provide all documents and information required to the Evaluator, organize meetings/interviews with relevant UNRWA staff, provide backstopping and liaise regularly on the progress of the evaluation with internal UNRWA management. The evaluation deliverables will be quality-assured by the Evaluation Manager. Evaluation ToRs, inception report and draft evaluation reports are also reviewed by the Evaluation Reference Group. The Evaluator is expected to undertake the evaluation in consultation with UNRWA, in full accordance with the terms of references outlined herewith and in full compliance with the UNEG’s norms and standards for evaluation.
Qualifications/special skills
An advanced university degree (Masters or equivalent) from accredited educational institution in Public Health, evaluation, or another field of health relevance. Terms of Payment and Deliverables Any payment plan for the assignment would be as follows: 20% Upon successful completion of inception report 50% Upon successful completion of the first draft of the evaluation report 30% Upon completion of all required deliverables. A minimum of 3 years of evaluation experience in evaluating health programming, and the use of health information/data to analyse project results. Demonstrates a solid knowledge and familiarity with the region and UNRWA fields of operation, direct experience and knowledge of the Lebanese context is preferable. Proven experience in the application of quantitative and qualitative evaluation methods and impact assessments, including consulting local communities, preferably in humanitarian contexts. Expertise in applying gender, disability, and human rights dimensions in evaluation. Expertise in the application of principles for ethical research, including research involving vulnerable populations. – Strong analytical skills coupled with a proven capacity to synthesize data and other information and report writing