TERMS OF REFERENCE (TOR) FOR EXTERNAL EVALUATOR FOR PLAY & LEARN PROJECT
About the Aga Khan Development Network
The Aga Khan Development Network (AKDN) is a network of private, non-denominational development agencies that share a mission to improve the living conditions and opportunities for marginalised communities across 30 countries in Africa and Asia. Through innovative, community-driven solutions that span social, economic, and cultural dimensions, AKDN agencies aim to holistically improve quality of life.
AKDN’s education goal is to ensure all girls. boys, women, and men, are equipped with the knowledge, skills, attitudes, and values to help them interact effectively with the world and be contributing members of a pluralistic society. AKDN assumes a holistic and systems approach to education programming ensuring all interventions are globally informed, locally rooted, are embedded in government and community interests and structures, and work to promote social inclusion, equity, and pluralism. AKDN invests in all sectors of the formal education systems, from pre-primary to tertiary, as well as in other formal and informal life-long learning opportunities, including teacher professional development. AKDN education agencies operating in East Africa are the Aga Khan Foundation (AKF) and its affiliate, the Madrasa Early Childhood Programme (MECP), Aga Kan University (AKU), Aga Khan Academies (AKA), and Aga Khan Education Services (AKES).
About the Aga Khan Foundation
AKF seeks sustainable solutions to long-term problems of poverty, hunger, illiteracy, and ill-health, with special emphasis on the needs of rural communities in mountainous, coastal, and other resource-poor areas. In East Africa, AKF responds to local and regional priorities by developing and implementing effective programming, measuring, and documenting results and sharing lessons with governments, donors, and development actors to influence policy and practice. AKF operates a multi-sectoral portfolio that includes programming in Education, Economic Inclusion, Civil Society, Health and Early Childhood Development. Gender Equality is a cross cutting theme in the portfolio. AKF will be the primary interlocutor for this assignment but engaging all AKDN education agencies in the process will be critical.
About MECP-K
- Madrasa Early Childhood Programme-Kenya (MECP-K); an affiliate of the Aga Khan Foundation (AKF) has been operating in the Coast region-Kenya for over 40 years now. The MECP is a regional initiative with operations in Kenya, Uganda and Tanzania. The programme works in partnership with underprivileged communities, civil society organizations, national and county governments to promote inclusive, equitable and gender responsive holistic Early Childhood development (ECD) services benefiting girls and boys aged 0-8 years to have the best start in life. This is achieved through establishing and supporting a network of high-quality, gender responsive, culturally relevant and sustainable community owned preschools in marginalized communities. Over 4,000 ECD teachers have graduated from MECP-K’s professional development courses, thus, directly benefiting over 300,000 boys and girls through improved ECD services in approximately 1,000 schools in the Coast region- Kenya. Further, the programme has a strategic focus on enhanced support for children 0-3 years and their families. This is achieved through collaboration with caregivers and families as well as with county departments of health and entails capacity strengthening key stakeholders’ aspects of stimulation, play, nutrition, hygiene and sanitation e.t.c. The trainings take into consideration local community strength and expertise thus, enhance uptake and sustainability of practice. Through this MECP-K has benefitted over 3,500 caregivers.
About Play and Learn Initiative
Play and Learn is a one-year pilot partnership between MECP-K and TheirWorld, spanning November 2023-November 2024. The project ‘Scaling playful innovations amongst caregivers from underprivileged communities” aims to ensure children receive the care and stimulation necessary for their optimal development. Key activities include:
- Supporting caregivers to deliberately engage in playful interactions with their children aged 2-5 in 25 community pre-schools in Kwale County-Coast region, Kenya.
- Providing Home learning and play kits (HPLK) for caregivers to use at home with their children aged 2-5.
The project targets 25 community preschools, 25 Preschool Teachers, 25 community Facilitators, 500 caregivers, 750 children aged 2-5. The key assumptions are thatcaregivers will willingly engage in playful interactions with their children and enhance caregiver-child bonding as well as stimulate the child’s learning and development. Additionally, the HPLK costs will be affordable and sustainable, and teachers will be flexible and available to participate in the initiative. MECP-K has already conducted the baseline evaluation to assess the entry behaviour as well as document the situation prior to the initiative in order to be able to compare this with the situation at the end of the initiative. The findings of the baseline evaluation will also be used to inform program roll-out. The evaluation will be used to assess the impact of the Play & Learn initiatives against the baseline study.
External Evaluation Objectives and Scope
MECP is planning to conduct an external evaluation of the initiative aimed at providing a comprehensive understanding of the “Play and Learn” project’s effectiveness, challenges, and potential for sustainability and scalability, thereby informing future decision-making and programming in similar contexts. MECP is looking for an external consultant to conduct the evaluation. The scope of the evaluation for the “Play and Learn” project encompasses several key areas:
- Project Outcomes: Evaluate how effectively the project has achieved its goals in ensuring that children from underprivileged communities receive the care and stimulation they need to thrive. Assess whether the project has met its intended outcomes as outlined in its design. This includes evaluating the success of supporting caregivers in engaging in playful interactions with their children and the distribution and utilization of Home Learning and Play Kits (HPLK).
- Documentation of success factors and challenges: Document both internal and external factors that have contributed to the success of the project as well as any challenges faced.
- Sustainability and scalability assessment: Assess the extent to which the project achievements can be sustained beyond the pilot phase and recommend pathways for scaling up to reach a larger population.
Table 1: Project Result Framework
Ultimate/Outcome Improved stimulation and home learning support for children aged 2-5
Indicator(s) % of children aged 2-5 receiving stimulation and home learning support
Data Source Reports
Target 65% of caregivers regularly engage in interactive play activities with their children
Data Collection method FGDs-6 caregivers per school, interviews for 25 lead caregivers, household visit tool, reflection meetings tool
Frequency of Data Collection FGDs-6 caregivers per school-mid-point and End
Interviews-25 Lead caregivers-mid point and End.
Reflection meetings-(tool to be developed)
Mentorship and support tool/household visit tool. Twice a month for 4 months (8 visits)
Immediate outcome: Increased participation of caregivers in developmentally appropriate play activities with their children aged 2-5
Indicators % of caregivers engaging in interactive play activities with their children disaggregated by sex.
Data Source Data from Mentorship and support tool.
Target 65% of caregivers regularly engage in interactive play activities with their children
Data Collection method Data from Mentorship and support tool
Frequency of Data Collection FGDs-6 caregivers per school-Mid point and End
Interviews-25 Lead caregivers- Mid point and End.
Reflection meetings-(tool to be developed)
Mentorship and support tool/household visit tool. Twice a month for 4 months (8 visits)
Immediate Outcome Enhanced knowledge, skills and practices of caregivers on playful interactions with their children aged 2-5
Indicators % of caregivers demonstrating increased knowledge, skills and practices on playful interactions
Data Source Data from Mentorship and support tool.
Target 65 % of caregivers demonstrate increased knowledge, skills and practices on playful interactions
Data Collection method Data from Mentorship and support tool
Frequency of Data Collection FGDs-6 caregivers per school-Mid point and End
Interviews-25 Lead caregivers- Mid point and End.
Reflection meetings-(tool to be developed)
Mentorship and support tool/household visit tool. Twice a month for 4 months (8 visits)
Methodology
The evaluation will be conducted in select households in one county – Kwale County from June 2024 – November 2024. The exercise will collect and analyze data aligned with the project expected outcomes and indicators. The consultant will be required to propose a detailed evaluation methodology. The study should adopt a mixed-methods approach, combining quantitative and qualitative research methods as follows:
Quantitative Methodology
- The consultant will review and refine existing quantitative data collection tools to ensure relevance and alignment with evaluation objectives.
- Quantitative data will be collected through surveys, questionnaires, or other appropriate instruments.
- Statistical analysis will include descriptive statistics, inferential statistics, and correlation analysis to measure quantitative impact.
Qualitative Methodology
- The consultant will enhance existing qualitative data collection tools and develop new tools, such as interview guides and focus group discussion outlines.
- Qualitative data will be collected through key informant interviews and focus group discussions.
- Data analysis will involve thematic analysis to extract key themes, patterns, and narratives from qualitative data.
Deliverables:
The External Evaluator is expected to deliver the following:
- Final inception Report: Presentation of a finalized inception report for validation
- Strengthened qualitative data collection tools and approaches, with a focus on capturing diverse participants perspectives including fathers’ perspectives on play.
- Data Collection and Analysis Plan: Develop/review a comprehensive data collection and analysis plan outlining the statistical and qualitative techniques that will be employed to analyze the collected data. This plan should include details on statistical software and methodologies to be used.
- Draft external evaluation report: Submission of a draft report for discussion and validation by MECP-K team.
- Final external evaluation report: Completed evaluation report including data visualizations, and analysis of findings. The completed final report will detail the project impact, outcomes, challenges, and highlight key recommendations.
- Study datasets: Raw data and complete datasets in English. The Consultant will also be expected to provide a fully ‘cleaned-up’ dataset in an agreed file format (e.g., XLS, SPSS) accompanied by the code used to carry out analysis and a variable codebook.
- Transcribed transcripts: Complete, edited transcriptions and translations in English of all focus group discussions and key informant interviews conducted.
Timeline:
The consultancy is expected to span approximately 12 weeks – spread out across 4 months, starting from 15th June 2024. A detailed timeline, including milestones and deadlines, will be collaboratively established during consultancy contracting.
Ethical Standards and Safeguarding
The consultant must conscientiously abide by AKF safeguarding Manual, and all members of the Consultant team must sign AKF/MECP-K’s Safeguarding Statement of Commitment upon contracting. The design and implementation of the studies must be in line with the research ethical standards including protection of participants from potential research-related harms, confidentiality, informed consent, beneficence, and safeguarding principles to protect respondents from potential harms such as sexual exploitation and abuse, sexual harassment, and bullying.
Rights to Deliverables
Upon successful completion of the consultancy, all deliverables, including evaluation tools, reports, visualizations, and findings, will become the exclusive property of MECP-K.
Qualifications and Expertise:
The ideal candidate for this consultancy should possess the following qualifications:
- Minimum of 5 years of experience in conducting evaluations, collecting data, and producing quality mixed-methods baseline/end line education focused study reports, preferably for non-profit organizations.
- Demonstrated experience in designing baseline and end line studies including proven experience in sound sampling, mixed methods approach (quantitative and qualitative), tool development, enumerator training, etc.
- Proficiency in statistical analysis tools (e.g., SPSS, STATA) and qualitative analysis software (e.g., NVivo, MAXQDA, ATLAS.ti).
- Demonstrated experience in employing KoBo Collect or other computer assisted personal interviewing software.
- Demonstrated experience in data management, including designing and managing gender- and age-disaggregated data and information systems capable of handling large datasets for study purposes.
- Strong analytical skills to interpret complex data sets and derive meaningful insights.
- Demonstrated experience in safeguarding and respect for vulnerable populations.
- Prior experience in early childhood context evaluations is a strong asset.
- Fluency in English is mandatory. Additional relevant languages (e.g., Swahili), an asset.
- Reporting and Coordination
The External Evaluation Consultant will be supervised by MECP-K Programme Director for guidance, feedback, and project coordination. The consultant will work closely with the MECP-K M&E Officer.
- Remuneration
The compensation for this consultancy will be negotiated based on the candidate’s qualifications, experience, and the intricacy of the evaluation process.
- Evaluation Criteria
Candidates will be evaluated based on their qualifications, previous relevant experience, sample work, and the proposed approach to conducting a robust mixed-methods evaluation.
CRITERIA MAXIMUM SCORE
Technical Component
Demonstrated experience in conducting and coordinating similar studies 25
Education and Technical Experience of Team 20
Proposed Methodology and Approach 30
Total Technical Component 75
Financial Component
Realistic and appropriate costs for proposed work 15
All relevant costs included to carry out proposed work 10
Total Financial Component 25
Total Technical and Financial Marks 100
The successful service provider is expected to duly comply with MECP-K contractual guidelines, National and international best practices in conducting programme evaluations throughout the period of engagement with MECP-K.
How to apply
- Application Procedure
Qualified and interested parties are asked to submit the following:
- Cover letter including the names and contact information of two previous clients who can be contacted regarding relevant experience. Consultants are also expected to disclose any conflict of interest related to this mandate with AKF/MECP-K.
- Detailed technical proposal of not more than 5 pages clearly demonstrating a thorough understanding of this request for proposals and including the following:
- Description of mixed-methods study approach and methodology, including preliminary sampling strategy, data collection, data management and data analysis and ethical standards
- Demonstrated previous experience in mixed methods and other qualifications outlined in this TOR
- Team composition and level of effort of each proposed team member
- A financial proposal with a detailed breakdown of costs for the study (taking into account an additional WHT)
- CVs of all proposed team members outlining relevant experience (annexed to technical proposal)
- A copy of a previous report of similar work undertaken on: a) baseline study; OR b) endline study
- Copy of legal registration (for registered firms)
Submissions must be received by MECP-K no later than Thursday 30th May 2024 at 5pm EAT through the email address: [email protected] along with ‘APPLICATION FOR EXTERNAL EVALUATOR – THEIRWORLD’ in the subject line. Only shortlisted candidates will be contacted