END OF PROJECT EVALUATION LIFESAVING WASH, HEALTH AND PROTECTION EMERGENCY RESPONSE IN BAIDOA and LUUQ, SOMALIA 2023-2024
Organization
TERMS OF REFERNCE FOR END OF PROJECT EVALUATION
Lifesaving WASH, health and protection emergency response in Baidoa and Luuq, Somalia
28 October 2022– 31 March 2024
GLOSSARY
AWD Acute Watery Diarrhea
CBO Community Based Organization
CMR comprehensive medication review
CWB Child Wellbeing
DME Design Monitoring & Evaluation
EARO East African Regional Office
FGD Focus Group Discussion
GAC Global Affairs Canada
GBV Gender Based Violence
H/Hs Households
IDP Internally Displaced Persons
ITT Indicator Tracking Table
KIs Key Informants
MCHCs Maternal Child Health Centers
MoE Ministry of Environment
MoH Ministry of Health
NFI Non Food Items
NGO Non-Governmental Organization
NO National Office
ODK Open Data Kit
PDU Programme Development Unit
QA&S Quality Assurance & Strategy
SLT Senior Leadership Team
SO Support Office
SPSS Statistical Package for Social Sciences
SWS South West State
ToR Term of Reference
UASC Unaccompanied and Separated Children
UN United Nations
WASH Water Sanitation and Hygiene
WVC World Vision Canada
WVI World Vision International
WVSOM World Vision Somalia
Description of the project to be evaluated
The protracted humanitarian crisis in Somalia, brought about by conflict and political instability for close to 3 decades, is among the most complicated around the globe. Ongoing armed conflict and climate shocks have affected the lives and livelihoods of children and their families in Somalia. A UNOCHA report from September 2020 indicates that more than 1.3 million people face food insecurity, 849,000 of which are children at risk of acute malnutrition. The 2020 Post-Gu Seasonal Food Security and Nutrition Analysis report from October1 warns that even with humanitarian assistance, 1.7 million women, girls, boys and men are in danger of an acute food security crisis (IPC Phase 3 or higher) through May 2021. The proposed project area has been affected by cumulative shocks of drought, insecurity and displacement. The humanitarian situation is expected to deteriorate due to the triple crisis in Somalia concerning desert locust infestations, COVID-19 pandemic and cyclical flooding. Since July 2020, seasonal Hagaa flooding has resulted in the displacement of nearly 342,000 people, impacting 294 villages, destroyed property and approximately 20,000 hectares of assorted crops mainly in rural areas along the Juba and Shabelle river valleys.2
Natural Disasters: Humanitarian needs will further increase in 2021 due to the influence of La Nina. Potential drought conditions are expected as a result of a below average 2020 Deyr rainy season (Oct-Dec), an anticipated harsh 2021 dry Jilaal season (Jan-Mar), and a probable delayed or poor performance of the 2021 Gu (Apr-Jun) rainy season. The impact of drought is evident with an estimated 79,000 IDPs and 74,000 rural populations in Bay Region at risk of food insecurity (IPC Phase 3) with about 27,000 of the IDPs classified as emergency IPC 4. Consequently, the GAM nutritional status of IDPs is critical at 15-29.9%. In contrast, Somalia experiences annual flooding during the April to June and September to November rainy seasons. Since 2010, on average more than 70,000 people have been displaced by annual floods3. Recent floods from the ongoing Deyr seasonal rains (October-December) in Bay region have affected 66,000 people in Baidoa town and displaced 6,000 households4.
Insecurity: Conflict remains at the center of the crisis with chronic levels of insecurity and violence taking a heavy toll on civilians for decades, damaging livelihoods, and hampering economic progress and development. Conflict in Somalia largely stems from the activities of non-state armed actors who attack civilians and civilian infrastructure, impose illegal taxation and forcefully recruit children into armed groups.
Displacement: The cyclical nature of droughts, coupled with insecurity, has led to massive displacement of people, with an estimated 2.6 million people living in 2,000 displaced persons camps across Somalia5. In South West State, the majority of IDPs fled their areas of origin mainly Bay, Bakool and Lower Shabelle regions due to drought and conflict, choosing Baidoa town as a hub for safety, access to humanitarian assistance and livelihood opportunities. According to an EU Delegation to Somalia report, the city of Baidoa hosts the second largest number of IDPs in Somalia, after Mogadishu. A joint site verification exercise of August 2020 indicates that Baidoa district hosts 517 IDP sites, with 392,274 individuals (64% women and girls) from 59,921 households. Compared to the last site verification conducted in March 2020, there was an increase of 34 new IDP sites and 4,916 households6. The majority of IDPs create new temporary shelters in the outskirts of the city while others have joined the existing informal camps. 85% of IDP sites in Somalia are informal with inconsistent service provision or exclusion from accessing humanitarian support7. While there is good coordination between humanitarian agencies responding to need in Baidoa; efforts are limited in relation to the continually growing need to provide durable solutions and emergency support to more IDPs.
1.1 Project Goal
Reduced vulnerability, increased and maintained human dignity (especially for women and girls), and vulnerable lives saved of displaced populations affected by the protracted crisis in Baidoa, Somalia
Table1: Project Outcome and Outputs
Narrative
Outcome
Indicator(s)
Project outcomes
1100: Increased use of gender-responsive, culturally appropriate and environmentally friendly WASH services for W, G, B and M
# of girls, boys and women of reproductive age treated for severe or moderate acute malnutrition
# of WGBM accessing curative and preventive multisectoral services to prevent and respond to malnutrition
# of WGBM reached with essential emergency gender-responsive healthcare, protection, and nutritional services
# of persons with access to improved gender responsive, life-saving water and sanitation services
# of girls, women, boys, men reached with household gardening innovations to improve nutritious food intake
% of beneficiaries who receive cash who report satisfaction with assistance received
% of discharged cases with moderate/severe acute malnutrition who have recovered
% of discharged cases with moderate/severe acute malnutrition who have defaulted
GAC KPI: # beneficiaries attended awareness raising sessions/trainings on good nutrition, infant and young child
feeding, and care practices
GAC KPI: # of <5 girls and boys screened for acute malnutrition
# of CNVs supported with working tools and visibility materials (T-shirts, Lessos/lawas, gumboot, raincoats, umbrella,
back packs) to CNVs to facilitate village level screening
# of health workers trained on management of diarrhea in CU5 with SAM
# of health workers trained on screening CU5
# of Health & Nutrition Workers trained on CMAM and IYCFe
# of <5’s children without medical complications identified and treated at the nutrition site
GAC KPI: # of community members trained on SGBV prevention, protection risks, and available SGBV response
services
# of Child Friendly Spaces established/supported
# of Unaccomanied and Seperated Children reunited with family or placed in transitional care
GAC KPI: # of people reached through community outreach activities/ awareness raising on SRHR services or SGBV
prevention and response services
GAC KPI: # of community health workers/volunteers trained in SRHR or SGBV services
GAC KPI: # of beneficiaries who have experienced, or are at risk of, any form of SGBV that have received related
SGBV services
GAC KPI: # of beneficiaries provided with primary care consultations
GAC KPI: # of beneficiaries who have received sexual and reproductive health services, including access to
contraception
# of children under 1 and women of reproductive age immunized
# of individuals with gender segregated household sanitation facilities
Primary Sector/s
NA
Project Duration
NA
Objective: The evaluation is intended to establish the extent to which the project’s objectives (Outcomes and outputs) were achieved, understand the project’s contribution to child wellbeing, facilitate analysis of lessons learnt (what went well and what did not go well), and make recommendations that will influence future interventions in Baidoa and Luuq as well as similar contexts. Largely the evaluation will help in informing all project stakeholders on the extent to which the intended outcome and output objectives were achieved as well as highlight the relevance, appropriateness, effectiveness, sustainability, impact and lessons learnt from the interventions.
This will be an external assessment which will necessitate, undertaking a complete review and analysis of the program design document (Narrative Proposal) and its annexes including: Budget, Log frame, M&E plan and ITT together with progress reports (financial, narrative & monitoring) and available strategies (Somalia NDP & WVS NO Strategy).
Scope
Geographic area and target population
Since the project has been implemented in selected areas within Baidoa and Luuq districts, the evaluation will take place in these primary project areas.
Depth of investigation
Since this is the summative evaluation, it will assess the progress of the project towards the project goal and outcome starting 28th October 2022 to March 2024 and delve into the details in table 3.
Community level partners
The evaluation will involve all stakeholders that were involved in the project design and implementation including:
This will be a cross sectional and will employ a mixed method approach in data collection, triangulation and analysis drawing on quantitative and qualitative data. Both primary and secondary data will be collected, reviewed and analyzed. Household questionnaire survey (HS), focus group discussions (FGDs) and key informant interviews (KIIs) will be used to gather primary data involving both direct beneficiaries & indirect beneficiaries. The study will have three phases intertwined in the evaluation process:
In order to further ensure the quality of evidence, the evaluation will be designed with reference to the Bond Evidence Principles Checklist. Specific reference will be made to the 5 key dimensions of voice and inclusion, appropriateness, triangulation, contribution and transparency.
Quantitative Data: While designing the survey tool, the consultant will consider various factors. For this evaluation, quantitative approach will only be employed in gathering information at household level with focus on mothers with under five children and/or household care takers. The sources of indicator vary for different indicators. Items covered in the questionnaire will be framed from project outputs, outcomes and goal. Further in this approach, the evaluation team will adopt probability proportional to size (PPS) sampling technique designed by Taro Yamane (1967) with taking into consideration sample size used at baseline. This should be well elaborated in the inception report. With the help of local administration, the household numbers for the target villages will be obtained before commencement of study to facilitate scientific determination of the sample size. This will be based on the household numbers from each village falling within the project scope. Probabilistic methods will then be used to sample the key respondents at household level.
Scientific sampling will guarantee randomness in selection of respondents, validity and reliability of data collected. To generate the sample size from the population, the team will use the online sample calculator (generator) to arrive at the total households to be targeted for the evaluation.
Qualitative Data: Physical observations, Focus Group Discussions (FGDs) and Key Informant Interviews (KIIs) will be carried out with beneficiaries and stakeholders to understand the extent to which the project achieved the intended objectives and addressed community needs. The FGD guides will contain a checklist of questions generated from the main project objectives and activities. The evaluation team working together with project staff will select the participants of the FGDs based on the focus of the study. Mothers with children under five will be purposively included for the FGD.
The focus groups will target key stakeholders including beneficiaries, comprising; Pregnant and lactating Women (PLW), CHWs, WASH Committees, Protection Committees, facility staff, camp leaders, youth clubs and children’s clubs among others. Every FGD will have standard 6-12 participants within the project areas. For purposes of plural investigation, the exercise will be conducted with a broad range of representation within the community to enable triangulation of findings and incorporate wide-ranging perspectives.
Key Informant Interviews (KIIs) will be conducted with a wide range of stakeholders including; WV Project staff, Government officials from MOH, MoEWR & Min of Gender, Health facility in-charges and other implementing partners working within the project area.
Note: The evaluation team will also consider gathering information on the impacts of COVID-19 as a cross cutting theme by deliberately incorporating key questionnaires related to the pandemic. Key questions will be developed by the evaluation team or provided by WV QA&S team.
The evaluation team will undertake a thorough review of all available project documents as deemed necessary to gather vital information on the project design, implementation and monitoring processes as well as capture significant lessons registered over the lifetime of the project. Of necessity, the evaluation team will review the following literature:
Additionally the team will look at government strategic documents and reports, SDG targets and WVI child wellbeing outcomes to enable comparisons (where applicable) and validation of the findings.
Selection and training of enumerators: Enumerators with previous research experience and the ability to speak Somali language will be recruited. Key government official from Ministry of health and Water resources and social affairs will be engaged in order to support the process. Depending on sample size, a total of [xxx male, xxx female] enumerators will be engaged for evaluation and these will work hand-in-hand with M&E Officers during data collection. The field team (enumerators & M&E Officers) will be trained in data collection techniques using KOBO platform and mastery of the data collection tools before field data collection exercise. During the training, the field team will also be briefed on the objectives of the evaluation, how to identify the appropriate respondents at various levels and how to fill in the questionnaire appropriately. Emphasis will be put on research ethics, accuracy, and completeness among others
Data collections tools and methods: As indicated, the evaluation will use both quantitative and qualitative tools and methods for data collection, analysis and presentation. Information triangulation (using different methods) will be used to guarantee that the evidence supporting any conclusion drawn is representative of the situation. The evaluation will employ the following data collection methods during the evaluation;
Research ethical code will be applied throughout the study design, data collection and analysis phases. This will be reinforced during the survey team training.
Field data collection: The team will use mobile phones configured with KOBO mobile application for data collection (Uses WV server). The interviews and discussions will be conducted mainly in Somali language. Focus group discussions will be conducted by a key researcher using a predetermined focus group guide with relevant themes and sub-themes, while Key informant interviews will be conducted using a predetermined guide with open ended questions for specific informants.
Analytical framework: The evaluation team will use the project logic model and the evaluation criteria set out below to address the evaluation objectives as attached below. The tools and data collection will be aligned to this framework with emphasis on gender and social inclusion, and other cross cutting issues such as COVID-19 and locust invasion.
Table 2: Evaluation criteria and questions[1]
Evaluation Objectives
Suggested Key Questions
Rationale for asking Key Questions
Suggested Approach (To explore key question)
Assess the Relevance and appropriateness of the project design to the community needs
A thorough elaboration on these questions, will enable world Vision and stakeholders;
Assess the efficiency of the project in delivering and achieving project outputs and outcomes.
The key questions will enable WV and partners to:
Assess project effectiveness (progress of implementation and key outcomes achieved including the contribution on wellbeing of children)
These questions will enable WV and partners to:
Assessing sustainability of key activities, outputs and outcomes that have been initiated by project
Under this objective, WV and partners will be able to:
To establish and document key lessons learned during the project implementation
This will help WV and Partners;
TIMEFRAME
The assessment is expected to take …15… working days including preparation, data collection, and analysis and reporting. The Consultant should be able to undertake some of the tasks concurrently to fit within the planned time-frame, without compromising the standards of the assessment. The assignment is expected to commence on the 1st of February 2024
To be shared by consultant
Security in Baidoa and Luuq is volatile with likelihood of disruptions that may restrict staff from accessing some field locations hence affecting the quality of the evaluation. This will be mitigated by working together with our local line ministry staff right from the inception to the end, as this will empower them to adequately supervise the evaluation and update lead evaluators on a daily basis.
Support from WV Somalia
World Vision will be responsible for the following:
The Consultant will be responsible for the following:
6.1 EVALUATION CRITERIA
6.1.1 Mandatory documents
Note - Applicant who will not meet the above mandatory requirement will not be considered for Technical Evaluation.
6.1.2 Technical Evaluation by a Technical Team
6.1.3 Financial Evaluation
All bidders are advised to submit the following Financial Proposal:
As part of this Evaluation survey the evaluation team will perform the following tasks:
The deliverables resulting from this evaluation will be:
During the planning process (and throughout the evaluation process), the Evaluation team will review/use the following key program and other documents.
Final Evaluation report will be structured as below:
Qualified and interested parties are asked to submit the following;
Note:
As part of the Consultant selection process, the best five candidates will be required to make a presentation of the technical proposal to the Supply Chain and Core Project Technical team to inform the final decision on the award of the contract.
Payment Process
Payment upon submission and acceptance of the Inception Report to World Vision Somalia
40%
Payment upon submission of first draft of the assessment report to World Vision Somalia
30%
Payment upon submission and acceptance of Final assessment report to World Vision Somalia
30%
16 Provision of Taxation
1. The payment shall be subjected to withholding tax as required by the Law at the time of payment.
2. Payment will be bank transfer
3. The consultancy quotation will only include professional fees and related travel cost
.
Email Subject line: END OF PROJECT EVALUATION Lifesaving WASH, health and protection emergency response in Baidoa and Luuq, Somalia 2021-2024
[1] All data collection tools should be designed by aligning the research questions with a gender and social inclusion emphasis.
A prospective bidder making an inquiry relating to the tender document may notify WVS in writing at somalia_procurement@wvi.org. WVS will only respond to requests for clarification received no later than 10th Jan 2024.
Submission of Proposals:
Interested individuals and firms should submit their proposal accompanied with a cover letter, in English by email to somo_supplychain@wvi.orgon or before 30th Jan 2024.
Proposals should be submitted in three distinct/separate attachments, namely Mandatory Requirements, Technical Proposal and Financial Proposal (Bidders who will combine both technical and financial proposals shall be disqualified)
Email title should be; : END OF PROJECT EVALUATION Lifesaving WASH, health and protection emergency response in Baidoa and Luuq, Somalia 2021-2024
Bids received after deadline shall not be considered.
Interested individuals and firms should submit their proposal accompanied with a cover letter, in English by email to somo_supplychain@wvi.orgon or before 30th Jan 2024.
Proposals should be submitted in three distinct/separate attachments, namely Mandatory Requirements, Technical Proposal and Financial Proposal (Bidders who will combine both technical and financial proposals shall be disqualified)