Healthcare Standards

Midterm & Final Programme Evaluations and Beneficiary Satisfaction Survey


1. BACKGROUND OF THE PROGRAMME

The MENTOR Initiative has been present in C.A.R. since 2008. From the very start of the mission, core activities have focused on supporting community-based healthcare services on a large scale. This approach has proven to be effective and fully appropriate in a context where standard health system structures are virtually non-existent outside of main towns and where malaria, a disease that is easily diagnosed and treated if caught on time, is endemic. MENTOR is currently operational in eight sub-prefectures in north-western C.A.R.

In 2015, The MENTOR Initiative, International Medical Corps, Save the Children (phase out to Cordaid in 2016), and Oxfam entered into a Consortium partnership for the first Phase of the current programme, with International Medical Corps as lead of the Consortium. Phase 1 aimed to implement a health, nutrition, and WASH programme response reaching an estimated 432,665 persons across four prefectures of Central African Republic (Ouham, Ouham Pendé, Haute-Kotto, and Basse-Kotto) and aimed at preventing and reducing excess morbidity and mortality of the population affected by the ongoing crisis. A final programme evaluation from Phase 1 was undertaken and is available.

The 4 INGOs entered into Phase 2, with The MENTOR Initiative as lead, on October 1, 2016 for a period of 2 and a half years. Oxfam, as the partners specialized in WASH activities, was phased out as from February 1, 2018. The health, nutrition, and WASH programmes were implemented in the same 4 prefectures with the aim to build upon what took place during Phase 1 while harmonizing on best practices and developing standardized community sensitization tools. However, the security situation in the C.A.R. remained extremely volatile throughout the entire Phase 2 with frequent crisis and security deterioration. Consortium partners had to adapt and revise the programme to be able to implement pertinent emergency activities where necessary including centralizing the procurement of essential anti-malaria drugs to be able to face potential stock-outs

On April 1, 2019, the Consortium entered a third new phase, for a duration of four years, until March 31, 2023, with the same three health partners, The MENTOR Initiative, International Medical Corps and Cordaid, and Solidarités International as a new specialized member in charge of the WASH component of the programme. The focus of the consortium remained the same. The Consortium third phase was also informed by the findings and recommendations of the external evaluation conducted during and at the end of previous project’s phase-two. During this phase the consortium plans to improve its performance in terms of value-for-money building on the savings expected to be generated following the purchase of capital assets, such as new vehicles, significantly reducing rental costs. The consortium has expanded its community health strategy through community health workers as IMC has progressively adopted this approach. This phase of the project also aims at providing all health facilities with quality WASH infrastructure and access to safe water. The project has been designed with flexibility, with an emergency financial reserve allocation, to be able to respond to sudden crisis given the volatile operational and security context.

2. MID TERM AND ENDLINE EVALUATIONS

2A) GOAL AND OBJECTIVES

The project foresees two external evaluations during its lifetime: an immediate Midterm Evaluation by the end of project’s year-2 (March 2021) and a Final Evaluation to be conducted towards the end of the project on early 2023. The goal of the two evaluations is to assess and learn from the process and achievements of the programme in the four sub-prefectures, evaluate progress towards the program objectives and results, implementation strategies, assess the extent to which the programme demonstrates good “Value for Money” (using FCDO’s Value for Money approach), analyse consortium coordination mechanism, its advantages and recommend areas for its potential improvement, document lessons learned, and to inform the partners, FCDO, beneficiaries and other relevant stakeholders about results and findings..

Specific aspects for consideration

1)The evaluation will be guided by the evaluation criteria of relevance, effectiveness, efficiency, impact, and sustainability.

The evaluation should specifically assess:

· Relevance: Assess to what extent local needs and priorities have been addressed, activities and output are consistent with the 4 intended outcomes of the programme.

· Effectiveness: Assess what has been accomplished in relation to expected outcomes and results set in the logframe in each area of intervention and from a central level, specifying the major factors which have contributed to the achievement or not of the intended objectives; assess the extent to which the Consortium coordinates with other relevant actors in each zone and identify where this could be improved upon; assess the Consortium’s ability to adapt to evolving security and political contexts in specific intervention zones and C.A.R. in general.

· Efficiency: Assess how inputs (human, financial, material resources) have been translated into results and if results have been achieved at an acceptable cost – see Value for Money below; assess to what extent the Consortium approach and the management of the Consortium programme has provided added value in attaining achievements or has hindered it.

· Impact: Assess to what extent the programme has contributed to reducing excess morbidity and mortality of the targeted population affected by ongoing crises; assess how the programme has leveraged other complementary projects to achieve greater impact. Being a health project, a relevant area of focus shall be to evaluate the appropriateness and consistency of diagnoses and treatment, the rational use of drugs and the diagnostic steps followed by health personnel, this could be achieved by checking whether Community Health Workers and Health workers are following their diagnostic and treatment algorithms.

· Sustainability: Assess to what extent the programme will produce benefits in the long-term to the communities and the national Health System and identify gaps that could be filled.**

2) Accountability How each INGO has been accountable to beneficiaries, national and local authorities, and the donor.

“Value for Money”: In line with DFID’s policy, the evaluation will analyse the achievement of the Consortium towards identified “Value for Money” indicators and evidence that partners have effectively assessed and considered value for money considerations in the management of the programme. The evaluation should identify areas where improved synergies could translate into better Value for Money in the future.

Due to chronic insecurity in certain intervention zones, Consortium partners have developed and are implementing remote management tools. The evaluation should identify to what extent these tools and processes impede or promote accountability.

3) Accessibility How access to health services affected the various population in the 4 sub-prefectures specifically pertaining to health, nutrition, and WASH aspects. Identify if there have been factors preventing or restricting access of certain groups or minorities to healthcare services and potential strategies to address them.

4) Quality Evaluate quality of interventions in terms of indicators chosen and reported against, implementation methodology, competence of staff, team building, gender, and if health care provided through the programme is socially and ethically acceptable.

5) Lessons learned Evaluate the extent to which the Consortium has thus far acted upon lessons learned during Phase 2 (including evidence and data gaps) and identify further lessons learned, both positive and negative, that can be adopted during the current Phase

2B) EVALUATION TIMEFRAME, METHODOLOGY, AND DELIVERABLES

Design and methodology used in the evaluation will be proposed by the consultant, but should include quantitative and qualitative methods, a desk review of relevant documents, interviews of relevant stakeholders, focus group discussions and observation. Bids for the consultancy should provide a clear description of the design and methodology the consultant will use to answer the key questions, including recognized evaluation methods, proposed counterfactuals if/where appropriate, data collection methods, analytical methods, and approach to synthesis.

Visits to field sites are expected; however, security will be assessed before each field mission and the consultant would be expected to travel from Bangui to field sites only when partners have deemed security is sufficient.

The consultant will be provided with previous M&E documents including logframe, databases of medical data since the beginning of the programme, quarterly reports, etc. The consultant will be responsible for the identification and provision of any new primary data needed for the purposes of the independent evaluation. The consultant will need to determine which arrangements would be most cost-effective overall and least burdensome on beneficiaries or programme implementers.

MENTOR requests that all databases containing qualitative and quantitative data collected during the consultancy are to be shared in a commonly used format, together with clear metadata, and which is anonymized and safeguards confidentiality.

The timeframe for the immediate Midterm Evaluation is proposed as follows:

Task

Description

Duration

Deliverables

DESK REVIEW

The consultant will initially conduct a desk review of all materials related to this programme to familiarize her/himself with the programme background as well as its goals and objectives; The MENTOR Initiative will provide any relevant documents required by the consultant for this evaluation.

Methodology to be adopted, sampling strategy, work plan, list of persons to be interviewed, templates of data collection tools have to be shared with the partners through the Consortium Coordinator in an inception report produced after the review of the background materials.

5 days (approximate dates: March 1-5 2021)

Inception report

FIELD WORK

The fieldwork will be conducted in the three areas of intervention (Ouham Pendé, Haute-Kotto, Basse-Kotto) where possible and at coordination level in Bangui to start approximately the first week of March 2021 for approximately 3 weeks, depending on flights availability and accessibility in the areas of intervention. The consultant will be responsible for managing the evaluation process in-country, but will be supported by the Consortium Coordinator who will provide technical assistance (e.g. make in-country travel arrangements, assist in making appointments, provide data, documents and information available, etc.) and staff of the 4 different organizations while conducting activities.

The external consultant is expected to debrief the Consortium Coordinator and the 4 Country Directors on initial findings.

20 days (approximate dates: March 8-26th 2021)

Debriefing on initial findings/conclusions

REPORT

The evaluation will result in an initial draft evaluation report, which will be sent for comments to the Consortium Coordinator and all 4 Consortium partners. Ideally, this draft will be in French. The short papers will also ideally be provided initially in French.

After consultation, a final evaluation report is expected no later than 10 days after receipt of comments.

The final evaluation report (ideally provided in English and French by the consultant) should include a 1-page executive summary, background information on the programme and evaluation, the methodology adopted, presentation of key programme achievements, results and analysis of evaluation criteria clearly linked to evidence, summary of strengths and weaknesses, challenges encountered, conclusion, lessons learned and recommendations to respond to the objectives of the evaluation, as outlined above. The report should be a maximum of 50 pages with 11pt. single-spaced type.

10 days (approximate dates: March 27 – April 6 2021)

Draft evaluation report

Final evaluation report along with 1-2 short papers on key themes identified during debriefing on initial findings

Reports should communicate overall findings in an accessible way for non-technical readers, including presentation of data in visually appealing ways, highly structured and rigorous summaries of findings and robust and accessible syntheses of key lessons. Recommendations should be timely, realistic, prioritized, and evidenced-based.

The timeframe for the Final Evaluation to be completed in February 2023 will be proposed by the consultant, but is expected to be shorter (approximately 6 weeks) as the consultant will already be familiar with the programme. Relevant field visits to assess progress are expected where necessary especially to programme areas that could have not been accessed during the mid-term review due to security constraints. A draft and final evaluation report are expected.

3. BENEFICIARY SATISFACTION SURVEY

3A) Objective.

The survey aims to inform the consortium about how its procedures, activities and implementation are perceived and whether it is meeting beneficiaries needs. The information collected will enable the consortium to respond to beneficiaries needs within the scope of the programme. Although consortium partners already have feedback mechanisms in place, the survey aims at providing additional standardized information on the impact of the project and its perception among beneficiaries.

The objective is to engage and strengthen beneficiaries’ capacity to take an active role in shaping the programmes future, and bring about the changes needed to address vulnerabilities and build resilience in the specific context of CAR and within the scope of preventing morbidity and mortality and building dignity by enabling communities to access quality, timely health care.

3B) Methodology

The survey shall focus on qualitative aspects and shall cover beneficiaries and stakeholders at various level including direct project beneficiaries, disaggregated by type of health care service received, gender, eventual presence of disability, community leaders, local health partners. In order to maximize efficiency, the survey shall be conducted along with the mid-term project review by the end of March 2021 and a second survey will be conducted alongside the final evaluation scheduled for the first quarter of 2023) and it is expected to last up to two weeks.

The Consultant shall define in consultation with the Consortium Coordination team the most effective and efficient mix of investigation and analytical tool and methodologies to gather and present the data in a final report. Methodologies shall include a review of previous held beneficiary survey held within the consortium, key informant interviews, focus group discussions and community participatory techniques. The contractor may want to consider UNICEF’s Ground Truths Solutions surveys and shall use the Washington question to disaggregate beneficiaries by type of disability to analyze accessibility to and quality of health care provided to beneficiaries, including disabled ones. Results shall be compared to a similar satisfaction survey conducted during project’s phase-2 and identify what lesson can be drawn out of it to inform programming and response.

The Consortium, through its partners, shall identify, recruit and pay incentives to surveyors locally who will conduct the data collection in order to overcome the challenges due to local language and culture.

The consultant shall train the surveyors on the agreed methodology to collect data.

The Consortium shall provide logistical support to the teams of surveyors in selected areas.

3C) Deliverables

The output of the survey shall be a Beneficiaries Satisfaction Survey Report detailing the methodologies adopted, sets of questionnaires utilized by type of beneficiary, presentation of findings, sites covered, list of meetings and interviews held, challenges faced during the survey and potential limitations of the findings.

4. COMMUNICATION MANAGEMENT

The consultant’s primary contact in C.A.R. will be the Consortium Coordinator. Secondary contacts include the Consortium manager (based in UK) and MENTOR’s Head of Mission (based in Bangui).

5. CONSULTANT PROFILE

· Higher university degree in relevant field, i.e. Public Health, Development Studies, Humanitarian Response, etc.

· Demonstrable relevant practical experience in conducting evaluations in development and humanitarian settings using qualitative and quantitative research methodologies.

· Proven experience in using participatory evaluation methods, conducting evaluations for complex multi-sectoral, health programs in challenging settings, experience with FCDO-funded projects and/or Consortium programmes is an asset.

· Strong understanding of C.A.R. context is an asset.

· Strong computer skills including appropriate statistical analysis software.

· Excellent spoken and written communication skills in English and French, knowledge of Sango is an asset .

· Strong analytical and communication skills for complex, fast-paced environments.

· A strong commitment to delivering timely and high-quality results.

· Expected to sign and follow MENTOR’s Code of Conduct and the 4 principles and standards elaborated below.

Principles and standards

  1. The evaluation and survey should adhere to international best practice standards in evaluation, including the OECD DAC criteria for humanitarian evaluations and FCDO’s Ethics Principles for Research and Evaluation. Bids should demonstrate how they will achieve this.

  2. In line with Paris Declaration principles, the consultant should draw on existing data where available, ensure new data collection is complementary to existing systems and that new data are made available to stakeholders as far as possible.

  3. Disaggregation of data, including by sex, gender, and disability will be useful throughout the evaluation.

  4. The consultant will need to comply with MENTOR’s policies on fraud and anti-corruption and cooperate with any checks required from MENTOR for the duration of the evaluation.

  5. For the Beneficiary Satisfaction Survey, we expect the Consultant will need to work with a team of surveyors, who’s skills and background will need to be outlined in the proposal.

6. BUDGET

An appropriate budget including detailed expenditure estimated is to be submitted along with bids. Bidders are requested to be very clear about methodology providing a detailed breakdown of costs for the different significant activities to be undertaken during the evaluation and survey. The budget range for the 2 evaluations, for example is expected to be between $30,000 – $40,000, all included. Bidders are strongly encouraged to compete on the basis of their commercial proposal, demonstrating value for money, as well as technical proposal.

7. PAYMENT

There will be three payments for each evaluation/ survey enumerated as follows:

20% of fees for Evaluation/ Survey upon submission of the inception report

50% of fees for Evaluation/ Survey upon presentation of the draft evaluation/ survey report

30% of fees for Evaluation/ Survey upon submission of final report and all documentation as required by The MENTOR Initiative for final payment

8. BIDDING

Interested bidders should submit a CV, a well-developed methodology, timeline, a defined strategy of how any translators/ enumerators may be used and a financial proposal. The CV should clearly explain past evaluation experience, skills, and qualifications. Financial proposals should include detailed expenses.

Bids will also be considered for either the Evaluations or the Beneficiary Satisfaction Survey, if bidding for both please outline how



Source link

Most Popular

To Top