The Evaluation FAQs are divided into these topics:
1. What is evaluation?
Evaluation is an assessment that is as systematic and impartial as possible. It focuses on expected and actual accomplishments, examining the results chain, processes, contextual factors and causality to understand achievements or the lack thereof. It aims to determine the relevance, effectiveness, efficiency, impact and sustainability of WFP’s activities, operations, strategies and policies, and their contribution to the development and humanitarian processes of countries that receive WFP assistance.
Audit and evaluation are part of WFP’s accountability and learning framework. Audit and evaluation have different, but complementary approaches to ensure internal oversight and learning. Internal audit findings on processes complement evaluation findings on performance and results.
Internal audit provides WFP with an objective and independent assessment of whether WFP’s risk management, governance and internal control processes as designed and operated by WFP management provide assurance of accomplishing WFP objectives in compliance with WFP rules, regulations and policies.
Evaluation provides WFP with an objective and independent assessment of WFP’s activities, operations, policies and strategies to determine whether results were achieved and to explain why so that lessons can be learned and applied in other contexts.
Monitoring and evaluation are closely linked and mutually supportive. Evaluation needs and relies on a good monitoring system with sound performance indicators and reliable tracking. However, monitoring is a day-to-day management tool to ensure that input deliveries, work schedules, targeted outputs and other required actions are proceeding as planned. Evaluation, in contrast, is a strategic and periodic assessment of outcome-level results, and draws conclusions regarding performance not just for the intervention as such but for programming in general.
4. What are the guiding principles of evaluation in WFP?
WFP is committed to the principles of independence, credibility and utility for its evaluations. These three principles are equal cornerstones that determine the quality of evaluation. They ensure that evaluation can fulfil its objectives of accountability and learning.
Independence of evaluation means that it is free from influences that would bias the conduct, findings, conclusions or recommendations of an evaluation. It is essential to ensure evaluations are impartial. At WFP, structural, institutional and behavioural measures are in place to safeguard the independence of evaluation.
Credibility of evaluation is the extent to which evaluation findings and conclusions are believable and trustworthy. Credibility is determined by objective factors, such as accuracy of an evaluation report and transparency of the evaluation process, and by subjective factors such as the perceived or demonstrated impartiality and competence of evaluators.
Utility of evaluation is defined as the degree to which evaluations are useful to decision-makers within WFP and outside. Utility is determined through evaluation selection and planning, an intentionality that is shared by evalutors and managers of programmes under evaluation, the timeliness of the evaluation process and its report, and the accessibility of the evaluation.
The Evaluation Policy is the framework in which WFP through its Office of Evaluation put in place measures to safeguard these three principles.
The Office of Evaluation conducts
- Strategic Evaluations: take a global perspective and focus on strategic issues for WFP as a whole
- Policy Evaluations: examine how effective WFP policies have been in achieving their objectives
- Country Portfolio Evaluations: help Country Offices understand their portfolio and what difference it makes
- Impact Evaluations: provide an in-depth analysis of the impact that WFP has on beneficiaries
- Operations Evaluations: tell the story of relevance, efficiency and effectiveness of an operation
Yes, Country Offices undertake project reviews, including self-evaluations, and decentralized evaluations. Self-evaluations are typically conducted by the office itself or by staff from the regional bureau. Decentralized evaluations engage consultants.
7. Is the Office of Evaluation independent?
The director of the Office of Evaluation reports to the Executive Director of WFP. The Office of Evaluation is therefore independent from those directly responsible for designing and managing policies, strategies and operations.
The Office of Evaluation has direct control over the financial resources approved for its evaluation work programme and chooses the consultants for its evaluation teams.
The Office of Evaluation prepares an annual work programme and budget with an outlook for two additional years. This is part of WFP’s rolling 3-year Management Plan, which is approved annually by the Executive Board. (Prior to 2012, planning was biennial)
- Strategic Evaluations are selected through a consultative process that involves WFP stakeholders and the Executive Board to ensure greatest utility of the selected evaluations;
- Policy Evaluations are undertaken when the need arises for revising and updating an existing policy. They have also been requested directly by the Executive Board;
- Country Portfolio Evaluations are selected using country-specific and WFP-related criteria;
- Impact Evaluations are selected on the basis of WFP’s overall portfolio, focusing on areas with large numbers of beneficiaries and/or resources spent, and within the selected programme activity on countries that meet specific criteria;
- Operations Evaluations are selected on the basis of criteria that aim to generate a representative sample of the geographical distribution, size, and programme categories of WFP’s overall portfolio.
The Office Evaluation is funded from WFP’s Programme Support and Administrative budget. The resource level is determined annually by WFP’s Management during the process of establishing the management plan.
The resources cover the cost of the evaluation work programme and of the staff. In 2010 there are 11 staff including 1 director, 7 evaluation officers, and 3 support staff.
The Office of Evaluation is responsible for carrying out its work programme. Evaluation Officers, in their role as evaluation manager, are responsible for
- Developing the terms of reference for the evaluation;
- Consulting with stakeholders on issues to be addressed in the evaluation;
- Selecting, recruiting and briefing the evaluation team and undertaking the quality assurance of evaluation products.
Teams of independent consultants are hired to conduct the evaluation. The team composition is determined in the terms of reference and depends on the subject of evaluation.
The Office of Evaluation developed an Evaluation Quality Assurance System that sets out standards for processes, provides guidance on the content of evaluation products such as the terms of reference, inception reports, and evaluation reports, and includes quality checklists that ensure systematic quality assurance.
Evaluation findings are taken up in follow-up operations, in country strategies and in WFP policies and strategies.
All evaluation reports are posted on WFP’s public website.
The Office of Evaluation also has a mailing list which receives evaluation reports once completed.
The Office of Evaluation also uses its reports in the Closing the Learning Loop initiative which present lessons on specific topics or relevant to particular countries.
13. What are the main skills of an evaluation officer?
- Competency and experience in evaluation.
- Independent judgement.
- Good people’s skills to manage sensitive situations to maintain independence, while ensure evaluations are useful.
- Solid analytical skills.
- Ability to synthesize ideas clearly and concisely, both orally and in writing.
The Office of Evaluation has established a roster of consultants, which can be used to identify suitable consultants with experience in monitoring and evaluation.
The Office of Evaluation has established a roster of consultants, which is used to identify suitable consultants with experience in evaluation and with the right technical expertise. If you would like to register on the roster, please email WFP.Evaluation@wfp.org
In addition, advertisements are made through evaluation list servers when experts are required for specific evaluations and not found on the roster.
In addition to recruiting consultants individually, the Office of Evaluation is using companies, which are contracted through tendering processes.