Evaluability Assessments: A Good Strategy?

The evaluability assessment is an evaluation strategy which is also known as a clarificatory evaluation. The strategy is a process of examining the extent to which a program or project can be evaluated in a reliable and credible manner.  It informs whether or not your project is ready for an evaluation, preparing an organization to ensure that a planned evaluation will produce useful and relevant results. It is there to clarify all the necessary issues and information before you begin to carry out an evaluation for your project.

The evaluability assessment is an underutilized strategy in project management. It is fair to state that most NGOs do not carry out evaluability assessments. Focus has mainly been on carrying out actual project evaluations particularly the most common ones, (baseline assessments, midterm evaluations , impact assessments and end of term evaluations) without carrying out any preparatory work to ensure successful evaluations. At the end of the day, most evaluation findings have been substandard, lacking credibility and 'useless', because they have not been properly planned for. This is where an evaluability assessment comes in. The strategy if executed correctly can be the distinguishing factor between a successful and an unsuccessful evaluation. In this case, an unsuccessful evaluation would be one where evaluation findings do not answer stakeholder questions, findings are not conclusive and are not credible.

As far as the quality of an evaluation report is concerned, an evaluability assessment should be a prerequisite to carrying out project evaluations. The benefits that come from carrying out the assessment can not be overstated. They include the ones listed below:

• An evaluability assessment will inform the timing of your evaluations (whether to wait, proceed or not proceed with a planned evaluation).
• Prepares a program or project to generate all the necessary information that will be needed for all evaluation questions to be answered. This enhances the credibility of the evaluation findings.
• Fills the gaps that may have been left out during program design and M and E planning.
• Will inform you if the program or project is not ready for evaluation.
• Improves the prospects of an evaluation yielding useful results.
• Informs better allocation of scarce resources.
• Treats the 'ritualization' of evaluation processes.
• Can be used to reconcile differences between stakeholders and evaluators thus increasing the probability of successful evaluations.

The benefits of the assessment cover three important aspects of a program or project: the program design (analysis and or development of logic models and program theories), availability of relevant information, and context (is the context conducive to carry out an evaluation that is credible?)

Foregoing an evaluability assessment may seem like a great idea until you get into a situation where you have evaluations that do not offer any solutions because the evaluator could not obtain all the necessary information they needed to answer your evaluation questions or because the program's objectives were not SMART enough among many other issues. An evaluability assessment will help define the actual objectives of a program and measure the coherence or linkages between the objectives, activities, outputs and outcomes of the program.

The steps to consider when carrying out the Evaluability Assessment

1. Determine Purpose, Secure Commitment, and Identify Work Group Members. 

2. Define boundaries of Program to be Studied. 

3. Identify and Analyze Program Documents. 

4. Develop/Clarify Program Theory. 

5. Identify and Interview Stakeholders. 

6. Describe Stakeholder Perceptions of Program. 

7. Identify Stakeholder Needs, Concerns, and Differences in Perceptions. 

8. Determine Plausibility of Program Model. 

9. Draw Conclusions and Make Recommendations. 

10. Plan Specific Steps for Utilization of EA Data. 

The evaluability assessment is therefore useful to ensure the realization of quality evaluations. It will not only save you from wasting scarce resources on 'useless' evaluations but will add value to project management by identifying and correcting any shortfalls which may have occurred during project design. Most importantly it will help develop a robust logic model and program theory for the project. 

M &E for NGOs only? NO, And Here Is Why You Need It Too.

From IFRC, M &E Guide (2011)

Monitoring and evaluation is at the center of all projects in the non-governmental sector. From the momentum it has gained and continues to garner, maybe it is high time that governments and the profit making sector embraced the concept too, particularly because of its value to project and program success. 

In very simple terms, monitoring and evaluation are processes which are used to improve a project or program's performance and achieve results. More precisely, monitoring refers to the routine collection and analysis of information to track progress against set plans, and check compliance to established standards. The aim of monitoring is to regularly inform management and stakeholders of what is happening on the ground and inform decision making. On the other hand, evaluation refers to a systematic and objective assessment of an ongoing or completed project, program or policy, its design, implementation and results. Its aim is to determine the relevance, efficiency, effectiveness, impact and sustainability of a program or project in the fulfillment of its objectives and development goals. 

Some of the glaring reasons why any organization, government or private company implementing projects or programs should consider having a monitoring and evaluation team include the following:

• you will know early if you are making mistakes and correct them accordingly. 
• you will receive accurate, evidence based reports to inform decision making and improve project performance. 
• you will build organizational learning and knowledge sharing by reflecting upon and sharing experiences and lessons.
• you will be able to track any changes made during the implementation period for reference purposes.
• you will uphold accountability and compliance by demonstrating whether or not work has been carried out as agreed and in compliance with established standards.
• you will provide opportunities for stakeholder feedback, beneficiaries and clients to provide input into and give perceptions of your work. This will model openness to criticism, and willingness to learn from experiences and to adapt to changing needs. 
• you will promote and celebrate your work by highlighting your achievements thus building morale and team work. 

One of the biggest misconceptions is that monitoring and evaluation is only limited and valuable  to non-governmental organizations, and that it is of no value to governments and profit making organizations. However, what the non-governmental sector is gaining from their monitoring and evaluation systems is exactly what every government and business strives to achieve; SUCCESS! 

Monitoring and evaluation is not only a valuable management tool but also serves to ensure both transparency and accountability from project start to completion. An implementer will know what inputs they have, what each input has produced, the quality of products produced and whether the products serve their intended purpose or not, the hows and the whys. An organization must be accountable to itself (e.g to its board of directors) but also to those from where its resources are sourced. These could be taxpayers, clients, as well as donors. While this strengthens the relationship between an organization and its sponsors, donors and clients, it also builds trust for a more sustainable relationship. 

There is nothing worse than having your sponsors, donors or clients not understand and even get confused about what their money and other resources were used for after they see the finished product. It is every implementer's dream and goal to be celebrated for a job well done and that stakeholders are fully satisfied with the final product, project monitoring and evaluation ensures that!

The beauty of incorporating monitoring and evaluation into your projects is that it increases efficiency and effectiveness. Project managers will be able to identify mistakes, gaps and errors early and as result make quick but informed decisions on whether to continue with a project; correct the mistakes or terminate the project altogether. Such a situation will greatly save an organization from failure and from wasting scarce resources whilst at the same time increasing the chances of success for future projects. When a monitoring and evaluation system is in place, every stage of implementation is scrutinized and weighted against set goals thus eliminating chances for failure or substandard results.

Implementing a project without a monitoring and evaluation plan is like taking a road trip to nowhere but with expectations, you know you will get somewhere but have no idea if your final destination will be exactly as you anticipated. Thus having a project plan is not enough, a monitoring and evaluation plan is needed to ensure the success of the project plan. 

From the very beginning at project design stage, a monitoring and evaluation plan will show you what success will look like, and so when you are headed for failure, you will see it! This is because in the planning stage, a monitoring and evaluation plan will clearly articulate the indicators for each output and each outcome. As a result, the project team and all stakeholders are on the same level of understanding as to what they expect to see as success and hence any output and outcome that does not satisfy its indicator is questioned and scrutinized.  

For the government of Zimbabwe, one cannot help but wonder if monitoring and evaluation is the missing link in the continued failure of most if not all its past and present programs. From the land reform program of the 1980s and the fast track land reform of 2001; Zim-Asset of 2013; to the most recent Command Agriculture, the monitoring and evaluation aspect is silent. Just as democracy requires checks and balances for those in power to operate effectively, project and program success require checks and balances in the form of monitoring and evaluation in order to track and measure success and failure, answer the hows and whys.

Having said all the above, it cannot be disputed that monitoring and evaluation is a necessary management tool which must be adopted and utilized by all success oriented governments, organizations and managers. The costs of setting up a monitoring and evaluation system and hiring a team to do the job may seem like an unnecessary burden when one has no full understanding of the benefits offered by monitoring and evaluation. It is in the best interests of all organizations profit making or not-for-profit, to consider investing in a monitoring and evaluation team that will set up a monitoring and evaluation system and consequently set you up for success. Also remember, monitoring and evaluation data is only relevant when it is utilized. And No, monitoring and evaluation is not a thing for NGOs only, it is for all those committed to success!

Sources:

• IFRC. (2011). Project/programme monitoring and evaluation (M&E) guide. IFRC, Geneva\ (www.ifrc.com). 
• Frankel N., and A. Gage. (2007). M&E fundamentals, a self-guided mini-course. Chapel Hill, NC.