Meaning and Importance — did it actually work?
Introduction
Topics 9, 17, and 24 covered designing, sequencing, and delivering a capacity development programme. This topic opens the final block of the course by asking the question that comes after delivery: did it actually work? Evaluation is not an optional add-on tacked onto the end of a programme — it is the mechanism that closes the loop back to Topic 12's discrepancy definition, confirming (or disconfirming) whether the original gap between "what is" and "what should be" has actually narrowed.
๐ฏ Learning Outcomes
- Define evaluation using Raab et al.'s (1987) formulation, as used in extension training literature.
- List and explain the OECD-DAC's six evaluation criteria.
- Distinguish monitoring from evaluation.
- Explain why evaluation matters for accountability, programme improvement, and confirming that training was the right remedy in the first place.
๐ Why This Matters
Raab et al. (1987), cited directly in the Halim & Ali/FAO (1997) chapter already anchoring this course, define training evaluation as a systematic process of collecting information about a training activity specifically so that it can guide decision making and assess the relevance and effectiveness of the training's components — evaluation exists to be used, not simply to be conducted.
1. Defining Evaluation
The Halim & Ali/FAO (1997) chapter defines evaluation broadly as a process to determine the relevance, effectiveness, and impact of activities in light of their objectives. Raab, Swanson, Wentling, and Clark's (1987) more specific formulation — developed for FAO's own trainer's guide to evaluation — describes training evaluation as a systematic process of collecting information for and about a training activity, used to guide decision making and to assess the relevance and effectiveness of the various components of that training.
Both definitions share a key feature: evaluation is judged against the programme's own stated objectives, directly echoing Topic 12's discrepancy model. Without a clearly stated "what should be" from the outset, there is nothing meaningful to evaluate against.
2. The OECD-DAC's Six Evaluation Criteria
OECD-DAC's evaluation criteria — first established in 1991 and revised in 2019 — are the international standard framework for judging the merit and worth of a development intervention. The same OECD-DAC source already anchoring Topics 2 and 18 provides six criteria, applicable to capacity development programmes as much as to any other development intervention:
| Criterion | Core Question |
|---|---|
| Relevance | Is the programme doing the right things, given the context and genuine needs (Topic 12)? |
| Coherence | Does the programme fit well with other related efforts and policies? |
| Effectiveness | Is the programme achieving its stated objectives? |
| Efficiency | Are resources (Topic 10's organising elements) being used well relative to results? |
| Impact | What lasting difference has the programme made, beyond its immediate outputs? |
| Sustainability | Will the benefits continue after the programme itself ends? |
These six criteria are not sequential steps but complementary lenses — together they provide a rounded picture of a programme's worth that no single criterion could capture alone.
๐️ Historical Note — Where "Objectives-Based" Evaluation Comes From
The idea that evaluation should be judged strictly against stated objectives traces back to Ralph W. Tyler, an American educator often called "the father of evaluation." Working on the Eight-Year Study at Ohio State University in the 1930s, Tyler developed what became known as the Tyler Rationale — a curriculum model built around identifying purposes, organising instruction around them, and designing evaluation specifically to determine whether those stated purposes were achieved. He formalised this thinking in Basic Principles of Curriculum and Instruction (1949), proposing a repeating cycle: establish objectives, define them in observable terms, collect performance data, and compare that data against the objectives — with any discrepancy triggering revision. This is, in effect, the same discrepancy logic already established in Topic 12, applied specifically to evaluation rather than needs analysis, and it remains the conceptual ancestor of both the Raab et al. definition and the OECD-DAC criteria used above.
3. Monitoring vs. Evaluation
Monitoring
Ongoing, continuous tracking of a programme's activities and outputs while it is being implemented — checking whether things are proceeding as planned in real time.
Evaluation
A periodic, more in-depth assessment of relevance, effectiveness, and impact — usually conducted at specific points (mid-term, end of programme) rather than continuously.
The two are complementary, not interchangeable: monitoring data often feeds directly into evaluation, but monitoring alone cannot answer deeper questions about whether a programme achieved genuine impact or represents good use of resources relative to alternatives.
4. Why Evaluation Matters
Accountability
Programmes funded through institutions like ATMA (Topic 22) are typically required to report progress and demonstrate results to justify continued funding.
Improving Future Design
Evaluation findings feed back into Topic 9's cyclical design process, informing the needs analysis (Topic 12) for the next round of programming.
Confirming Training Was the Right Remedy
Evaluation retrospectively checks Topic 13's Mager and Pipe distinction — if outcomes didn't improve despite good training delivery, the underlying problem may never have been a genuine skill deficiency.
Built Into Project Design From the Start
Topic 23's logframe already embeds evaluation directly into project design through its Indicators and Means of Verification columns — evaluation is not an afterthought but a planned feature from day one.
๐พ Extension Angle
A KVK farmer training programme's relevance can only be judged against the specific district-level needs identified in Topic 12's discrepancy analysis; its effectiveness against whether farmers actually adopted the practice taught; and its sustainability against whether that adoption persists after the training team moves on to the next village — three genuinely different questions that a single "did farmers like the training" reaction survey cannot answer on its own.
๐ฎ๐ณ Indian Institutional Context
ATMA's district-level funding structure (Topic 22) requires evaluation and reporting of extension activities back up through its Governing Board and Management Committee, directly operationalising the accountability function of evaluation described in Section 4 — funding for future programming is tied to demonstrated results from past programming.
๐ Beyond Agriculture
Corporate L&D functions face an almost identical accountability pressure — training budgets are increasingly justified to company leadership using evidence of business impact rather than participant satisfaction alone, echoing the "evaluate to show results" competency already introduced in Topic 21.
Frequently Asked Questions
- Raab, R. T., Swanson, B. E., Wentling, T. L., & Clark, C. D. (Eds.). (1987). A Trainer's Guide to Evaluation. Rome: Food and Agriculture Organization of the United Nations.
- Halim, A., & Ali, M. M. (1997). Training and professional development. In B. E. Swanson, R. P. Bentz, & A. J. Sofranko (Eds.), Improving Agricultural Extension: A Reference Manual (Ch. 15). Rome: Food and Agriculture Organization of the United Nations.
- OECD-DAC. (1991, revised 2019). DAC Criteria for Evaluating Development Assistance. Paris: Organisation for Economic Co-operation and Development.
- Tyler, R. W. (1949). Basic Principles of Curriculum and Instruction. Chicago: University of Chicago Press.