From gold-standard trials to the economic surplus method used across agricultural economics
Introduction
Topic 31 established impact as a counterfactual comparison (impact = Y1 − Y0) and flagged attribution and counterfactual estimation as its central challenges. This topic surveys the actual methods used to meet that challenge — ranging from the most rigorous experimental designs to the specific, long-established economic approach used across hundreds of published agricultural research impact studies.
๐ฏ Learning Outcomes
- Explain why randomised controlled trials provide the strongest counterfactual.
- Describe quasi-experimental alternatives used when randomisation isn't feasible.
- Identify the limitations of weaker, non-experimental designs.
- Explain the economic surplus method and its role in agricultural research impact assessment.
- Recognise the value of participatory approaches alongside quantitative methods.
๐ Why This Matters
The economic surplus method, first developed by Schultz (1953) and Griliches (1958) and later refined by Akino and Hayami (1975), has been used in hundreds of agricultural research impact assessments and remains the main analytical approach for assessing returns to agricultural research investment today — meaning it is not one option among many but the discipline's standard tool.
1. Experimental Methods: Randomised Controlled Trials
Randomised Controlled Trials (RCTs) randomly assign eligible participants to either a treatment group (receiving the programme) or a control group (not receiving it). Because random assignment makes the two groups statistically equivalent in every respect except the intervention itself, comparisons between beneficiaries randomly assigned to treatment versus control provide the strongest available evidence of a genuine causal relationship between the intervention and the measured outcome.
2. Quasi-Experimental Methods
Random assignment is often infeasible or ethically difficult in real extension settings — it may be unfair to deny a beneficial programme to a randomly chosen control group. Quasi-experimental methods approximate a credible counterfactual without true randomisation:
Difference-in-Differences
Compares the change over time in a treatment group against the change over time in a non-random comparison group, isolating the programme's effect from a general time trend — exactly the method illustrated numerically in Topic 31.
Matching Methods (e.g., Propensity Score Matching)
Constructs a comparison group by statistically matching non-participants to participants on observable characteristics, approximating what a randomly assigned control group would look like.
3. Non-Experimental Designs and Their Limitations
| Method | Key Limitation |
|---|---|
| Before-after without a comparison group | Cannot separate genuine programme impact from a general time trend, as Topic 31's worked example showed directly. |
| Surveys | May be subject to response bias and may not capture complex or nuanced outcomes. |
| Case studies | Offer rich detail but may not be generalisable to other contexts. |
| Econometric analysis (general) | Requires high-quality data and can be sensitive to how the underlying model is specified. |
These methods remain in use — often because of the cost and time constraints already flagged in Topic 31 — but their results should be interpreted with appropriate caution regarding attribution.
4. The Economic Surplus Method: Agricultural Economics' Standard Tool
Distinct from the general impact-evaluation methods above, the economic surplus method is specific to assessing the returns from agricultural research and extension investment. Pioneered by Schultz (1953) and Griliches (1958), and modified by Akino and Hayami (1975), it models how research-induced technology causes a commodity's supply curve to shift outward — increasing quantity produced and lowering price — and measures the resulting benefits to both producers and consumers using standard economic surplus concepts.
Results are typically expressed as Net Present Value (NPV), Internal Rate of Return (IRR), or a Benefit-Cost Ratio (BCR). Studies using this method across many countries and crops have found consistently high returns to public agricultural research — for instance, a Malawi groundnut research impact study using this exact method found substantial internal rates of return, part of a broader pattern in which public agricultural R&D studies typically report social rates of return well above 40%.
๐ Cross-Reference — Topic 19's T&V System
Evenson's (2001) work on the economic impacts of agricultural research and extension, cited in Topic 31 regarding the Kenyan T&V evaluation, sits within this same economic surplus tradition — showing that the same T&V system already studied for its personnel structure (Topic 19) has also been evaluated using this specific economic methodology.
5. Participatory Impact Assessment
Alongside these quantitative approaches, participatory methods — gathering beneficiaries' own accounts and judgements of what changed and why — offer a complementary perspective, particularly valuable when quantitative counterfactuals are infeasible or when local context and unintended effects matter as much as a single headline number. This approach connects directly to Topic 21's "value diversity" and "be humble" competencies: it treats farmers' own assessment of impact as genuine evidence, not merely anecdote to be set aside in favour of "harder" statistical methods.
๐พ Extension Angle
A KVK evaluating the impact of a new crop variety's promotion might combine a difference-in-differences comparison of adopting versus non-adopting farmers (Section 2) with an economic surplus calculation of the variety's aggregate benefit to the district (Section 4) and farmer testimonies gathered through participatory methods (Section 5) — triangulating across methods rather than relying on any single approach alone.
๐ฎ๐ณ Indian Institutional Context
ICAR institutes and agricultural economics researchers in India have applied the economic surplus method to assess returns on Indian agricultural research investment across various crops, following the same Akino-Hayami tradition used internationally — situating Indian agricultural research evaluation within the same methodological mainstream described in Section 4.
๐ Beyond Agriculture
Randomised Controlled Trials are the dominant impact-evaluation method in global health and education development economics — from vaccination campaign evaluations to conditional cash transfer programmes — reflecting the same preference for the strongest possible counterfactual described in Section 1, just outside the agricultural-economics tradition that produced the economic surplus method.
Frequently Asked Questions
- Schultz, T. W. (1953). The Economic Organization of Agriculture. New York: McGraw-Hill.
- Griliches, Z. (1958). Research costs and social returns: Hybrid corn and related innovations. Journal of Political Economy, 66(5), 419–431.
- Akino, M., & Hayami, Y. (1975). Efficiency and equity in public research: Rice breeding in Japan's economic development. American Journal of Agricultural Economics, 57(1), 1–10.
- Alston, J. M., Norton, G. W., & Pardey, P. G. (1998). Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting. Wallingford: CABI.
- Evenson, R. E. (2001). Economic impacts of agricultural research and extension. In B. Gardner & G. Rausser (Eds.), Handbook of Agricultural Economics (Vol. 1). Amsterdam: Elsevier Science. [Cross-referenced — see Topic 31.]