Page 15 - Module_8_en

Judgment Module

Summative evaluation, accreditation, quality control and audits are examples of judgment-intended 8
evaluations. They follow a deductive method by setting clear criteria and standards with which to judge

performance, often using quantitative measures. Judgment-intended evaluation often is commissioned
by external parties (e.g., donors), and typically is performed by external evaluators. Such evaluation
could increase the credibility of an IEA process, given its impartiality and objectivity.

Improvement Abu Dhabi, UAE

Formative evaluation, empowerment evaluation (Fettermann 1996) and outcome mapping (Carden
2001) are examples of improvement or development-minded evaluations. The central intent of this
type of evaluation is making things better over time. Improvement evaluation is inductive, posing open-

ended evaluation questions. Evaluators are often internal; the participants, including some of the primary
users, conduct the evaluation. SWOT analysis (Strength, Weaknesses, Opportunities and Threats), TQM
(Total Quality Management), EMS (Environmental Management Systems) and ISO 14001 employ the
evaluation of improvement approach. Improvement-intended evaluation could increase the legitimacy
of the IEA process, given its users’ perspective.

Such evaluations often are applied to cyclical activities, like the IEA process, where performance
improvement is expected over time. This improvement can involve change in behaviour (e.g., improved
communication) or change in the state of the environment (e.g., improved water quality).

Outcome mapping (Carden 2001) focuses on changes in human behaviour, values, skills and knowledge,
and acknowledges the complexity and the life cycle of the outcome. Some outcomes (e.g., institutional
transformations) need decades to fully develop.

Knowledge creation

Knowledge-oriented evaluation - exemplified by action research, case studies, lessons learned and
policy recommendations - has been gaining attention recently because of its capability to generate
innovative ideas and deep insights for the intended users. Emerging knowledge can improve a known
process, and break new ground. Evaluators can be both internal and external, and the intended users
are actively engaged all along. Knowledge-oriented evaluation can increase the saliency of the process,
given its potential to generate new knowledge that the user needs for a pressing decision (Bernd
Siebenhüner 2005).

In reality, elements of all three categories can be found in an evaluation. For practical purposes it is
important to select and focus on one dominant approach from the onset. As an IEA process intends

IEA Training Manual Workshop for the National Reporting Toolkit (NRT) 13