Page 19 - Module_8_en

3 Attributes, Framework and Measures Module

Section 3 covers the attributes that will indicate the effectiveness of the IEA process and selecting an 8
evaluation framework. Then it will discuss formulating the key evaluation questions, and measures that
will help you collect data for monitoring and evaluation.

3.1 Attributes of Effective Assessments

This framework takes a look at key attributes that enhance the IEA report’s effectiveness in influencing Abu Dhabi, UAE
policy-makers. The notions of saliency, credibility and legitimacy - as key attributes of effective assessments
- arise from earlier academic research that focused on better understanding the factors that determine
the effectiveness of assessments (Box 2; Figure 3).

BOX 2 Attributes of Effective Assessments

Through a five-year consultative process involving hundreds of professional evaluators
internationally, the American Evaluation Association identified four criteria for effective
evaluation: utility, feasibility, propriety and accuracy (Patton 1997). A couple of years later, the

Social Learning Group’s international research team arrived at a similar conclusion from
a different point of departure, namely studying what makes environmental assessments
effective, and what makes them utilized. The Social Learning Group found that the user

(i.e. policy-maker’s) perception of the assessment’s saliency, credibility and legitimacy was
critical (Farrell and J?ger 2005).

The saliency-credibility-legitimacy attribute triad acknowledges that the process is subject to political
interests. We do not suggest that the process should bend to those interests, but emphasize the need
to attract political attention when legitimacy and credibility are not convincing enough on their own
merits. It also implies that without credibility and legitimacy, political saliency is not enough to attract
and maintain attention.

The assessment of stratospheric ozone depletion is a good example, because it was perceived by
policy-makers as salient, credible and legitimate:

IEA Training Manual Workshop for the National Reporting Toolkit (NRT) 17