Method – Multi-criteria Analysis

Description

Multi-criteria analysis (MCA) is a framework able to deal with complex decision problems, to consider multiple different type of objective and criteria, to process various types of information, both objectives (adaptation option performance) and subjective (stakeholders´ preferences), and to integrate all into the decision-making process in a systematic and constructive way .

MCA allows assessment of options against different criteria, as was used in the preparation of National Adaptation Programmes of Actions

Quantification and other systematic approaches to select options have many virtues. However, they also have limitations. For instance, most of these methodologies do not account for a range of critical factors such as leadership, institutions, resources, and barriers.

MCA has many advantages over informal judgement unsupported by analysis:

  • It is open and explicit.
  • The choice of objectives and criteria that any decision-making group may choose are open to analysis and to change if they are felt to be inappropriate.
  • Scores and weights, when used, are also explicit and are developed according to established techniques.

MCA evaluation approaches have been applied increasingly the last decades for climate policy evaluation in various decision-making contexts: in integrated assessment of climate policy (Bell et al., 2003); in adaptation option analysis; and climate-change impact assessment and adaptation planning in water resources or water-related integrative sustainability assessment.

It is crucial to involve the stakeholders in the criteria definition and weighting steps. Stakeholders’ objectives and policy priorities should be taken into account and even get incorporated into the decision-making process in a structured, systematic and transparent way. Criteria weighting elicitation techniques have been developed within the framework of MCA to integrate information related to the stakeholders’ preferences into the decision-making process. This step requires a stakeholder dialogue to set the criteria.

Use in decision framework

Select adaptation approaches
Assess adaptation options

Input

It can address problems marked by various conflicting stakeholder to obtain the most satisfactory possible option. However, in that case for an optimal result the involvement of all relevant stakeholders is needed. The option evaluation can be quantitative or qualitative. If quantitative analysis is selected, information to evaluate all the option based on the criteria needs to be available.

Outline

The main procedure of the MCA consists of 6 steps (Figure 1).

main-mca-steps

Figure 1. The main steps of a multi-criteria analysis and the involvement of the stakeholders.

MCA facilitates the active engagement of relevant stakeholders through the process of criteria selection (step 2) and weighting (step 5).

Identifying criteria

The purpose of this step is to identify criteria for assessing the consequences of each option. Criteria should be specific and measurable objectives. Depending the data availability, the link with the objectives and preferences of the stakeholders a set of criteria will be selected.

The process to select and validate evaluation criteria consists of a literature review, screening of initially selected indicators and stakeholders’ validation (workshop with local stakeholders). The selected criteria need to meet certain conditions: completeness, Redundancy, Operationally, Mutual independence of preferences, Double counting, and Size Impacts occurring over time among other.

Quantification

The scoring can be quantitative or qualitative and different types of scales are valid. The source of information to support the scoring need to be identified and used to score the criteria. In case different scales are used for the selected criteria, a normalization step is required after scoring. Last, the consistency of the scores on each criterion should be checked. In the following lines some of the criteria a further explained:

  • The environmental performance of the adaptation options: Determine the environmental performance (effectiveness) of the adaptation options. Determine the amount of risk (hazard, exposure and/or vulnerability) the option will reduce (this will determine how effective is the option), but also risks* introduced or increased by the measure.
  • Economic performance: Cost-benefit of each adaptation option, not only of direct costs and revenues, but also economic impact for the surrounding economy

* this risk is broader than the climate risk assessed and includes for example safety issues affected by the options (e.g. road safety when elevated to function as evacuation routes)

Normalisation

Values can be normalised between e.g. 0-1 so that a fair comparison can be done among the options.

Weighting

Assign weights for each of the criterion to reflect their relative importance to the decision. The weighting of criteria plays an important role in the ranking of adaptation options. The aim of this step is to select a weighting method and apply.

A selection of the weighting method need to be done. Then the methodology is applied in a workshop with stakeholders. Assigning weights to criteria highlights which criteria count most. Finally, a general group discussion follows and a final agreement on the weight is sought.

classification-weighting-methods

Figure 2. Classification of weighting methods.

Ranking

The options will be ranked according to the prioritisation result and the contribution of each criterion to the result will appear. If the prioritisation gives unexpected results (from the stakeholders ‘perspective), then a sensitivity analysis is recommended.

The obtained ranking gives an indication of which one is the preferred option and how much better this option is over the following one. The sensitivity analysis consists on meetings with stakeholders to understand the outcomes of the prioritization. A re-checking of the appraisal preferences or weights is done.

Output

A rank of options is obtained after the MCA process.

Experiences

Climact PRIO tool has been applied and compared to simpler tools such as ticking methods for prioritisation of options in the city of Paris with good results.