This study is based on work carried out by Jeroen P. van der Sluijs, Jose Potting et al., for the Dutch National Research Programme on Global Air Poluution and Climate Change. The work is published online as Report no 410 200 104 (2002): Uncertainty assessment of the IMAGE/TIMER B1 CO2 emissions scenario, using NUSAP methods. Link: www.nusap.net/workshop/report/finalrep.pdf.

Introduction

This project used the NUSAP methodology (Numeral Unit Spread Assessment Pedigree) to assess the qualitative and quantitative uncertainties present in the TIMER energy model used as part of the RIVM IMAGE Integrated Assessment model (a model used to help inform cost effective decision making in global scale climate change mitigation) Link: www.mnp.nl/en/themasites/image/index.html.. A large amount of information is available for NUSAP from the NUSAP.net web pages: link www.nusap.net.

Figure 1.1 (reproduced with kind permission of the authors) shows how the TIMER model fits into the broader IMAGE integrated assessment scheme.

From the outset, the authors clearly state all possible types of uncertainty that may be present within the TIMER model. These are:

• Inexactness - limits on the numerical precision of data.

• Unreliability - level of confidence in using well accepted methods.

• Value loading - perspectives and preferences for future developments.

• Ignorance - "we can not know what we do not know".

These components of uncertainty manifest in two key ways: (1) through the choice of model structure; and (2) through the selection of the model parameters.

Model intercomparison

To assess how model structure may contribute to uncertainty, the authors carried out a model intercomparison of the 6 models used by the IPCC for the Special Report on Emissions Scenarios (SRES) (Link: www.grida.no/climate/ipcc/emission). The authors found that in some cases the models used similar structural assumptions based on work that had come to be accepted as 'standard'; in other cases, the models treated some fuels (for example renuwables and nuclear) differently.

The important message here is that each group of model builders has to make a series of assumptions about the underlying 'real world' process and this set of assumptions generates uncertainty in the results from the model. A thourough modelling analysis (as carried out by the authors) should consider these issues.

However, every modeller knows that at some point he/she has to go ahead and define a model structure. At this point the second manifestation of uncertainty comes into play: parameter specification.

Types of variables

The TIMER program generates a set of carbon emissions variables. These form the state variables of the model and are either selected as outputs or moved and transformed within the model.

The model requires three types of parameters:

• Structural variables - these transform the state variables in some way, they can be fixed or time-varying and are effectively treated as input variables by the model.

• Initialisation values - these are the values of the state variables at the first time step of a model run.

• Input variables - these values operate at the system boundary and are the time-series of driving forces for the model.

The TIMER model has a total of some 300 separate variables each potentially changing on every time step of a model run leading to ~ 160000 separate data points that need to be specified by the modeller!

Figure 2.1 (reproduced with kind permission of the authors) shows a small selection of the 300 variables specified in the computer code audit.

Sensitivity analysis

Each parameter is subject to varying degrees of uncertainty generated from the four categories defined above (Inexactness, Unreliability, Value loading and Ignorance); however, some variables have a more significant effect on the model output than others. The object of sensitivity analysis is to try and identify these key parameters. However, for anything other than the simplest of models, the number of model runs required to adequately cover the parameter space becomes prohibitive. To address this issue a number of schemes have been developed; the authors use one of these: the Morris method for sensitivity analysis (Morris 1991).

The application of a Morris sensitivity analysis, ascribes 'ellementary effects' to each variable expressed as a mu and sigma. mu provides an estimate of the ratio of the rate of change of the output to the rate of change of the variable, and sigma provides an indication of the level of interaction between the variable and other variables.

The sensitivity analysis performed on the TIMER model (performed in quite a complex way; see the original report) was able to identified a number of parameters on which the model output is sensitive. Interestingly, these coincided with the intuitive feelings of the model builders as to which parameters would turn out to be important.

Parameter strength

Central to the NUSAP approach is the idea that a model parameter can be qualified not only by a Numeral, its Unit and a Spread but that the parameter can also be Assessed by expert judgement and ascribed a Pedigree. The Pedigree provides an evaluation of the quantitative information describing how the parameter was produced. A pedigree matrix is used to derive the assessment.

The choice of the criteria and ranking within the pedigree matrix is selected according to the specific goals of the uncertainty analysis.

The results from the Morris sensitivity analysis were used to define the 40 most critical model parameters. These parameters were then explored by experts during guided workshops (elicitation workshops) using a number of pedigree matrices (similar to that shown in Table 1.1) and a scoring card for each variable (an example is shown in Table 2.1).

Table 2.1 (reproduced with the kind permission of the aurthors)

The results from the scoring cards were used to define a strength score for each of the 40 parameters. By combining the twin indices of sensitivity (criticality) and strength (inverse of weakness) it is possible to identify the potentially most problematic model parameters. Figure 3.1 illustrates the concept.

Figure 3.1 (reproduced with the kind permission of the aurthors) shows a descriptive graph to identify parameters that may cause concern. The y-axis shows criticality; this was the subject of the previous section i.e., it is identified using a suitable sensitivity analysis method. The x-axis is the subject of this section where the parameter values are assessed qualitatively using a pedigree score this is then used to define parameter weakness. Parameters that end up in zone 1 are a cause for concern as they are of both weak pedigree as well as being sensitive parameters within the model.

Results

Figure 4.1 Shows the combined results from the Morris sensitivity analysis and the strength assessments generated from the expert elicitation workshop. The authors refer to the plot as a diagnostic diagram.

The sensitivity axis measures the criticality of a parameters uncertainty (this is a quantitative measure). The numeric scale and key refer to a number a details described at length in the original report.

The strength axis displays the pedigree scores for each variable averaged over the selected pedigree criteria and the experts who ranked the variable. These values have been ascribed error bars representing one standard deviation around the average value defined by the sample of values generated by the group of experts.

The "danger zone" is represented by the top right hand quadrant of the graph where sensitivity is high and strength is low.

Figure 4.1(adapted from Jeroen P. van der Sluijs, Jose Potting et al., reproduced with permission) Shows the location of the selected TIMER variables within the plane describing criticality and strength (inverse of weakness in Figure 3.1)

Conclusions

The authors state that this case study was the first time that NUSAP methods had been applied to a model as complex as TIMER.

The key conclusions can be summarised as:

• NUSAP method can be adapted and applied to complex models in a meaningful way.

• NUSAP helps to discipline and guide the process of model quality control.

• The NUSAP methods helped to identify and major areas of concern and associated pitfalls in the complex mass of qualitative and quantitative uncertainties present in the TIMER modelling exercise.

• Model intercomparison provided some insight in the potential roles of model structure uncertainties.

• Sensitivity analysis combined with expert elicitation provided a means to identify and prioritise key areas of uncertainty.

• The diagnostic diagram (Figure 4.1) puts spread and strength together to inform prioritisation of key uncertainties.

References

• Jeroen P. van der Sluijs, Jose Potting1, James Risbey, Detlef van Vuuren, Bert de Vries, Arthur Beusen, Peter Heuberger, Serafin Corral Quintana, Silvio Funtowicz, Penny Kloprogge, David Nuijten, Arthur Petersen, Jerry Ravetz, 2002. Uncertainty assessment of the IMAGE/TIMER B1 CO2 emissions scenario, using the NUSAP method, Dutch National Research Programme on Global Air Pollution and Climate Change, Report no: 410 200 104 (2002).

• Morris, M. D., 1991, Factorial sampling plans for preliminary computational experiments. Technometrics, 33, 161–174.

• Risby, J.S., J.P. van der Sluijs and J. Ravetz, 2001. Protocol for Assessment of Uncertainty and Strength of Emission Data, Department of Science Technology and Society, Utrecht University, Report no: E-2001-10, 22 pp.

Useful links

• The original report link: www.nusap.net/workshop/report/finalrep.pdf.

• NUSAP.net link: www.nusap.net.

• The original Morris SA paper: link http://sensitivity-analysis.jrc.ec.europa.eu/tutorial/Morris.pdf.

• Sensitivity analysis tutorial site: link http://sensitivity-analysis.jrc.ec.europa.eu/tutorial/index.asp.

• Powerpoint presentation of the case study described here: link www.nusap.net/downloads/NOP15nov2001.ppt.

• Poster of the case study described here: link www.nusap.net/downloads/posterNUSAPTIMER.pdf.

Hello --Tue, 19 Aug 2008 00:57:06 +0100

I'm new here, just wanted to say hello and introduce myself.

Cheap viagra --Sun, 21 Dec 2008 16:32:52 +0000

Very nice site!

buy genericviagra cheap --Thu, 25 Jun 2009 02:59:29 +0100

Thanks