Skip to content. | Skip to navigation

FloodRiskNet

Personal tools

Log in
Join

You are here: Home → Catalogue of methods → Evaluating Model Performance and Conditioning of Uncertainties as Data are made available

FRMRC

Development of decision tree funded by RPA9 of the

Related terms

What's up ?

Be notified when a document is published in this folder or below.

Views

View
Edit

Actions


Most problems in flood risk management require the identification of appropriate model structures or parameter values. This is usually carried out in a model calibration exercise by comparing observed and predicted responses and attempting to improve some measure of model performance. Since all models are in error, if only because of observation measurement error, then the model calibration problem is intrinsically linked to the estimation of model uncertainties. In some problems that can be formulated in a statistical framework (such as fitting a particular flood frequency distribution or a water level / damage cost curve by regression analysis), there is a strong and clear relationship between error and uncertainty. However, it is often difficult to formulate a statistical model of the errors in calibrating a complex environmental model. There is, in hydrology and hydraulics, a very extensive literature on methods of model calibration (see Gupta et al., 2005), and a wide variety of performance measures have been used in model calibration and evaluation or confirmation. The importance of uncertainties in model calibration and confirmation has been recognised for a long time. Early in the history of distributed rainfall-runoff modelling (1976), for example, Stephenson and Freeze (Stephenson and Freeze, 1974) recognised that complete validation or verification of a model will always be impossible because of uncertainties about the inputs, initial conditions, and observations against which model performance will be compared. This question has also been discussed as a problem in modelling philosophy by Oreskes et al. (1994) and Beven(2002) . All models and calibrated parameter values are sensitive to the data used in calibration and evaluation (Beven, in press; Yapo et al., 1996). Gupta and others (Crawford and Linsley, 1966; Gupta and Sorooshian, 1985; Sorooshian et al., 1983) showed that the quality of data is sometimes more important than the length of the record. ‘Quality’ includes not only the accuracy of the input and output data available, but also the variety of hydrological and hydraulic behaviour. If a model is supposed to work not only for a restricted range of conditions, but also more globally, it is important that e.g. wet as well as dry periods are included in the record used for calibration and evaluation (Anderson and Bates, 1994; Gan and Biftu, 2003). This raises the question if a model, which is designed to predict floods should also performing well under low flow conditions. Ideally, a physically based model should perform under all conditions otherwise the adequacy of the physically based model can be questioned. However, this requires that the effective observational error]s allowed are also physically consistent and that the effective parameters are stationary over time. This argument will be briefly explored for a flood inundation model (for a more detailed discussion please see Pappenberger et al(Pappenberger et al., submitted)). It is for example often the case that when images of inundation are projected back onto the available geometry, then the interpolated inundation heights are not physically consistent, even allowing for the difficulties in finding the inundation boundary. In this case it may be very difficult to find a model that provides simulation results that are globally consistent with the observations. There are then four possible responses: investigate those regions of the flow domain where there are consistent anomalies between model predictions and range of observations; avoid using data we don’t believe or that is doubtful; introduce local parameters if there are particular local anomalies; make error bounds wider in some way where data is doubtful; and if none of the above can be done (because, for example, there is no reason to doubt anomalous data) then resort to local evaluations in assessing local uncertainties. == References == Anderson, M.G. and Bates, P.D., 1994. Evaluating Data Constraints on 2-Dimensional Finite-Element Models of Floodplain Flow. Catena, 22(1): 1-15. Beven, K.J., 2002. Towards a coherent philosophy for modelling the environment. Proceedings of the Royal Society of London Series A- Mathematical Physical and Engineering Sciences, 458(2026): 2465-2484. Beven, K.J., 2005. A Manifesto for the equifinality thesis. Journal of Hydrology, --(--): --. Crawford, N.H. and Linsley, R.K., 1966. Digital simulation in hydrology:Stanford watershed model IV. Technical report No. 30, Dept. Of Civil Eng., Stanford University. Gan, T.Y. and Biftu, G.F., 2003. Effects of model complexity and structure, parameter interactions and data on watersched modeling. In: Q. Duan, H. Gupta, S. Sorooshian, A.N. Rousseau and R. Turcotte (Editors), Advances in Calibration of Watershed Models. American Geophysical Union, Washington. Gupta, H.V., Beven, K.J. and Wagener, T., 2005. Model calibration and uncertainty estimation, Section 11.9. In: M.G. Anderson (Editor), Encyclopaedia of Hydrological Sciences. Wiley, Chichester. Gupta, V.K. and Sorooshian, S., 1985. The Relationship between Data and the Precision of Parameter Estimates of Hydrologic-Models. Journal of Hydrology, 81(1-2): 57-77. Oreskes, N., Shrader-Frechette, K. and Belitz, K., 1994. Verification, Validation, and Confirmation of Numerical-Models in the Earth-Sciences. Science, 263(5147): 641-646. Pappenberger, F., Matgen, P. and Beven, K., submitted. The influence of rating curve and structural uncertainty on flood inundation predictions. Advances in Water Resources. Sorooshian, S., Gupta, V.K. and Fulton, J.L., 1983. Evaluation of Maximum-Likelihood Parameter-Estimation Techniques for Conceptual Rainfall-Runoff Models - Influence of Calibration Data Variability and Length on Model Credibility. Water Resources Research, 19(1): 251-259. Stephenson, G.R. and Freeze, R.A., 1974. Mathematical simulation of subsurface flow contributions to snowmelt runoff, Reynolds Creek watershed, Idaho. Water Resource Research, 10(2): 284-294. Yapo, P.O., Gupta, H.V. and Sorooshian, S., 1996. Automatic calibration of conceptual rainfall-runoff models: Sensitivity to calibration data. Journal of Hydrology, 181(1-4): 23-48.
Optional change note:
Upload a file or image:
For editing help, see HelpPage .

Powered by Plone CMS, the Open Source Content Management System

This site conforms to the following standards: