ClimateChange Assessment Guide.pdf - University of Waterloo

Guide for Assessment of Hydrologic Effects of Climate Change in Ontario58Guidance:A sensitivity analysis of the most important modelprocess parameters should be conducted beforecalibration as this analysis guides the parameteradjustment process.Continuous simulation models are evaluated bycomparing summary statistics from simulated andobserved time series of streamflow at key sites. Somemeasure of goodness of fit between the simulated andobserved values such as a sum of squares of residuals isused. The Nash-Sutcliffe Coefficient (ASCE, 1993) is oneexample of a sum of squares of residuals coefficient thatmeasures the level of agreement between the simulatedand observed time series. Other important comparisonsin hydrologic model calibration include total volumes ofstreamflow in event, daily, monthly and annual terms andflow frequency distributions over the simulation period.Specific statistics can also be compared, such as low flowstatistics (i.e., 7Q 2, 7Q 20), average flow rates.The objective of calibration is to conduct severalmodel tests to evaluate the hydrologic model in allaspects, including the ability of the model to simulateaverage conditions, seasonal events (i.e., snowmeltand accumulation) and extreme events (i.e., droughts).The model test procedure should, therefore, examinecumulative metrics (i.e., flow volumes), extreme statistics(i.e., flow frequency distribution) and overall agreement.While event based models are not preferred for waterbalance studies, they may have application in somestudies. Calibration of event based models is limited tostorm events and thus model performance is evaluatedin terms of event runoff volume and timing. Generally, aset of storm events of differing magnitudes is used formodel testing. If the model is intended for a 12 monthsimulation, the selected events should encompass allseasons and include minor as well as major storms.Event models are not capable of assessing the effectsof sequences of events on the hydrologic outcomesand therefore cannot be used to assess climatechange effects if changes are expected in storm eventsequences.Model validation is conducted to confirm that the modelparameters are well calibrated and robust; therefore,validation must be conducted under different conditionsthan calibration. This implies a different time periodand possibly changes in other conditions in the studyarea (i.e., land use). A robust model performs well underboth calibration and validation conditions. Validationperiods may range from selected events to several yearsof continuous streamflow. Model evaluation metrics andprocedures are the same as for the calibration phase.Rigid criteria for determining the success of thecalibration/validation processes are not generally used.Rather judgement is applied to the calibration/validationresults while considering the model objectives and theneed for accuracy. A model that simulates flow volumeswithin 10%-15% of observed volumes on a long-termbasis (i.e., monthly, seasonally or annually), is generallyconsidered reasonably accurate in terms of overall waterbalance. Simulated storm event total flow volumeswithin 10% of the observed are considered reasonable.When comparing flow frequency distributions, particularattention is given to the median flow rate and low flowstatistics. The modeller should try to eliminate any zonesin the flow duration curves where there is significantdeparture between the simulated and observeddistributions as these indicate a potential systematicweakness in the model.Guidance:The model calibration and validation processshould be well documented as these resultsreflect upon the validity of model simulationsand conclusions drawn from the study at allsubsequent stages.6.3.3 Determine Errors and ConfidenceOne of the most difficult concepts in hydrologicmodelling is the expression of confidence in simulationresults and the identification of the sources of error.This is difficult because the sources of error cannot beprecisely quantified or described; and the overall impactof the errors is difficult to quantify because of the largenumber of possible sources of error and the interactionbetween them.