- Bioregional Assessment Program
- Hunter subregion
- 2.6.1 Surface water numerical modelling for the Hunter subregion
- 220.127.116.11 Uncertainty
- 18.104.22.168.2 Qualitative uncertainty analysis
The major assumptions and model choices underpinning the Hunter subregion surface water model are listed in Table 10. The goal of the qualitative uncertainty analysis is to provide a non-technical overview of the model assumptions, and their justification and effect on predictions, as judged by the modelling team. This will also assist in an open and transparent review of the modelling.
Each assumption in Table 10 is rated against three attributes (data, resources, technical) and their effect on predictions.
- The data column is the degree to which the question ‘If more or different data were available, would this assumption/choice still have been made?’ would be answered positively. A low rating indicates the assumption is not influenced by data availability, while a high rating indicates that this choice would be revisited if more data were available.
- The resources rating reflects the extent to which resources available for the modelling, such as computing resources, personnel and time, influenced this assumption or model choice. Again, a low rating indicates the same assumption would have been made with unlimited resources, while a high rating indicates the assumption is driven by resource constraints.
- The technical rating reflects the extent to which the assumption is influenced by technical and computational issues. A high rating flags assumptions and model choices that are predominantly driven by computational or technical limitations of the model code. These include issues related to spatial and temporal resolution of the models.
The most important rating relates to the effect the assumption or model choice has on the predictions. This is a qualitative assessment by the modelling team of the extent to which a model choice will affect the model predictions, with low indicating a minimal effect and high a large effect.
Table 10 Qualitative uncertainty analysis as used for the Hunter subregion surface water model
A discussion of each of the assumptions, including the rationale for the scoring, follows.
Selection of calibration catchments
The parameters that control the transformation of rainfall into streamflow are adjusted based on a comparison of observed and simulated historical streamflow. Only a limited number of the model nodes have historical streamflow. To calibrate the surface water model, a number of catchments are selected outside the Hunter subregion. The parameter combinations that achieve an acceptable agreement with observed flows are deemed acceptable for all catchments in the subregion.
The selection of calibration catchments is therefore almost solely based on data availability, which results in a medium score for this criterion. As it is technically trivial to include more calibration catchments in the calibration procedure and as it would not appreciably change the computing time required, both the resources and technical columns have a low rating.
The regionalisation methodology is valid as long as the selected catchments for calibration are not substantially incompatible with those in the prediction domain in terms of size, climate, land use, topography, geology and geomorphology. The majority of these assumptions can be considered valid (see Section 22.214.171.124) and the overall effect on the predictions is therefore deemed to be low.
High-flow and low-flow objective function
AWRA-L simulates daily streamflow. High-streamflow and low-streamflow conditions are governed by different aspects of the hydrological system and it is difficult for any streamflow model to find parameter sets that are able to adequately simulate both extremes of the hydrograph. In recognition of this issue, two objective functions are chosen, one tailored to medium and high flows and another one tailored to low flows.
Even with more calibration catchments and more time available for calibration, a high-flow and low-flow objective function would still be necessary to find parameter sets suited to simulate different aspects of the hydrograph. Data and resources are therefore scored low, while the technical criterion is scored high.
The high-streamflow objective function is a weighted sum of the Nash–Sutcliffe efficiency (E) and the bias. The former is most sensitive to differences in simulated and observed daily and monthly streamflow, while the latter is most affected by the discrepancy between long-term observed and simulated streamflow. The weighting of both components represents the trade-off between simulating short-term and long-term streamflow behaviour. It also reflects the fact that some parameters are more sensitive to daily behaviour and some are more sensitive to long-term hydrology.
The low-streamflow objective function is achieved by transforming the observed and simulated streamflow through a Box-Cox transformation (see Section 126.96.36.199). By this transformation, a small number of large discrepancies in high streamflow will have less prominence in the objective function than a large number of small discrepancies in low streamflow. Like the high-streamflow objective function, the low-streamflow objective function consists of two components, the E transformed by a Box-Cox power of 0.1 and bias, which again represent the trade-off between short-term and long-term accuracy.
The choice of the weights between both terms in both objective functions is based on the experience of the modelling team (Viney et al., 2009). The choice is not constrained by data, technical issues or available resources. While different choices of the weights will result in a different set of optimised parameter values, experience in the Water Information Research and Development Alliance (WIRADA) project, in which the AWRA-L is calibrated on a continental scale, has shown the calibration to be fairly robust against the weights in the objective function (Vaze et al., 2013).
While the selection of objective function and its weights is a crucial step in the surface water modelling process, the overall effect on the predictions is marginal through the uncertainty analysis, hence the low rating.
Selection of goodness-of-fit function for each hydrological response variable
The goodness-of-fit function for each hydrological response variable for uncertainty analysis has a very similar role to the objective function in calibration. Where the calibration focuses on identifying a single parameter set that provides an overall good fit between observed and simulated values, the uncertainty analysis aims to select an ensemble of parameter combinations that are best suited to make the chosen prediction.
Within the context of the bioregional assessment (BA), the calibration aims to provide a parameter set that performs well at a daily resolution, while the uncertainty analysis focuses on specific aspects of the yearly hydrograph.
The goodness-of-fit function is tailored to each hydrological response variable and averaged over a number of selected catchments that contribute to flow in the Hunter subregion modelling domain. This ensures parameter combinations are chosen that are able to simulate the specific part of the hydrograph relevant to the hydrological response variable, at a local scale.
Like the objective function selection, the choice of summary statistic is primarily guided by the predictions and to a much lesser extent by the available data, technical issues or resources. This is the reason for the low rating for these attributes.
The impact on the predictions is deemed low as it is an unbiased estimate of model mismatch and because it summarises the same aspect of the hydrograph as is needed for the prediction.
Selection of acceptance threshold for uncertainty analysis
The acceptance threshold ideally is independently defined based on an analysis of the system (see companion submethodology M09 (as listed in Table 1) for propagating uncertainty through models (Peeters et al., 2016)). For the surface water hydrological response variables, such an independent threshold definition can be based on the observation uncertainty, which depends on an analysis of the rating curves for each observation gauging station as well as at the model nodes. There are limited rating curve data available, hence the medium rating. Even if this information were available, the operational constraints within the BA prevent such a detailed analysis – although it is technically feasible. The resources column therefore receives a high rating while the technical column receives a medium rating.
The choice of setting the acceptance threshold equal to the 90th percentile of the summary statistic for a particular hydrological response variable (i.e. selecting the best 10% of replicates) is a subjective decision made by the modelling team. By varying this threshold through a trial-and-error procedure in the testing phase of the uncertainty analysis methodology, the Assessment team learned that this threshold is an acceptable trade-off between guaranteeing enough prediction samples and overall good model performance. While relaxing the threshold may lead to larger uncertainty intervals for the predictions, the median predicted values are considered robust to this change. A formal test of this hypothesis has not yet been carried out. The effect on predictions is therefore scored medium.
Interaction with the groundwater model
The coupling between the outputs of the groundwater model and the surface water models, described in the model sequence section (Section 188.8.131.52), represents a pragmatic solution to account for surface water – groundwater interactions at a regional scale. Even if a suitable algorithm for integrated coupling of fluxes between the surface water and groundwater models were available, the differences in spatial and temporal resolution would require non-trivial upscaling and downscaling of spatio-temporal distributions of fluxes. For these reasons and also for practical reasons related to run times and computational storage issues, the modelling methodology for the Hunter subregion involves a one-directional feed of changes in the groundwater flux to streams from the groundwater model, rather than a fully coupled implementation. Thus the rating for the technical attribute is high.
The data and resources columns are rated medium because even if it were technically feasible to fully integrate the models, the implementation would be constrained by the available data and the operational constraints. In an integrated model, a simulation would likely involve multiple iterations between the groundwater and stream components and increase the computational load significantly.
The overall effect on predictions is rated as medium. While changes in baseflow are generally small compared to runoff, the low flow regime can be sensitive to them.
Implementation of the coal resource development pathway
The CRDP is implemented through the interaction with the groundwater models and by removing the fraction of runoff in the catchment that is intercepted by the mine footprint from the total catchment runoff. The key choices that are made in implementing the CRDP are (i) determining which mining developments are included, and (ii) deciding on the spatial and temporal development of their hydrological footprints.
In catchments in which the mine footprint is only a small fraction of the total area of the catchment, the precise delineation of the spatial extent of the mine footprint is not crucial to the predictions. In catchments in which the footprint is a sizeable fraction, accurate delineation of the mine footprint becomes very important.
Similarly, the temporal evolution of mine footprints is crucial as it will determine how long the catchment will be affected. This is especially relevant for the post-mining rehabilitation of mine sites, when it becomes possible again for runoff generated within the mine footprint to reach the streams.
In the Hunter subregion, the accuracy of the mine footprints represented in the model largely reflects the accuracy of the mine footprints published or provided by the mine proponents. This therefore is one of the crucial aspects of the surface water model as it potentially has a high impact on predictions and it is driven by data availability rather than availability of resources or technical issues. The data attribute is therefore scored high, while the resources and technical columns are scored low. The effect on predictions is scored high.
Streamflow routing is taken into account in the Hunter AWRA-R as the Hunter River is a large system and routing can lag flows by several days. Streamflow routing is not taken into account in the Macquarie-Tuggerah lakes basin since it is unregulated and sufficiently small that lags in streamflow due to routing will be within a daily time step. The effect on the prediction of not incorporating routing is therefore minimal. Given the small potential for impact, resourcing the development of a river-routing model for this region was not warranted. All attributes are rated low as it is technically feasible and within the operational constraints of the BA to carry out streamflow routing. Doing so would only minimally affect the predictions.
Product Finalisation date
- 184.108.40.206 Methods
- 220.127.116.11 Review of existing models
- 18.104.22.168 Model development
- 22.214.171.124.1 Spatial and temporal dimensions
- 126.96.36.199.2 Location of model nodes
- 188.8.131.52.3 Choice of seasonal scaling factors for climate trend
- 184.108.40.206.4 Representing the hydrological changes from mining
- 220.127.116.11.5 Modelling river management
- 18.104.22.168.6 Rules to simulate industry water discharge
- 22.214.171.124 Calibration
- 126.96.36.199 Uncertainty
- 188.8.131.52 Prediction
- Currency of scientific results
- Contributors to the Technical Programme
- About this technical product