- Home
- Assessments
- Bioregional Assessment Program
- Namoi subregion
- 2.6.1 Surface water numerical modelling for the Namoi subregion
- 2.6.1.4 Calibration
- 2.6.1.4.1 Australian Water Resources Assessment landscape model
Data
Data needed for calibration of the Australian Water Resources Assessment (AWRA) landscape model (AWRA-L) include climate data and streamflow. The calibration period used for AWRA-L model development is from 1983 to 2012, which includes both wet and dry periods.
AWRA-L is run using gridded data of maximum temperature, minimum temperature, incoming solar radiation and precipitation. Daily grids (cell resolution of 0.05 x 0.05 degrees; ~5 x 5 km) of these variables are generated for the Australian continent by the Bureau of Meteorology (Dataset 1). Details of this dataset are provided in Section 2.1.1 of companion product 2.1-2.2 for the Namoi subregion (Aryal et al., 2018).
Daily stage and streamflow data from 11 streamflow gauging stations with unregulated catchments located in and near the Namoi subregion were used to calibrate AWRA-L. These gauging stations and the catchment boundaries are shown in Figure 8. Of the 11 stations, six are located within the modelling domain (Section 2.6.1.3) and five are located in the adjacent basins. Site details are summarised in Section 2.1.4 of companion product 2.1-2.2 for the Namoi subregion (Aryal et al., 2018). Observed daily mean streamflow data for the above gauges for 1983 to 2012 were obtained from the NSW Office of Water (Dataset 3).
Criteria for selecting the calibration catchments include that they:
- have catchment areas greater than 50 km2
- have long-term measurements (more than 20 years from 1983)
- are currently not impacted by coal mining or coal seam gas (CSG) extraction or other major extractive industries
- have no significant flow regulation (e.g. dams)
- are not nested (i.e. not directly upstream or downstream of another selected gauge)
- are located within or close to the Namoi subregion and have similar catchment sizes and climate regimes.
Catchment boundaries for the 11 calibration catchments were delineated using the Australian Hydrological Geospatial Fabric (Geofabric) (Bureau of Meteorology, Dataset 1).
Figure 8 Location of the 11 catchments used in AWRA-L model calibration for the Namoi subregion
AWRA-L = Australian Water Resources Assessment landscape model
Data: Bureau of Meteorology (Dataset 1), Bioregional Assessment Programme (Dataset 2)
Calibration evaluation metrics
As per companion submethodology M06 (as listed in Table 1) for surface water modelling (Viney, 2016), two regional model calibrations were undertaken: one biased towards predicting high-streamflow behaviour, and the second towards predicting low-streamflow behaviour. Three metrics were used to evaluate model performance:
- Daily efficiency (Ed),
- Model bias (B), the cumulative prediction error divided by the sum of observations, which gives an indication of the model’s overall tendency towards underprediction or overprediction
- F value, which seeks to combine efficiency and bias into a single metric. The high-streamflow calibration uses a version of F given by F1 = (Ed(1.0) + Em)/2 - 5|ln(1 + B)|2.5,where Ed(1.0) is the efficiency of daily streamflow (without transformation, or with a Box-Cox lambda value of 1.0) (Box and Cox, 1964), Em is the efficiency of the monthly predictions assessed against monthly observations, and B is the bias. The low-streamflow calibration uses a version of F given by F2 = Ed(0.1) - 5|ln(1 + B)|2.5, where Ed(0.1) is the efficiency of daily streamflow transformed with a Box-Cox lambda value of 0.1.
Model calibration results
Figure 9 and Table 7 summarise results of the two model calibrations for the 11 calibration catchments in terms of the three performance metrics. The high-streamflow calibration yields a reasonable Nash–Sutcliffe efficiency of daily streamflow, indicated by a median Ed(1.0) of 0.59 and 25th and 75th percentile values of 0.54 to 0.65, respectively. Overall model bias is low with a median bias of 0.04, but the interquartile range from negative to positive values indicates variability across the 11 catchments with considerable overestimation in some catchments. The high-streamflow calibration yields a median F1 of 0.66. The Ed(1.0) value is greater than 0.3 for 9 of the 11 catchments, with negative values (suggesting poor simulation) for one catchment: –0.73 at gauging station 419029 on Halls Creek (tributary of the Namoi River), with a prediction bias of 0.65.
The low-streamflow calibration is evaluated against the daily streamflow data transformed with a Box-Cox lambda value of 0.1. The low-streamflow calibration yields a lower efficiency than the high-streamflow calibration, indicated by a median Ed(0.1) of 0.36. The Ed(0.1) is greater than 0.3 for 7 of the 11 catchments and there are three gauging stations with negative Ed(0.1) values. The median bias of –0.09 indicates an overall tendency for underprediction by the low-streamflow calibration, although the interquartile range (–0.26 to 0.16) indicates less variability than for the high-streamflow calibration.
The 11 calibration catchments cover a wide range of climate and topographic conditions, with mean annual streamflow ranging from 11.2 GL/year at catchment 418033 (Bakers Creek at Bundarra) to 50.7 GL/year at catchment 420017 (Castlereagh River at Hidden Valley) (Table 7). This suggests that AWRA-L can predict streamflow variability reasonably in the Namoi subregion where climate conditions vary widely.
Figure 9 Summary of two AWRA-L model calibrations using 11 catchments for the Namoi subregion
AWRA-L = Australian Water Resources Assessment landscape model
In each boxplot, the bottom, middle and top of the box are the 25th, 50th and 75th percentiles, and the bottom and top whiskers are the 10th and 90th percentiles. F1 is the F value for high-streamflow calibration; F2 is the F value for the low-streamflow calibration; Ed(1.0) is the daily efficiency with a Box-Cox lambda value of 1.0; Ed(0.1) is the daily efficiency with a Box-Cox lambda value of 0.1; B is model bias.
Data: Bioregional Assessment Programme (Dataset 2)
Table 7 Calibration statistics for the 11 AWRA-L calibration catchments
AWRA-L = Australian Water Resources Assessment landscape model; F1 = F value for high-streamflow calibration; F2 = F value for low-streamflow calibration (see companion submethodology M06 (as listed in Table 1) for surface water modelling (Viney, 2016)); Ed(1.0) = daily efficiency with a Box-Cox lambda value of 1.0; Ed(0.1) = daily efficiency with a Box-Cox lambda value of 0.1
Data: Bioregional Assessment Programme (Dataset 2)
Nine hydrological response variables (Table 8) have been chosen to characterise the hydrological changes of coal resource development on water resources. Figure 10 shows the model bias in the prediction of the hydrological response variables based on the high-streamflow and low-streamflow calibrations. Overall, the low-streamflow calibration outperforms the high-streamflow calibration across eight of the nine hydrological response variables with smaller median model biases and narrower interquartile ranges for the 11 catchments. Only for annual flow (AF) is the median bias of high-streamflow calibration marginally better than the low-streamflow calibration. Both calibrations tend to over estimate the interquartile range in daily flow (IQR). As many of the calibration catchments are intermittent or ephemeral, the metric describing the 1st percentile of daily flow rate (P01) is typically zero and is therefore difficult to predict with both calibration schemes tending to over predict. Similarly, neither calibration predicts zero-flow days (ZFD) well, although the low-streamflow calibration slightly outperforms the high-streamflow calibration on a catchment-by-catchment basis. Box plots of both these hydrological response variables are truncated in Figure 10. The high-streamflow calibration generally over estimates the daily flow rate at the 99th percentile (P99), whereas the low-streamflow calibration generally under estimates P99, but the latter has an overall smaller bias. Both calibrations under estimate the low-streamflow metrics, low-flow days (LFD), low-flow spells (LFS) and length of low-flow spell (LLFS) similarly. Overall the biases of hydrological response variables are less for low-streamflow calibration than for the high-streamflow calibration.
Table 8 Details of the nine hydrological response variables
Hydrological response variable |
Definition |
Unit |
---|---|---|
P01 |
The daily flow rate at the 1st percentile (ML/day). This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
ML/day |
zero-flow days (ZFD) |
The number of zero-flow days per year. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
Days |
low-flow days (LFD) |
The number of low-flow days per year. This is the maximum value of the 90-year period (from 2013 to 2102). The threshold for low-flow days is the 10th percentile from the simulated 90-year period (2013 to 2102). |
Days |
low-flow spells (LFS) |
The number of low-flow spells per year. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
Times/year |
length of low-flow spell (LLFS) |
The length (days) of the longest low-flow spell each year. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
Days |
P99 |
The daily flow rate at the 99th percentile (ML/day). This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
ML/day |
high-flow days (FD) |
The number of high-flow days per year. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). The threshold for high-flow days is the 90th percentile from the simulated 90-year period (2013 to 2102). In some early products, this was referred to as ‘flood days’. |
Days |
annual flow (AF) |
The volume of water that discharges past a specific point in a stream in a year, commonly measured in GL/year.. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
GL/year |
interquartile range (IQR) |
The interquartile range in daily flow (ML/day); that is, the difference between the daily flow rate at the 75th percentile and at the 25th percentile. This is typically reported as the maximum change due to additional coal resource development over the 90-year period (from 2013 to 2102). |
ML/day |
In each boxplot, the left, middle and right of the box are the 25th, 50th and 75th percentiles, and the left and right whiskers are the 10th and 90th percentiles.
Shortened forms of hydrological response variables are defined in Table 8.
Data: Bioregional Assessment Programme (Dataset 2)
It is noted that the parameter sets consisting of a single value for each parameter obtained from the regional model calibrations are not applied directly to the simulation and uncertainty analysis. However, the 3000 stochastic parameter sets used in simulation are drawn from distributions that are biased towards the values obtained in the two calibrations. The calibration parameter sets are also taken as reference values to evaluate performance of the best 10% of parameter sets (i.e. 300 model runs) selected for each hydrological response variable.
Implications for model predictions
Results from the regional model calibrations (Table 7 and Figure 9) suggest that AWRA-L performs reasonably well in estimating high streamflow and low streamflow in the Namoi subregion.
It is noted that when the model is calibrated against observations from 11 streamflow gauges it does not generate a uniform model performance. Though the model performs well overall, it performs poorly in some catchments and does not estimate the suite of hydrological response variables equally effectively. Both the high-streamflow and low-streamflow model calibrations generate a poor overall model performance for gauge 419029 (Halls Creek) and gauge 420014 (Magometon Creek), both with Ed(1.0) lower than zero and large absolute biases (Table 7). Gauge 419029 has the largest percentage of daily streamflow data tagged as unverified implying a less reliable data. Gauge 419051 (Maules Creek) lies on the subregion boundary with the Hunter-Mooki Thrust Fault System passing through its catchment (Figure 8). This fault-line may be interfering with the surface flow, causing leakage through it which may result in the model highly over estimating streamflow in both calibrations. Note that distribution of observed runoff coefficients showed that the median runoff coefficient of gauge 419051 lies close to the 10th percentile indicating a comparatively very low runoff coefficient. Also, the annual runoff depth of gauge 419051 is much less than nearby gauges, especially 418027 (Table 7).
A key characteristic of a regional model calibration approach (i.e. a single simultaneous calibration against observations from multiple catchments) is that, unlike with local calibration (i.e. separate calibration for each individual catchment), there is little degradation from model calibration to model prediction (Viney et al., 2014). Except for the gauges discussed above which are affected by data quality or geography, the biases at the majority of individual stream gauging sites are in the range of –30% to +40%. Given the regionalisation methodology applied here, it is reasonable to assume that prediction biases in ungauged parts of the subregion will be similar. This provides confidence when applying AWRA-L to each model node where there are no streamflow observations.
Although not directly calibrated towards any of the hydrological response variables, an assessment of how well the two calibration sets predict the hydrological response variables serves as a further assessment of model performance. In general, one or other of the two calibration sets provides predictions of the high-streamflow hydrological response variables with little bias, although (apart from P01) both tend to under predict the low-streamflow hydrological response variables. This suggests that less confidence may be ascribed to the prediction of the low-streamflow hydrological response variables in Section 2.6.1.6 than to the prediction of the other variables.