- Bioregional Assessment Program
- Propagating uncertainty through models
- 1 Background and context
A bioregional assessment (BA) is a scientific analysis, providing a baseline level of information on the ecology, hydrology, geology and hydrogeology of a bioregion with explicit assessment of the potential impacts of coal resource development on water and water-dependent assets. The Methodology for bioregional assessments of the impacts of coal seam gas and coal mining development on water resources (the BA methodology; Barrett et al., 2013) provides the scientific and intellectual basis for undertaking BAs. It is further supported by a series of submethodologies of which this is one. Together, the submethodologies ensure consistency in approach across the BAs and document how the BA methodology has been implemented. Any deviations from the approach described in the BA methodology and submethodologies are to be noted in any technical products based upon its application.
A critical part of the BA is characterising the uncertainty in the results from the groundwater and surface water models. A robust and comprehensive uncertainty analysis underpins the understanding of risk and the likelihood that certain changes or impacts may occur. The uncertainty analysis is deeply integrated with the rest of the BA methodology and processes, particularly through the groundwater and surface water models, and the receptor impact modelling which depends on the outputs from those models and uncertainty analyses. This submethodology applies overarching principles outlined in the BA methodology to the specifics of quantifying uncertainty in numerical groundwater and surface water models that are reported in products 2.6.1 (surface water numerical modelling) and 2.6.2 (groundwater numerical modelling), respectively.
To provide context for this submethodology, Section 1.1 provides an overview of an entire BA from end to end, and the key concepts and relationships between activities within components. See Figure 3 for a simple diagram of the BA components. See Figure 4 for a more detailed diagram of the BA process that includes all the submethodologies, supporting workshops and technical products.
Figure 3 The components in a bioregional assessment
Figure 4 A bioregional assessment from end to end, showing the relationship between the workflow, technical products, submethodologies and workshops
CRDP = coal resource development pathway, HRVs = hydrological response variables, RIVs = receptor impact variables
1.1 A bioregional assessment from end to end
1.1.1 Component 1: Contextual information
In Component 1: Contextual information, the context for the BA is established and all the relevant information is assembled. This includes defining the extent of the subregion or bioregion, then compiling information about its ecology, hydrology, geology and hydrogeology, as well as water-dependent assets, coal resources and coal resource development.
An asset is an entity having value to the community and, for BA purposes, is associated with a subregion or bioregion. Technically, an asset is a store of value and may be managed and/or used to maintain and/or produce further value. Each asset will have many values associated with it and they can be measured from a range of perspectives; for example, the values of a wetland can be measured from ecological, sociocultural and economic perspectives.
A bioregion is a geographic land area within which coal seam gas (CSG) and/or coal mining developments are, or could, take place and for which BAs are conducted. A subregion is an identified area wholly contained within a bioregion that enables convenient presentation of outputs of a BA.
A water-dependent asset has a particular meaning for BAs; it is an asset potentially impacted, either positively or negatively, by changes in the groundwater and/or surface water regime due to coal resource development. Some assets are solely dependent on incident rainfall and will not be considered as water dependent if evidence does not support a linkage to groundwater or surface water.
The water-dependent asset register is a simple and authoritative listing of the assets within the preliminary assessment extent (PAE) that are potentially subject to water-related impacts. A PAE is the geographic area associated with a subregion or bioregion in which the potential water-related impact of coal resource development on assets is assessed. The compiling of the asset register is the first step to identifying and analysing potentially impacted assets.
Given the potential for very large numbers of assets within a subregion or bioregion, and the many possible ways that they could interact with the potential impacts, a landscape classification approach is used to group together areas to reduce complexity. For BA purposes, a landscape class is an ecosystem with characteristics that are expected to respond similarly to changes in groundwater and/or surface water due to coal resource development. Note that there is expected to be less heterogeneity in the response within a landscape class than between landscape classes. They are present on the landscape across the entire BA subregion or bioregion and their spatial coverage is exhaustive and non-overlapping. The rule set for defining the landscape classes is underpinned by an understanding of the ecology, hydrology (both surface water and groundwater), geology and hydrogeology of the subregion or bioregion.
Most assets can be assigned to one or more landscape classes. Different subregions and bioregions might use different landscape classes. Conceptually landscape classes can be considered as types of ecosystem assets, which are ecosystems that may provide benefits to humanity the landscape classes provide a systematic approach to linking ecosystem and hydrological characteristics with a wide range of BA-defined water-dependent assets including sociocultural and economic assets. Ecosystems are defined to include human ecosystems, such as rural and urban ecosystems.
Two potential futures are considered in BAs:
- baseline coal resource development (baseline), a future that includes all coal mines and coal seam gas (CSG) fields that are commercially producing as of December 2012
- coal resource development pathway (CRDP), a future that includes all coal mines and coal seam gas (CSG) fields that are in the baseline as well as those that are expected to begin commercial production after December 2012.
The difference in results between CRDP and baseline is the change that is primarily reported in a BA. This change is due to the additional coal resource development – all coal mines and coal seam gas (CSG) fields, including expansions of baseline operations, that are expected to begin commercial production after December 2012.
Highlighting the potential impacts due to the additional coal resource development, and the comparison of these futures, is the fundamental focus of a BA, as illustrated in Figure 5, with the baseline in the top half of the figure and the CRDP in the bottom half of the figure. In BAs, changes in hydrological response variables and particular receptor impact variables are compared at receptors (points in the landscape where water-related impacts on assets are assessed).
Hydrological response variables are defined as the hydrological characteristics of the system or landscape class that potentially change due to coal resource development (for example, drawdown or the annual streamflow volume). Receptor impact variables are the characteristics of the system that, according to the conceptual modelling, potentially change due to changes in hydrological response variables (for example, condition of the breeding habitat for a given species, or biomass of river red gums). Each landscape class and/or asset may be associated with one or more hydrological response variables and one or more receptor impact variables.
Figure 5 The difference in results under the baseline coal resource development (baseline) and coal resource development pathway (CRDP) provides the potential impacts due to the additional coal resource development (ACRD)
Figure 6 Hazard analysis using the Impact Modes and Effects Analysis
The italicised text is an example of a specified element in the Impact Modes and Effects Analysis. (a) In the simple case, an activity related to coal resource development directly causes a hydrological change which in turn causes an ecological change. The hazard is just the initial activity that directly leads to the effect (change in the quality and/or quantity of surface water or groundwater). (b) In the more complex case, an activity related to coal resource development initiates a chain of events. This chain of events, along with the stressor (for example, surface water (SW) flow and total suspended solids (TSS)), causes a hydrological change which in turn causes an ecological change. The hazard is the initial activity plus the subsequent chain of events that lead to the effect.
The hazards arising from coal resource development are assessed using Impact Modes and Effects Analysis (IMEA). A hazard is an event, or chain of events, that might result in an effect (change in the quality and/or quantity of surface water or groundwater). In turn, an impact (consequence) is a change resulting from prior events, at any stage in a chain of events or a causal pathway (see more on causal pathways below). An impact might be equivalent to an effect, or it might be a change resulting from those effects (for example, ecological changes that result from hydrological changes).
Using IMEA, the hazards are firstly identified for all the activities (impact causes) and components in each of the five life-cycle stages. For CSG operations the stages are exploration and appraisal, construction, production, work-over and decommissioning. For coal mines the stages are exploration and appraisal, development, production, closure and rehabilitation. The hazards are scored on the following basis, defined specifically for the purposes of the IMEA:
- severity score: the magnitude of the impact resulting from a hazard, which is scored so that an increase (or decrease) in score indicates an increase (or decrease) in the magnitude of the impact
- likelihood score: the annual probability of a hazard occurring, which is scored so that a one-step increase (or decrease) in score indicates a ten-fold increase (or decrease) in the probability of occurrence
- detection score: the expected time to discover a hazard, scored in such a way that a one-unit increase (or decrease) in score indicates a ten-fold increase (or decrease) in the expected time (measured in days) to discover it.
Impact modes and stressors are identified as they will help to define the causal pathways in Component 2: Model-data analysis. An impact mode is the manner in which a hazardous chain of events (initiated by an impact cause) could result in an effect (change in the quality and/or quantity of surface water or groundwater). There might be multiple impact modes for each activity or chain of events. A stressor is a chemical or biological agent, environmental condition or external stimulus that might contribute to an impact mode.
The hazard analysis reflects the conceptual models and beliefs that domain experts hold about the ways in which coal resource development might impact surface water and groundwater, and the relative importance of these potential impacts. As a result, the analysis enables these beliefs and conceptual models to be made transparent.
1.1.2 Component 2: Model-data analysis
Once all of the relevant contextual information about a subregion or bioregion is assembled (Component 1), the focus of Component 2: Model-data analysis is to analyse and transform the information in preparation for Component 3: Impact analysis and Component 4: Risk analysis. The BA methodology is designed to include as much relevant information as possible and retain as many variables in play until they can be positively ruled out of contention. Further, estimates of the certainty, or confidence, of the decisions are provided where possible; again to assist the user of the BA to evaluate the strength of the evidence.
The analysis and transformation in Component 2 depends on a succinct and clear synthesis of the knowledge and information about each subregion or bioregion; this is achieved and documented through conceptual models (abstractions or simplifications of reality). A number of conceptual models are developed for each BA, including regional-scale conceptual models that synthesise the geology, groundwater and surface water. Conceptual models of causal pathways are developed to characterise the causal pathways, the logical chain of events ‒ either planned or unplanned ‒ that link coal resource development and potential impacts on water resources and water-dependent assets. The conceptual models of causal pathways brings together a number of other conceptual models developed in a BA, for both the baseline and the CRDP. The landscape classes and the hazard analysis are also important inputs to the process. Emphasising gaps and uncertainties is as important as summarising what is known about how various systems work.
The causal pathways play a critical role in focusing the BA on the impacts and their spatial and temporal context. They provide a basis for ruling out potential impacts for some combinations of location and assets; for example, a particular type of wetland might be beyond the reach of any type of potential impact given the activities and location of the specific coal resource development in the subregion or bioregion. The causal pathways also underpin the construction of groundwater and surface water models, and frame how the model results are used to determine the severity and likelihood of impacts on water and water-dependent assets.
Surface water models and groundwater models are developed and implemented in order to represent and quantify the hydrological systems and their likely changes in response to coal resource development (both baseline and CRDP). Surface water models are drawn from the Australian Water Resources Assessment (AWRA) modelling suite, which includes the landscape model AWRA-L for streamflow prediction and river systems model AWRA-R for river routing and management. The latter is only used in a subset of subregions or bioregions and depends on the nature of the river regulation and the availability of existing streamflow data. The groundwater modelling is regional, and the choice of model type and coding is specific to a subregion or bioregion depending on data availability and the characteristics of the coal resource development in the area.
The hydrological models numerically estimate values for the hydrological response variables which are further analysed and transformed for the impact analysis. The hydrological response variables are subjected to sensitivity analysis and uncertainty analysis that test the degree to which each of the model inputs (parameters) affects the model results. It does this by running the model thousands of times and varying the values of the input parameters through a precisely defined and randomised range of values. The most influential parameters identified are taken into an uncertainty analysis, where more carefully chosen prior distributions for those parameters are propagated through to model outputs.
The uncertainty framework is quantitative and coherent. The models are developed so that probabilities can be chained throughout the sequence of modelling to produce results with interpretable uncertainty bounds. Consistent and explicit spatial and temporal scales are used and different uncertainties in the analysis are explicitly discussed. The numerical and uncertainty model results are produced at specific locations known as model nodes. Results can be subsequently interpolated to other locations, such as landscape classes and/or assets.
The values for the hydrological response variables estimated by the numerical modelling are critical to assessing the types and severity of the potential impacts on water and water-dependent assets. This is achieved through a staged receptor impact modelling.
First, information and estimates are elicited from experts with relevant domain knowledge about the important ecosystem components, interactions and dependencies, including water dependency, for specific landscape classes. The experts have complete access to the assembled BA information, including preliminary results from the hydrological numerical modelling. The results are qualitative ecosystem models of the landscape classes (or assets) constructed using signed directed graphs.
Based on these qualitative models, the second stage is producing quantitative receptor impact models where experts estimate the relationships between meaningful hydrological response variables and the resulting measurable change in a key characteristic of the landscape class or asset (i.e. receptor impact variables). For example, a receptor impact model could be elicited for the relationship between reduced surface water quality and the change in condition of habitat of a given species (as per Figure 6(b)). As only a small number of receptor impact variables (at least one and no more than three) will be identified for each potentially impacted landscape class, the particular receptor impact variables selected for the receptor impact modelling should be considered to be a measure of a critical ecosystem function (e.g. the base of complex food webs) and/or be indicative of the response of the ecosystem to hydrological change more broadly.
The receptor impact models are, where available, evaluated for each landscape class; this links the numerical hydrological modelling results (hydrological changes due to coal resource development) with ecological changes in water and water-dependent assets of the subregion or bioregion. Therefore, the output of Component 2 is a suite of information of hydrological and ecological changes that can be linked to the assets and landscape classes.
1.1.3 Component 3: Impact analysis and Component 4: Risk analysis
Once all of the relevant contextual information about a subregion or bioregion is assembled (Component 1), and the hydrological and receptor impact modelling is completed (Component 2), then the impact and risk is analysed in Component 3 and Component 4 (respectively).
These components are undertaken within the context of all of the information available about the subregion or bioregion and a series of conceptual models that provide the logic and reasoning for the impact and risk analysis. Coal resource development and potential impacts are sometimes linked directly to assets (e.g. for water sharing plans); however, more often, the impacts are assessed for landscape classes which are linked to assets using conceptual models. Impacts for assets or landscape classes are assessed by aggregating impacts across those assets or landscape classes.
Results can be reported in a number of ways and for a variety of spatial and temporal scales and levels of aggregation. While all the information will be provided in order for users to aggregate to their own scale of interest, BAs report the impact and risk analysis via at least three slices (impact profiles) through the full suite of information.
Firstly, the hazards and causal pathways that describe the potential impacts from coal resource development are reported and represented spatially. These speak to the potential hydrological changes that might occur and might underpin subsequent flow-on impacts that could be considered outside BA. The emphasis on rigorous uncertainty analyses throughout BA will underpin any assessment about the likelihood of those hydrological changes. All hazards identified through the IMEA should be considered and addressed through modelling, informed narrative, considerations of scope, or otherwise noted as gaps.
Secondly, the impacts on and risks to landscape classes are reported. These are assessed quantitatively using receptor impact models, supported by conceptual models at the level of landscape classes. This analysis provides an aggregation of potential impacts at the level of landscape classes, and importantly emphasises those landscape classes that are not impacted.
Finally, the impacts on and risks to selected individual water-dependent assets are reported. These are assessed quantitatively using receptor impact models at assets or landscape classes, supported by the conceptual models. This analysis provides an aggregation of potential impacts at the level of assets, and importantly emphasises those assets that are not impacted. Given the large number of assets, only a few key assets are described in the technical product, but the full suite of information for all assets is provided on http://www.bioregionalassessments.gov.au. Across both landscape classes and assets the focus is on reporting impacts and risks for two time periods; a time related to peak production in that subregion or bioregion, and a time reflecting more enduring impacts and risk at 2102.
The causal pathways are reported as a series of impact statements for those landscape classes and assets that are subject to potential hydrological impacts, where there is evidence from the surface water and groundwater numerical modelling. Where numerical modelling results are not available, impact statements will be qualitative and rely on informed narrative. If signed directed graphs of landscape classes are produced, it might be possible to extend impact statements beyond those related to specific receptor impact variables, to separate direct and indirect impacts, and to predict the direction, but not magnitude, of change.
In subregions or bioregions without relevant modelled or empirical data, the risk analysis needs to work within the constraints of the available information and the scale of the analysis while respecting the aspirations and intent of the BA methodology. This might mean that the uncertainties are large enough that no well-founded inferences can be drawn – that is, the hazards and potential impacts cannot be positively ruled in or out.
1.2 Role of this submethodology in a bioregional assessment
This submethodology (M09) is intended to assist those conducting a BA to quantify uncertainty in the results from the groundwater and surface water models. It has been written to be generally applicable to all subregions or bioregions. This means that it is pitched at a conceptual level; specific details of the uncertainty analyses and the outputs from those analyses will be written in product 2.6.1 (surface water numerical modelling) and product 2.6.2 (groundwater numerical modelling) for each Assessment. Those products identify areas of a subregion or bioregion where the hydrological impact of coal resource development are modelled to occur due to changes in surface water or groundwater. The sensitivity analysis identifies the model parameters that most affect predicted hydrological response variables, and that therefore need to feature in the uncertainty analysis. The outputs from the uncertainty analysis play a key role in quantifying the range of hydrological changes that might occur. This is essential input for the receptor impact modelling which assesses how ecological and human-dominated systems (and water-dependent assets) respond to those potential hydrological changes. Interactions between several activities of a BA and the sensitivity and uncertainty analysis are depicted in Figure 7.
Figure 7 Process for sensitivity and uncertainty analysis (large blue box) and connections to other bioregional assessment activities (grey boxes)
Conceptual representation of the physical system, inputs to and from the groundwater and surface water models, and the sensitivity and uncertainty analysis which considers uncertainties in input parameters and carries them through to hydrological response variables. Surface water modelling uses the Australian Water Resources Assessment (AWRA) model suite, while the groundwater model varies between subregions and bioregions.
The uncertainty analysis relies on input from:
- the context statement (product 1.1)
- the coal and coal seam gas resource assessment (product 1.2)
- the hazard analysis (product 2.3)
- the conceptual model of causal pathways (product 2.3)
- surface water modelling (product 2.6.1)
- groundwater modelling (product 2.6.2).
The estimated uncertainty in hydrological response variables is input for:
- surface water modelling (product 2.6.1)
- groundwater modelling (product 2.6.2)
- receptor impact modelling (product 2.7).
Readers should consider this submethodology in the context of the complete suite of methodologies and submethodologies from the Bioregional Assessment Programme (see Table 1), particularly the BA methodology (M01 as listed in Table 1; Barrett et al., 2013), which remains the foundation reference that describes, at a high level, how BAs should be undertaken. Submethodology M09 is most strongly linked to the following submethodologies:
- submethodology M05 for developing a conceptual model of causal pathways (Henderson et al., 2016)
- submethodology M06 for surface water modelling (Viney, 2016)
- submethodology M07 for groundwater modelling (Crosbie et al., 2016)
- submethodology M08 for receptor impact modelling (as listed in Table 1)
- submethodology M10 for impacts and risks (as listed in Table 1)
- submethodology M11 for hazard analysis (Ford et al., 2016).
Any deviation in application of, or refinements to, the BA methodology will be submitted for approval and recording by the Programme Science Leadership Group (SLG).