Building Confidence in Safety Cases

Safety cases, consisting of a diverse set of arguments and evidence, are required for many technological developments including oil and gas extraction and storage, and the geological disposal of chemical and radioactive wastes. Evidence Support Logic, as implemented in TESLA, is an ideal tool for identifying uncertainties that need to be reduced in order to develop a safety case, and for showing that a safety case is robust with respect to the remaining uncertainties.

A safety case is an integrated collection of arguments and evidence to demonstrate that a system of interest is safe for a given application in a given operating environment (e.g. IAEA. 2003; MOD, 2007). The arguments and evidence are typically diverse. For example, the post-closure safety case for a radioactive waste repository will need to provide safety-related arguments and evidence concerning the wastes themselves (which may be of varied types), different natural and engineered barriers, the overall repository design (including the layout, dimensions and spatial distributions of emplaced wastes) and potentially impacted receptors, such as ecosystems and natural resources including groundwater aquifers. Furthermore, a safety case will need to take into account temporal variations in the characteristics and behaviour of the system of interest. In the case of a radioactive waste repository, the time scales to be covered by a post-closure safety case are typically very long (commonly 1 Ma in the case of a deep geological repository), with a consequent high probability of there being large environmental changes (e.g. glaciation and deglaciation) during the timeframe covered by the safety case.

These arguments and evidence will be associated with different uncertainties. Typically some of these uncertainties can be estimated from direct measurements (e.g. repeat measurements made by some analytical equipment), while others are obtained only expert judgement (e.g. the uncertainty in the proposition that a particular geological fault which has not ever been observed to move, is inactive).

It follows that to develop a safety case there is a need to:

  • structure the varied arguments transparently in a way that makes clear how each one contributes to the overall assertion that the considered system will be safe;
  • show relationships between different arguments and between each argument and its underlying evidence base; and
  • represent all uncertainties and show that they do not call into question the safety case.

Evidence Support Logic (ESL) can be used to help meet these goals by providing a systematic method for analysing the level of confidence that can be placed in a particular judgement, and allowing the analysis of that judgement to be represented graphically. A hypothesis of interest (e.g. ‘The system will be safe’) is broken down into two or more supporting ‘child’ hypotheses (e.g. ‘The system has been properly implemented’; ‘The components of the system are of sufficient quality’). Each of these supporting hypotheses may be broken down further, and their child hypotheses may be broken down in turn. This process of breaking down hypotheses is continued until a decision tree has been constructed at an appropriate level of detail. This level of detail needs to represent all the arguments and evidence that underpin the decision at the top level.

Once a tree has been constructed, the degree of confidence that each hypothesis at the lowest level of the tree is true or false is judged independently and represented numerically (on a scale of 0-1), based on the available evidence. The confidence for and against these hypotheses is then propagated through the tree, depending upon carefully selected logical operators and parameters called sufficiences, akin to weights. An important point is that the separate assessments of evidence ‘for’ and ‘against’ result in honesty about what is not known or is uncertain. ESL is therefore an ideal tool for:

  • identifying those uncertainties that need to be reduced in order to develop a safety case; and, once a safety case has been developed
  • showing that the safety case is robust with respect to the remaining uncertainties (i.e. these uncertainties do not call into question the assertion that the system will be safe).

The application of ESL in the development of safety cases for deep geological repositories for radioactive wastes was demonstrated for example by Umeki et al. (2003). A safety case for such a repository depends partly on the stability of the geological environment and the very slow rate of groundwater flow and solute transport through the geosphere. Arguments that these requirements are attained must be based on a conceptual site model that integrates numerous types of information in an internally consistent way. These types of information need to be integrated systematically and transparently. Furthermore, confidence in a particular conceptual model needs to be built by testing it against varied geological evidence. These ideas were illustrated by Umeki et al. (2003) who presented an example ESL tree to judge the hypothesis that ‘The interface between fresh and saline water at the site is stable’. The first three levels of their tree are reproduced using TESLA in the figure below (the original tree presented in Umeki et al. (2003) was constructed using the older software ‘Avanti’).

Tesla Tree

Umeki H, Seo T, Tsuchi H, Takase H and Metcalfe R, 2003. Integration of geological information in a structured approach to development of a safety case. Proceedings of the First Amigo workshop on building confidence using multiple lines of evidence Yverden-Les-Bains, Switzerland, 3rd -5th June 2003. Nuclear Energy Agency (NEA) Radioactive Waste Management Committee, Integration Group for the Safety Case (IGSC), Report NEA/RWM/IGSC(2004)8.

MOD, 2007. Safety management requirements for defence systems. Ministry of Defence Defence Standard 00-56, Issue 4.

IAEA, 2003. Radioactive waste management glossary. International Atomic Energy Agency, Vienna, ISBN 92–0–105303–7.