Workshop on Responsibility and Accountability in Safety-Critical AI Application

Project overview

Summary:

The workshop brought together participants from academia and industry to consider the cybersecurity risks arising from AI elements in socio-technical systems. The programme comprised a mix of scene-setting talks and small-group round-table discussions addressing themes such as:

1. The tension between classical systems engineering vs building rapidly without considering consequence or regard for process.

2. How to extend assurance processes to systems of systems incorporating AI technologies.

3. Small-group exercises to explore attack vectors to disrupt, deny, distract or destroy systems through scenario identification, analysis and consequence assessment.

4. The role of AI technologies in cybersecurity defence and offence, certification of processes involving AI.

5. Examination of the problems arising from the use of AI technologies in mental health therapies.

Highlights:

This innovation forum on Responsibility and Accountability in Safety-Critical AI Applications took place in School of Management at the University of Bath over 2-days (12+13th January 2026). It brought together 20 participants from industry and academia to take a multidisciplinary perspective on the socio-technical systems incorporating AI technologies: the focus was as much, if not more, on identifying unconsidered risks and harms than looking for solutions, although we could not resist some thinking about possible answers. 

The workshop was run under Chatham House rules, so there is not a list of speakers and associated slides. Instead, we had a mix of talks to provoke discussion and a group exercise. 

We began with a trio of talks from a commercial/industrial perspective on cybersecurity covering the role of standards, the use of AI for defence and offence and the organizational view of governance and certification. This provided the background for a group exercise where the objectives were to come up with scenarios to illustrate the quadrants of the matrix of Low and High vs Understanding and Technology (LULT, LUHT, HULT, HUHT) and what might be the possible consequences of an attack (disrupt, deny, distract, destroy), where a critical element of these narratives was of AI as an actor. 

On day two we started with a couple of use-case-driven informal talks: the first explored some of the issues that might arise from the use of AI in therapy (of some form) from the perspective of a psychologist; the second presented a commercial scenario of a goods transfer site for which there is a digital twin and an AI agent that processes machine learning generated forecasts in order to decide how to control the site. The question the group considered was “how to assure the AI agent?”. 

A more in-depth record, coupled with an analysis of the above is in preparation as a whitepaper. 

Project outputs

There are no immediate outputs.

A whitepaper on the topics covered is in preparation, and the content may be reworked for a journal paper.