Speaker
Description
Safety-critical systems — particle accelerators, reactors, enrichment cascades — occupy a category apart. Failure is not a degraded user experience but a physical hazard, and the properties that define these systems (hard real-time constraints, formally verified interlocks, regulatory traceability, and a low tolerance for opacity) are precisely the properties that make standard machine-learning deployment untenable. This talk draws on our experience deploying and pitching AI/ML control and optimization at the Paul Scherrer Institute and at nuclear fuel-cycle facilities to argue that the automation of safety-critical systems is a distinct engineering discipline, not an application domain.
We present an architecture built on separation of concerns for safety: a deterministic safety layer retains exclusive enforcement authority; the model operates at a deliberately throttled cadence, well below the machine's reaction envelope; every proposed action is simulated before it is surfaced; and no action reaches the plant without an explicit green light from the machine-safety layer. We describe the methodology behind this — the requirements for high-fidelity historical data, a validated simulation environment, and deep operator expertise — and the confidentiality protections required to work inside sensitive facilities.
Central to the approach is a progression continuum for the human-in-the-loop boundary: automation earns trust incrementally, moving from advisory to supervised to autonomous only as evidence accumulates. We close on operator engagement as the decisive factor — the humans who run these machines are not obstacles to automation but its most rigorous validators.
