Terra Incognita
Artificial intelligence

Can we mathematically prove that an AI system is safe?

Distant horizon

Why it matters

Wrong outputs from neural networks can have costly consequences, and tests alone may not show that an AI system avoids dangerous behaviour. Mathematical proofs could give strong safety guarantees, like the high standards already required for critical infrastructure.

What we know

Formal verification already proves the correctness of some critical software.

What blocks us

Neural networks are far too large and opaque for current proof methods.

Current leads

Verifying small parts or outputs, AI that writes its own checkable proofs.

Sub-questions

Related questions

Sources

Not yet reviewed by a specialist.

Last checked: 2026-11-18

History of this question

Back to the map →