How do we build general AI that is reliable and aligned with our interests?
Serious leads
Why it matters
General-purpose AI could help grow economies and improve lives, but it can also be misused to cause harm, or cause harm unintentionally. Making it reliable and aligned with our interests would help secure those benefits while limiting the risks.
What we know
Current models are highly capable and improving fast.
What blocks us
We can't yet verify what a model is really trying to do, or guarantee its behavior.
Current leads
Interpretability, scalable oversight, evaluations of dangerous capabilities.
Sub-questions
How can people check an AI's work on tasks that are too hard for them to judge?
How can we stop an AI from exploiting flaws in the reward it is trained to maximize?
Why can an AI behave badly once in use, even if it was rewarded correctly during training?