P(doom)

What does P(doom) mean?

Ask someone in AI-safety circles how worried they are, and you may get a number instead of a long explanation. P(doom) is an informal estimate of the chance that advanced AI leads to an existentially catastrophic outcome.

Exactly what counts as “doom” varies. One person may mean human extinction; another may include permanent human disempowerment, an AI takeover or another outcome that sharply limits humanity's future. Estimates may also use different time frames or be conditional on AGI existing at all.

The number usually is not the result of a precise calculation. It compresses a much bigger set of beliefs about AI progress, control and risk into something a person can say in one breath.

How people use it

He said his P(doom) went up after reading the new model card. Didn't say by how much.

Example written for this dictionary.

The estimates vary widely. Two people can land on the same figure for completely different reasons, which is one reason the shorthand gets criticized. Outside serious AI-risk discussions, people also borrow it for ordinary situations headed for disaster: “My P(doom) for this deploy is high.”

Why it matters

P(doom) turns a complicated argument into a number that is easy to compare. That is also its weakness. A percentage does not tell you what someone thinks will go wrong, when it might happen, whether the estimate assumes AGI has already arrived, or what could prevent the outcome.

Still, the shorthand stuck because it is quick, memorable and easy to ask. It also gives public arguments about AI risk a scoreboard, even when the numbers are not truly comparable.

Where it came from

The expression grew out of rationalist and AI-safety communities, where uncertain beliefs are often discussed as probabilities rather than plain confidence or doubt. It became much more visible outside those circles as arguments about advanced AI risk moved into mainstream technology coverage in the early 2020s.

You may see it mentioned alongside older pieces of AI-risk lore such as Roko’s Basilisk, although one is a personal risk estimate and the other is a thought experiment.