If anyone builds this, everyone dies

And that's not all! We're also racing to destroy all the conditions (as in, every. single. one.) that allow us to survive as a species.

So if we fail with AI, we still have quite a few alternatives to finish screwing our species (and all the others).

Superintelligence, by its nature, is able to bootstrap itself. Therefore, there does not exist any primitive before it.

If it can exist after a series of steps, it's not superintelligent. Just buggy.

Isn't that the real horror? Slightly broken software running amok, embedded deeply in systems we no longer understand.