Given our best current data, the AI self-improvement loop would need to be roughly 5–10× stronger to sustain itself, let alone run away. I expect incredibly rapid AI progress by the standards of nearly any other technology. But the evidence we have doesn't suggest a sudden explosion to incomprehensible superintelligence anytime soon.
"I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
In the future, our capacity to steward our votes and our capital, and to make sense of what's happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels.
2mo ago
Underscored — save the words that stop you in your tracks.
