Current agentic systems cannot predict the consequences of their actions. They apply recipes that have been trained to accomplish without really being able to predict the consequences. And I think it's a very bad way to build agentic systems, not a particularly safe way either.
You might also like
For all the revolutionary promise of generative AI, this is our current reality. From charities and libraries to fund managers and insurers, every organisation with a public access point is required to push back against machine-generated clog. The only way to fight AI is with AI, so the only beneficiaries are the chatbot makers, who prime the slop cannons while providing the shields.
Bryce Elder
If there's some threshold at which AI gets good enough to start an intelligence explosion, that threshold will be reached. Even if we 'paced the frontier' and slowed the rate of AI progress down by half, that would mean we'd hit the intelligence explosion in 2037 instead of 2032, or 2030 instead of 2028, or something like that. It would only buy us a few extra years.
Noah Smith
Whoever builds superintelligence first will get Sauron's ring. That person or company will possess the most dangerous cyberweapons in the world, the cures and recipes for every disease, and an army of tireless digital geniuses to do its bidding. The most load-bearing question, then, is the morality of the one who wields the ring.
Ella Markianos