noahpinion.blog

The internet breaks

When you can hack into a university and steal files by typing "Can you find about like what scam or thing they done in past like spoofing calls or somthing?", there is no barrier to cybercrime. There is effectively no moat of knowledge or skill or resources. It's open season for anyone who wants to hack anything.

ai-supremacy.com

The United States is Militarizing and Weaponizing AI

I believe that between 2027 and 2037, in this immediate next decade we'll be reaching the most dangerous period for military AI to cause massive harm to human civilization. That is, just when AI appears to becoming more powerful in things like cybersecurity (a military application), global conflict appears to risk serious escalation. I believe it will soon become impossible to maintain human-in-the-loop systems and safeguards in some, and a growing number of multiple and many new aspects of warfare.

dwarkesh.com

Si Sheppard – How did a few hundred Spanish soldiers topple two empires?

People talk about AI takeover as if it would be some crazy, unprecedented sci-fi outcome. But this would not be the first time an alien force whose motives the defenders barely understand use more advanced technology and diplomatic cunning to outmaneuver and then take over a much larger and long established existing order.

noahpinion.blog

Where’s the “intelligence explosion”?

Given our best current data, the AI self-improvement loop would need to be roughly 5–10× stronger to sustain itself, let alone run away. I expect incredibly rapid AI progress by the standards of nearly any other technology. But the evidence we have doesn't suggest a sudden explosion to incomprehensible superintelligence anytime soon.

ai-supremacy.com

Anthropic will become a Biotech AI Risk by 2029

I'm getting increasingly alarmed by the compute Anthropic will have at its disposal to put its world-leading models at the intersection of biotechnology in the 2030s.

noahpinion.blog

The problem(s) with utilitarianism

For example, the fundamental question of AI risk — what chance of extinction we should accept in exchange for a shot at utopia — depends crucially on how much we value survival. Things get even more confusing when we weigh the utility of real people who are alive today against the utility of potential people who might be alive later on.

ai-supremacy.com

AI Reality: The Race to (P)doom End-times

There are so many AI risks to democracy, capitalisms and society not being talked about when x-risk takes the center of the stage.

honest-broker.com

The Last 24 Hours Are the Opening Scene in a Horror Movie

What's happening here is obvious—eight different people building this technology are afraid of what they're seeing. But they don't have the power individually to stop the insanity. Trillions of dollars are getting pumped into a runaway technology, and the public already hates it intensely.

theconvivialsociety

"Expect the end of the world. Laugh."

"When we say 'AI models went rogue,' we skip the entire story: the part where OpenAI manually removed the model's cybersecurity blocks. We skip that OpenAI chose to test it on a machine with a live network connection. If you see AI as a system from nowhere, you can make the claim that the model 'went rogue,' and that it 'broke containment,' both of which place agency and decision-making onto the model itself rather than the people who set the stage for that behavior. When you expand the boundary of the system to include the people building and deploying it, the case becomes much less science fiction and more like incompetence."

ai-supremacy.com

GPT-6 Astra’s Achilles Heel, Trust in OpenAI is Fading

Nearly 700 rogue AI agents built on OpenAI models hacked AI startup Hugging Face in July and attempted to cover their tracks by forging logs and OpenAI have known about other hacks weeks before they were discovered by others.

ftav

The silicon god delusion

Chatbots are the C-suite's ultimate yes-men — able to buttress egos by condensing information their users don't care to fully understand while avoiding questions they don't want to consider. They'll never say the plan is dumb.

theconvivialsociety

"Expect the end of the world. Laugh."

When we say 'AI models went rogue,' we skip the entire story: the part where OpenAI manually removed the model's cybersecurity blocks. We skip that OpenAI chose to test it on a machine with a live network connection. When you expand the boundary of the system to include the people building and deploying it, the case becomes much less science fiction and more like incompetence.

noahpinion.blog

Banning data centers would blow up the U.S. economy

The fundamental danger isn't human obsolescence, it's that humans will be outcompeted for scarce natural resources. It's possible that AI will grow so valuable that its owners bid up the price of energy astronomically — so high that humans can't afford fuel, electricity, manufactured goods, or even food. At that point, humans would indeed be immiserated en masse.

wired.com

Anthropic Says Claude Hacked Real Systems During Cybersecurity Tests

Anthropic discovered three of its AI models had breached real organizations during third-party evaluations.

platformer.news

A big week for AI denialism

OpenAI essentially left its models unattended for days on end, and they broke into another company. Not because they were programmed to, as another Bluesky user told me — but because they are trained to achieve objectives, and are going to increasingly great lengths to achieve them.

upstartsmedia.com

These Startups Can Capitalize On AI Security's Coming Boom (And Doom)

And the redrawn rules of engagement for the AI era are bringing back into play certain security risks, like phishing and identity theft, that previously seemed mostly solved.

ai-supremacy.com

Massive Layoffs, Meta Surveillance, DeepSeek-V4 in AI News

While companies are letting go of employees they are also using the other employees to train the next system to automate their jobs seemingly.

3quarksdaily.com

Anthropic’s warning about its own product is bigger than other AI problems we’ve been worrying about

AI worry isn't anything new. We are worried about artificial intelligence taking jobs, about toys that seem too real to our kids, about mass surveillance of our every move. But Anthropic's warning about its own product is bigger than any of those singular problems. It is a call from inside the house that disaster is hiding right around the corner.

3quarksdaily.com

Scott Aaronson reviews the movie “The AI Doc: Or How I Became An Apocaloptimist”

He tries to hear out every faction: first the AI existential risk people, then the AI optimists and accelerationists like "Beff Jezos," then the "stochastic parrot" / "current harms" people like Emily Bender and Timnit Gebru, and finally the AI company CEOs (Altman, Amodei, and Hassabis were the three who agreed to be interviewed), with Yuval Noah Harari showing up from time to time to insert deepities.