The companies building the most powerful models cannot be left to write their own safety rules, change those rules when they become inconvenient, and decide for themselves when a system is safe enough to release.
I think one of the criticisms that has been somewhat fairly leveled at the big L labs has been that there's been a lot of focus on alignment that the way you protect against these models is you train them to not want to do bad things when maybe just putting a firewall in place could do it, right? Like we don't have to spend gigajoules of energy changing the model weights when you can just actually apply tried and tested cyber security ideas that we've had for decades to contain these models during training.
I am fairly certain they will say yes they need they know how to solve this problem and if if that's the case then that's the problem it's as simple as engineering and and now the alternative is that if They say the alternative which is there is no way to contain our experiments. There's just no way. When we test our AI models, uh it will get out and it will damage the world. Then I think the answer is we have to shut the labs down.
Rather than treating every AI failure as evidence of an alignment problem, he makes the case for familiar tools like cybersecurity, accountability, liability, market incentives, and stronger technical controls.
We've been hearing about okay anthropic and open AAI are going to Washington DC sharing the new model capabilities. Maybe the government's taking a look at the model card. Maybe there's potential for evals that the government would run and have an opinion on before it goes out. This is basically saying, "Hey, all of that is nonsense and not gonna matter because yeah, we're just going to drop the weights and there's not going to be an approval process of any kind."
We're now one month later and we have an open source model that doesn't by definition have to go through any discussion or any discussion in DC. Whereas we've been hearing about okay anthropic and open AI are going to Washington DC sharing the new model capabilities. This is basically saying, "Hey, all of that is nonsense and not gonna matter because yeah, we're just going to drop the weights and there's not going to be an approval process of any kind."
One probable outcome of an openweight model dominant world is full AI communism, which is precisely what China proposes rather than a market product. AI is a public good, which will ultimately be provided by the state as a kind of digital public infrastructure.
There's also a big irony here, and you'll hear Hayden and me get into that, too: Anthropic has spent years arguing that AI might soon be powerful enough to be dangerous — and that the government needed to get serious about regulating AI sooner rather than later. Well… now we're here, and Anthropic doesn't love the way it's playing out.
Here's a technology that is potentially more dangerous than nuclear weapons. We didn't let Oppenheimer start a company and start selling bombs to China.
With the mission of bringing AGI to the world comes a responsibility to help create a space for real constructive conversation about the changes AI creates with builders and people using the technology at the center.
7mo ago
Underscored — save the words that stop you in your tracks.
