I think one of the criticisms that has been somewhat fairly leveled at the big L labs has been that there's been a lot of focus on alignment that the way you protect against these models is you train them to not want to do bad things when maybe just putting a firewall in place could do it, right? Like we don't have to spend gigajoules of energy changing the model weights when you can just actually apply tried and tested cyber security ideas that we've had for decades to contain these models during training.
You might also like
I think if you said AI developers will move full speed ahead without taking precautions that would significantly slow them down, you're looking at significantly more than a 10% risk. I think we're looking at something like maybe a 1/3 chance that we have superintelligence within one year on the current development trajectory, if we do not slow.
Paul Christiano
But in AI research, the priorities are exactly reversed. AI labs mainly care about how any new algorithmic recipe empirically makes training more efficient or elicits smarter models. They don't care about theory for its own sake, and all the progress we've made in deep learning over the last decade and a half has not required much new theory anyway. The equivalent of grinding through random conjectures would be enormously valuable in machine learning in a way that it just isn't in mathematics.
Dwarkesh Patel
Escalating hacks affecting blood labs and grocery stores show the very real risks to critical infrastructure that come from basic security lapses involving internet-connected devices.
Lily Hay Newman, Matt Burgess