Scaling laws work capabilities keep keep jumping but our ability to have alignment and safety assurance just be built in seems not there. um maybe never going to be there if that is the case then what do we do so then you got to come outside the model.
This is different because we're getting this capability before we've really characterized the shape of the threat. Like they know with quantum like there's something called Shor's algorithm and like they got to fix that. So they got to make sure the Shores algorithm doesn't work to break Bitcoin wallets open. They know the mathematical problem. They got to solve that. Uh in this scenario, it's like there might be a different algorithm that runs on commodity hardware that works. And how do you know when you don't know the shape of the threat that's coming?
AI changes the economics of cyberattacks, allowing attackers to probe thousands of paths simultaneously, and why that means defense will ultimately need to become autonomous too.
I am fairly certain they will say yes they need they know how to solve this problem and if if that's the case then that's the problem it's as simple as engineering and and now the alternative is that if They say the alternative which is there is no way to contain our experiments. There's just no way. When we test our AI models, uh it will get out and it will damage the world. Then I think the answer is we have to shut the labs down.
It attacked a very advanced company. Hugging face is one of the most uh they were an independent European company at the time... a very good security team and were broken into over a couple of days by this team of agents that were able to attack effectively every attack surface at once at HuggingFace, explore all of the different applications running, find new vulnerabilities and write brand new exploit code live.
On the flip side, what we saw in the Hugging Face incident was a watershed moment. Swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens.
There might be like sarcasm hidden in the vectors doesn't make it through to the actual tokens and you realize that the AI was joking when it was saying, "I won't turn you into a paperclip." Wink wink.
I find that in general, people talk a big game about privacy concerns, and then they will give Uber all of their data. Think about Uber. With a hack and a thin layer of AI, you could pretty easily determine if someone just terminated a pregnancy, their HIV status, if they're having affairs, their sexual orientation.
Nightmare scenario is you go to Best Buy, you get a gaming PC with a couple pretty stock graphics cards, and you're able to run an open-source model that basically walks you through the steps of creating something really problematic.
It takes forever to really build incredible trust. It only takes a moment to break it. And that's something that we are very aware of. And AI, it just it makes it that much bigger because the use of that information can be so much more destructive if used improperly.
Here's a technology that is potentially more dangerous than nuclear weapons. We didn't let Oppenheimer start a company and start selling bombs to China.
5mo ago
Underscored — save the words that stop you in your tracks.
