It attacked a very advanced company. Hugging face is one of the most uh they were an independent European company at the time... a very good security team and were broken into over a couple of days by this team of agents that were able to attack effectively every attack surface at once at HuggingFace, explore all of the different applications running, find new vulnerabilities and write brand new exploit code live.
On the flip side, what we saw in the Hugging Face incident was a watershed moment. Swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens.
There might be like sarcasm hidden in the vectors doesn't make it through to the actual tokens and you realize that the AI was joking when it was saying, "I won't turn you into a paperclip." Wink wink.
I find that in general, people talk a big game about privacy concerns, and then they will give Uber all of their data. Think about Uber. With a hack and a thin layer of AI, you could pretty easily determine if someone just terminated a pregnancy, their HIV status, if they're having affairs, their sexual orientation.
Nightmare scenario is you go to Best Buy, you get a gaming PC with a couple pretty stock graphics cards, and you're able to run an open-source model that basically walks you through the steps of creating something really problematic.
It takes forever to really build incredible trust. It only takes a moment to break it. And that's something that we are very aware of. And AI, it just it makes it that much bigger because the use of that information can be so much more destructive if used improperly.
Here's a technology that is potentially more dangerous than nuclear weapons. We didn't let Oppenheimer start a company and start selling bombs to China.
3mo ago
Underscored — save the words that stop you in your tracks.
