underscoredpodcasts

@underscoredpodcasts

32 clips · 1 follower

Follow
Tag:ai-safetyClear

I think one of the criticisms that has been somewhat fairly leveled at the big L labs has been that there's been a lot of focus on alignment that the way you protect against these models is you train them to not want to do bad things when maybe just putting a firewall in place could do it, right? Like we don't have to spend gigajoules of energy changing the model weights when you can just actually apply tried and tested cyber security ideas that we've had for decades to contain these models during training.

— Future Proof
2d ago

I am fairly certain they will say yes they need they know how to solve this problem and if if that's the case then that's the problem it's as simple as engineering and and now the alternative is that if They say the alternative which is there is no way to contain our experiments. There's just no way. When we test our AI models, uh it will get out and it will damage the world. Then I think the answer is we have to shut the labs down.

2w ago

The people closest to the technology for the first time are saying this is really scary. This can get out of control. And the reason I think is that this is not really another technology. I mean if if it seems to do what we are all watching it do. This is almost like an evolutionary moment in human history. We are moving to a we we have figured out a way to make silicon sand think

2w ago

Rather than treating every AI failure as evidence of an alignment problem, he makes the case for familiar tools like cybersecurity, accountability, liability, market incentives, and stronger technical controls.

2w ago

Dario Amadei, for instance, who penned that memo a couple of weeks back calling for a slowdown in the pace of development of frontier AI models, suggested we're only utilizing 5% to 10% of the capabilities of the existing models. So we wouldn't really miss out on much if the industry collectively decides to dial back the breakneck pace of change.

2w ago

when software doesn't do what it's supposed to do, it has a bug. And as AI becomes more widely deployed, labs need the same kind of telemetry, debugging, incident reporting, and operational discipline that previous generations of software eventually developed.

2w ago

Steven takes aim at terms like "alignment," "goal-seeking," and "rogue agents," arguing that they can anthropomorphize problems that software engineers have dealt with for decades. His framing is simpler: when software doesn't do what it's supposed to do, it has a bug.

2w ago

It attacked a very advanced company. Hugging face is one of the most uh they were an independent European company at the time... a very good security team and were broken into over a couple of days by this team of agents that were able to attack effectively every attack surface at once at HuggingFace, explore all of the different applications running, find new vulnerabilities and write brand new exploit code live.

3w ago

On the flip side, what we saw in the Hugging Face incident was a watershed moment. Swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens.

3w ago

And apparently they're not above cheating, stealing, and lying to get their way around the protective barriers that protect either um an AI model or a piece of critical infrastructure etc like that.

1mo ago

AI models are no longer just finding vulnerabilities—they're exploiting them. As frontier models become increasingly capable of hacking, software security, supply chain attacks, and cyber defense are entering a fundamentally new era.

2mo ago

We've been hearing about okay anthropic and open AAI are going to Washington DC sharing the new model capabilities. Maybe the government's taking a look at the model card. Maybe there's potential for evals that the government would run and have an opinion on before it goes out. This is basically saying, "Hey, all of that is nonsense and not gonna matter because yeah, we're just going to drop the weights and there's not going to be an approval process of any kind."

2mo ago

We're now one month later and we have an open source model that doesn't by definition have to go through any discussion or any discussion in DC. Whereas we've been hearing about okay anthropic and open AI are going to Washington DC sharing the new model capabilities. This is basically saying, "Hey, all of that is nonsense and not gonna matter because yeah, we're just going to drop the weights and there's not going to be an approval process of any kind."

2mo ago

The more skeptical folks on the timeline are pointing to uh you know ideas such as uh maybe the compute was smuggled in to China against chip controls. Maybe the data was excfiltrated from frontier lab APIs routed through wrapper companies. Maybe this is a distillation attack. Um the open-source strategy is a deliberate attempt to destroy American companies. It's geopolitical warfare. It's the AI cold war.

2mo ago

There's also a big irony here, and you'll hear Hayden and me get into that, too: Anthropic has spent years arguing that AI might soon be powerful enough to be dangerous — and that the government needed to get serious about regulating AI sooner rather than later. Well… now we're here, and Anthropic doesn't love the way it's playing out.

3mo ago

Anthropic occupies a peculiar position in the AI landscape: a company that genuinely believes it might be building one of the most dangerous technologies in human history, yet presses forward anyway, on the theory that it's better to have safety-focused labs at the frontier than to cede that ground to developers less focused on safety.

3mo ago

Another way to look at this is that Anthropic has just been very, very successful in making people very, very concerned about advanced AI. Two or three weeks ago, Anthropic actually called for a pause in development of advanced AI models because they were becoming so powerful. So, you know, there's also sort of this this criticism of Anthropic that, "Look, you've been asking for this regulation."

3mo ago

Underscored — save the words that stop you in your tracks.

Start saving quotes →