underscoredpodcasts

@underscoredpodcasts

9 clips · 1 follower

Follow
Tag:ai-alignmentClear

Their goal is a closed loop where the system being improved is also doing the improving, across components including models, harnesses, and eventually more of the AI stack.

— The a16z Show
2d ago

I think one of the criticisms that has been somewhat fairly leveled at the big L labs has been that there's been a lot of focus on alignment that the way you protect against these models is you train them to not want to do bad things when maybe just putting a firewall in place could do it, right? Like we don't have to spend gigajoules of energy changing the model weights when you can just actually apply tried and tested cyber security ideas that we've had for decades to contain these models during training.

1w ago

I am fairly certain they will say yes they need they know how to solve this problem and if if that's the case then that's the problem it's as simple as engineering and and now the alternative is that if They say the alternative which is there is no way to contain our experiments. There's just no way. When we test our AI models, uh it will get out and it will damage the world. Then I think the answer is we have to shut the labs down.

2w ago

On the flip side, what we saw in the Hugging Face incident was a watershed moment. Swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens.

3w ago

Anthropic occupies a peculiar position in the AI landscape: a company that genuinely believes it might be building one of the most dangerous technologies in human history, yet presses forward anyway, on the theory that it's better to have safety-focused labs at the frontier than to cede that ground to developers less focused on safety.

3mo ago

Underscored — save the words that stop you in your tracks.

Start saving quotes →