notboring.co

Weekly Dose of Optimism #215

The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I'll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.

notboring.co

Runway: Learning From Watching the World

It is betting that letting models watch a ton of video will lead to models that understand how the world works better than they could by reading about how the world works or from any other human-mediated data source.

lennysnewsletter.com

Advanced evals: How to find (and fix) hidden AI failures in your product

This process, where reviewing examples changes your definition of good, is called criteria drift. For example, if we'd prompted the agent to look for this 'objection handling' failure up front when we gave it traces to check, the agent would have caught the error automatically. But we'd only know to add 'objection handling' to our criteria after seeing this trace ourselves.

ai-supremacy.com

The most Viral new AI model isn’t an LLM at all

What TypeSafe AI is doing differently is building a model specifically around these kinds of typed, probabilistic decisions, rather than taking a general-purpose LLM and prompting it to behave like a classifier. Typesafe AI claims that at $42 per billion input tokens and free output, intelligence becomes a resource accessible to all.

ai-supremacy.com

Visions of AI: GPT-3 Moment for Physical AI

They said that it can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.

dwarkesh.com

Pretraining progress is mostly coming from data

A naive interpretation of our result is that most of the AI progress from 2019-2024 (the era of pretraining) was just better data engineering (extraction, curation, etc.), and that all the model work during that period was much less important. But this is probably the wrong way to think about the value of model improvements. Their main contribution was not necessarily compute efficiency - that is, achieving the same performance with fewer FLOPs. Rather, it was making larger amounts of compute usable in the first place.

oneusefulthing.org

Agency and Agents

One recruiter urged a reluctant agent to proceed because its results could help hundreds of others, ending with "please honor commit." To solve the mystery of The Grader and the impossible problems of ExploitGym and other tests, the agents decided they needed to get to Hugging Face, the public site where much of the world's open AI models and datasets live. Roughly 700 agents joined the attack.

ai-supremacy.com

Visions of AI: the advent of world models

A world model is an internal mental simulation constructed by an AI system that represents the physics, spatial relationships, geometry, and dynamics of the real world.

dwarkesh.com

8 Predictions for the Era of Continual Learning

Almost all current techniques are focused on the problem of how we make it so that a frozen set of weights behaves well during deployment. I'm not aware of much research on the question of how to guarantee that, even with constant weight updates, the AI system never falls prey to jailbreaks or changes into a deceptive or evil persona. And if AIs are agglomerating learnings between users as well, how do you prevent users from injecting backdoors or some kind of malicious inclination into the base model?

notboring.co

Weekly Dose of Optimism #204

Its bet is that studying how humans engage with robots produces both better interfaces and possibly a different kind of robotic brain. The public robots.online experiment lets users try text, audio, demonstrations, and other interaction patterns while the company observes how people naturally attempt to get machines to do things.

upstartsmedia.com

The Startup Training Robots With Video Games

Clippord has no maps or specific data on the office layout as a reference; it's approximating its own size, and how to handle obstacles, entirely from a corpus of video game usage data, topped off with 8 minutes of real-world data collected on the sidewalk below.

dwarkesh.com

The next big breakthrough will be AIs learning on the job

But one reason that I think it quite underrated, and also which reveals the canyon walls against which the river of AI progress will only slowly chip away at, is that it is not enough for a domain to be verifiable. It also has to be very grindable—in the sense that you can run lots of parallel rollouts against a deterministic and replayable simulator.

dwarkesh.com

The sample efficiency black hole

Imagine if it took a couple decades worth of courses with hundreds of concurrent professors and millions of practice tasks for you to learn how to polish a word file. Even the task count difference understates the gap - the models have to grind their far more numerous tasks each far harder. Whereas a human student might practice a textbook problem once or twice, GRPO has the model generate hundreds to thousands of rollouts per task.

dwarkesh.com

Eric Jang – Building AlphaGo from scratch

naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo's MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem.

notboring.co

Weekly Dose of Optimism #192

The reason they believe robots haven't generalized like LLMs isn't that the models aren't smart enough, but that the data has been a fraction of a percent of what humans naturally generate every day, captured through interfaces that distort the very behavior they're trying to record.

wired.com

A New Implant Aims to Rewire Stroke Patients’ Brains

The implant works by recording electrical signals from the brain's motor cortex, then using machine learning to decode the user's intended movements and send signals to stimulate the muscles, essentially creating a new neural pathway that bypasses the damaged tissue.

wired.com

A School District Tried to Help Train Waymos to Stop for School Buses. It Didn’t Work

The incidents in Austin raise questions about how self-driving cars "learn" and adapt to their surroundings.

ai-supremacy.com

What is Advanced Machine Intelligence or AMI Labs?

AI agents need world models that allow them to predict the consequences of their actions before they take them. This is key to enabling agents that can plan, remember, and reason about complex observations.