Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's really interesting about transformers is that the weights are basically fixed after training. You load the weights once onto the chip, and then you just do inference over and over again. So if you build a chip that is specifically designed to run one model or one architecture, you can get massive efficiency gains because you're not paying for the flexibility of being able to run arbitrary code.

TBPN

5.6 Sol + Superapp Release, AI 2040, Tibothy Joins, New Robot Hand Alert, Meta Releases Muse 1.1

Already, SOL has been transforming our research program. As one example, GPT-5.6 SOL autonomously post-trained 5.6 Luna.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that I find most stunning about this is that transformers were invented in 2017, and the reason that they've taken over everything is that they're extremely good at using parallel compute. And so the question is, are transformers the last great neural network architecture? And our bet is yes, or at least that they will be dominant for the foreseeable future, which means that if you build a chip that is perfect for transformers, you will have a chip that is perfect for AI.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's really interesting about transformers is that the attention mechanism, which is the key innovation, is actually a fixed algorithm. It's not like the weights change the algorithm; the weights just parameterize the algorithm. And so if you know that the algorithm is fixed, you can build hardware that's specifically designed to run that algorithm as fast as possible.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's special about transformers is the attention mechanism, and the attention mechanism has this property that every single token has to interact with every other token. The amount of computation required scales with the square of the sequence length. So as you try to process longer and longer contexts, it gets quadratically more expensive.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's really interesting about transformers is that actually, when you look at what all these models are doing, they're all basically running the same algorithm, which is the transformer architecture. And so if you build a chip that's specifically designed to run transformers as fast as possible, you can get enormous speedups compared to a GPU, which is a general-purpose chip that has to be able to run any algorithm.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that I find most compelling about the transformer is that, unlike prior neural network architectures, the transformer's compute requirements are almost entirely matrix multiplications. And matrix multiplication is a very structured computation that you can build hardware specifically to do very efficiently.

The Business of Tech

Kiwi co-founded world‑builder hits $2.5 billion valuation

Odyssey AI is not trying to build yet another large language model. Instead, Jeff Hawk, the company's co founder, is focused on world models AI systems that learn from site and sound to understand and simulate how the real world works over time, predicting what happens next and letting us interact with those simulated futures.

Semafor

DeepMind Chief Demis Hassabis Says Google’s Still Winning AI Talent | Semafor Tech

It was basically sort of thought to be career suicide. Like of course we know AI doesn't work. You know, we tried it in the 90s, places like MIT and it was a dead end. You know, and that was the prevailing view. But we just felt the small band of us felt that that actually with the right ideas and using learning systems, reinforcement learning and betting on neural networks that a lot of fast progress could be made.

The Business of Tech

Kiwi co-founded world‑builder hits $2.5 billion valuation

world models AI systems that learn from site and sound to understand and simulate how the real world works over time, predicting what happens next and letting us interact with those simulated futures.

Decoder with Nilay Patel

Microsoft AI chief thinks superintelligence is near, but won't take your job

We covered everything from his approach to training new models to his criticisms of Anthropic talking about Claude as though it is conscious.

The a16z Show

From Models to Mobility: Building Waymo with Dmitri Dolgov

getting there required more than better models. It meant building a complete system for training, evaluating, and deploying a driver in the real world.

The a16z Show

What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado

what's actually required for AGI: the ability to keep learning after training and the move from pattern matching to understanding cause and effect