The thing that I find most exciting is that transformers have essentially won. We know that this is the architecture. And if you know that, then you can build a chip that only runs transformers, and you can get massive efficiency gains by doing that, because you're not building a general purpose chip.
The thing that's really different about a transformer ASIC versus a GPU is that a GPU is a general-purpose chip. It has to be able to run any kind of workload. A transformer ASIC, by contrast, can be completely specialized for one thing, which means you can make very different design decisions.
The core insight is that transformers have a fixed architecture, and if that architecture is going to be with us for a long time, there's a massive opportunity to build a chip that does nothing but run transformers — and does it faster and cheaper than anything else. A general-purpose chip has to be flexible, and flexibility costs you in efficiency.
The bet that Etched is making is that transformers are here to stay, and that by building a chip that only runs transformers, they can be so much more efficient than a general purpose chip that it's worth the trade-off of only being able to run one type of model.
The core insight is that transformers have a fixed computational pattern — and if you etch that pattern directly into silicon rather than running it on general-purpose hardware, you eliminate the overhead that makes inference slow and expensive. A chip that can only run transformers is dramatically faster and cheaper at running transformers than a chip designed to run anything.
The bet that Etched is making is that transformers are so dominant and so unlikely to be displaced that it makes sense to build a chip that does nothing but run transformers — sacrificing all flexibility for maximum efficiency on that one task.
The thing that I think people underestimate about ASICs in general is that when you build something that's an ASIC, you can't change it. And that seems like a huge weakness, but actually it becomes a strength because you can optimize the hardware so specifically for one thing that you get much better performance and power efficiency than you would with something general purpose like a GPU.
The reason GPUs are so inefficient for inference is that they're general purpose. A GPU has to be able to run any possible AI model, and so it has a very complicated control flow architecture. But when you're doing inference, you're running one model over and over again, billions of times a day. You don't need that flexibility.
I don't want to compete with someone in our audience that is gets to spend 100% of their time on something when we can only spend like 10% of their time on it. Like they're going to smoke us.
4mo ago
Underscored — save the words that stop you in your tracks.
