TED Radio Hour

Trusting medicine in an age of doubt

What I appreciated was that in your talk, you showed a little bit of regret that you were so focused, so intent on creating this vaccine to deal with the coronavirus in record time that maybe you forgot some of these things because you were a little bit in the scientific bubble? I am currently in the scientific bubble because that's how I've been trained. I've been trained since I was 19 to think about science strictly for the purpose of science.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's really interesting about transformers is that the architecture has been so stable. If you look at what people are running in production today, it's the same basic architecture that was published in 2017. And the bet that we're making is that this architecture is going to be around for a long time, which means you can build hardware that's specifically optimized for it.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that I think is underappreciated is that transformers are probably here to stay. People have been trying to beat transformers for years and years and years, and nobody has been able to do it at scale. And so if you believe that transformers are here to stay, then you can build a chip that only runs transformers.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The key insight is that transformers are incredibly regular and predictable in their compute patterns, which means you can build a chip that does nothing but run transformers extremely efficiently. A GPU is a general-purpose chip that has to handle any kind of computation, so it wastes enormous amounts of die area and power on flexibility you don't need for inference.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that I find most exciting is that transformers have essentially won. We know that this is the architecture. And if you know that, then you can build a chip that only runs transformers, and you can get massive efficiency gains by doing that, because you're not building a general purpose chip.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that's really different about a transformer ASIC versus a GPU is that a GPU is a general-purpose chip. It has to be able to run any kind of workload. A transformer ASIC, by contrast, can be completely specialized for one thing, which means you can make very different design decisions.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The core insight is that transformers have a fixed architecture, and if that architecture is going to be with us for a long time, there's a massive opportunity to build a chip that does nothing but run transformers — and does it faster and cheaper than anything else. A general-purpose chip has to be flexible, and flexibility costs you in efficiency.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The bet that Etched is making is that transformers are here to stay, and that by building a chip that only runs transformers, they can be so much more efficient than a general purpose chip that it's worth the trade-off of only being able to run one type of model.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The core insight is that transformers have a fixed computational pattern — and if you etch that pattern directly into silicon rather than running it on general-purpose hardware, you eliminate the overhead that makes inference slow and expensive. A chip that can only run transformers is dramatically faster and cheaper at running transformers than a chip designed to run anything.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The bet that Etched is making is that transformers are so dominant and so unlikely to be displaced that it makes sense to build a chip that does nothing but run transformers — sacrificing all flexibility for maximum efficiency on that one task.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The thing that I think people underestimate about ASICs in general is that when you build something that's an ASIC, you can't change it. And that seems like a huge weakness, but actually it becomes a strength because you can optimize the hardware so specifically for one thing that you get much better performance and power efficiency than you would with something general purpose like a GPU.

Invest Like the Best with Patrick O'Shaughnessy

Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]

The reason GPUs are so inefficient for inference is that they're general purpose. A GPU has to be able to run any possible AI model, and so it has a very complicated control flow architecture. But when you're doing inference, you're running one model over and over again, billions of times a day. You don't need that flexibility.

TBPN

FULL INTERVIEW: Kevin Espiritu on Growing Epic Gardening

I don't want to compete with someone in our audience that is gets to spend 100% of their time on something when we can only spend like 10% of their time on it. Like they're going to smoke us.