The thing that's special about transformers specifically is that you can make an ASIC, an application specific integrated circuit, specifically for transformers, and that chip will be so much faster and cheaper than a GPU at running transformers that it's almost incomprehensible. GPUs are a general-purpose chip that has to do everything, whereas an ASIC can be laser-focused on one task.
The thing that I think people underestimate about ASICs in general is that when you build something that's an ASIC, you can't change it. And that seems like a huge weakness, but actually it becomes a strength because you can optimize the hardware so specifically for one thing that you get much better performance and power efficiency than you would with something general purpose like a GPU.
1w ago
Underscored — save the words that stop you in your tracks.
