The core insight is that transformers are actually very predictable in their memory access patterns — you know exactly what weights you're going to need before you need them. And so if you build a chip that's specifically designed around that access pattern, you can be much more efficient than a general-purpose chip that has to handle arbitrary memory access patterns.
1w ago
Underscored — save the words that stop you in your tracks.
