The thing that I find most compelling about the transformer is that, unlike prior neural network architectures, the transformer's compute requirements are almost entirely matrix multiplications. And matrix multiplication is a very structured computation that you can build hardware specifically to do very efficiently.
2w ago
Underscored — save the words that stop you in your tracks.
