closing that loop still doesn't guarantee runaway "takeoff," and why measuring real progress requires accounting for both capability gains and the resources required to achieve them.
The thing that I find most compelling about the transformer is that, unlike prior neural network architectures, the transformer's compute requirements are almost entirely matrix multiplications. And matrix multiplication is a very structured computation that you can build hardware specifically to do very efficiently.
4mo ago
Underscored — save the words that stop you in your tracks.
