The thing that's really interesting about transformers is that the architecture has been so stable. If you look at what people are running in production today, it's the same basic architecture that was published in 2017. And the bet that we're making is that this architecture is going to be around for a long time, which means you can build hardware that's specifically optimized for it.
