The thing that's really interesting about transformers is that the model architecture is essentially fixed. The operations that transformers do are the same whether you're running GPT-4 or any other large language model — it's always matrix multiplications and attention. And if you build a chip that only does those operations, you can make it much faster and much cheaper than a general-purpose GPU.
You might also like
The record of a man who makes consistently good decisions in complex situations is almost always a record of a man who has reduced the number of decisions he has to make. The more choices you face, the more your judgment degrades.
Charlie Munger — Poor Charlie's Almanack
The chains of habit are too light to be felt until they are too heavy to be broken. A young person who consistently saves a small amount will find, decades later, that the habit has built a fortress; a young person who defers saving will find the habit of spending has built a prison.
Warren Buffett — University of Florida Business School Speech
Mimicking the herd invites regression to the mean. You will never produce a superior investment record by buying what everybody else is buying. The decisions that look best in retrospect are often the ones that felt most uncomfortable at the time.
Howard Marks — The Most Important Thing: Uncommon Sense for the Thoughtful Investor