Chip companies in the future might stop obsessing so much as they do right now about how small the transistors in a semiconductor should be and they might focus more instead on how fast the data moves through a semiconductor.
You might also like
Three years ago we were still in the ChatGPT era of AI, and I was very excited about the possibility of local inference. Then came the reasoning era, blowing up KV cache (which increases the need for more memory) and emphasizing the importance of decode (to generate that many more tokens). Now we're in the agentic era, where CPU performance is incredibly important. To that end, the ideal setup for a local agent is strong local CPU performance and calling out to the cloud for inference.
Ben Thompson
In the end, something has to transform electrons to tokens. The transformation of electrons to tokens and making those tokens more valuable over time is hard to completely commoditize.
Jensen Huang
Helium is as far as the Iran War and the semiconductor supply chain goes: as close to a "fundamental physical constraint" as we can get.
Michael Spencer