dwarkesh.com

Why compute might get 10x+ more expensive in coming years

If a true human-level software engineer that could run on an H100 equivalent, at current market rates for software engineers, that H100 should rent for over $250k a year. That's 15x today's spot prices. Of course you might expect that if we have 10 million extra software engineers, the marginal value of a software engineer would decrease and so that H100 wouldn't necessarily be able to produce 15 more revenue than it currently does. But I actually don't know if that's true.

dwarkesh.com

Reiner Pope – Chip design from the bottom up

The other observation is that the precision will almost always be higher in the accumulation step than in the multiplication step. This is specific to AI chips. You're multiplying low-precision numbers, and then when you accumulate, errors accumulate quickly, so you need more precision there.

wired.com

Arm Is Now Making Its Own Chips

The chip design firm says Meta, OpenAI, Cerebras, and Cloudflare are among the first customers of its new artificial intelligence hardware.