when AI solved some of its hardest problems: Navier-Stokes, the quasi-Riemann hypothesis, the unique games conjecture and hundreds of other problems.
You might also like
Invert, always invert. Many hard problems are best solved by asking what you want to avoid rather than what you want to achieve. Tell me where I'm going to die, so I'll never go there.
Charlie Munger — Poor Charlie's Almanack
Anthropic's prospectus says he led the engineering of GPT-3 there and that, 'along with Dario Amodei, [he] co-invented deep RLHF, which was central to converting base LLMs into modern conversational assistants such as Claude.'
Polina Pompliano
But in AI research, the priorities are exactly reversed. AI labs mainly care about how any new algorithmic recipe empirically makes training more efficient or elicits smarter models. They don't care about theory for its own sake, and all the progress we've made in deep learning over the last decade and a half has not required much new theory anyway. The equivalent of grinding through random conjectures would be enormously valuable in machine learning in a way that it just isn't in mathematics.
Dwarkesh Patel