As public benchmarks saturate and models get better at optimizing for the tests themselves, Rayan makes the case for independent, continuously evolving evaluations.
You might also like
The last of these — the progressives who simply couldn't admit that billionaire-funded private industry could build something powerful enough to endanger humanity — were the strangest of all, since their refusal to acknowledge the effectiveness of AI basically put them in an alliance with libertarians, hawks, and Trump.
Noah Smith
It did not take much time using the product for us to begin taking large language models and the companies building them extremely seriously. Four months into our run, my cohost's long conversation with an unhinged version of Microsoft Bing captivated the world and brought AI safety discussions into the mainstream. (It also drew a fair amount of ridicule, previewing the ways in which the commentariat would continually twist itself into pretzels to deny that anything important was happening with LLMs.)
Casey Newton
They said that it can be taught 10-minute long tasks that it has never seen before, from one video prompt without any fine-tuning.
Michael Spencer