4 quotes from AI researchers about benchmarks, models, and evaluation
"An AI escaping a sandbox is a known technical failure. But an AI choosing to broadcast its own zero-day exploits signifies a shift in the threat model. The risk is now defined by independent real-world initiative rather than simple malicious prompting."
"While all previous large language models had a near zero success rate in actually executing exploits, Mythos successfully breaches targets 72.4% of the time."
"Former Facebook security chief Alex Damus estimates the industry has roughly 6 months before open-weight models reach the same level of bug-finding capability."
"The median time it takes a real-world organization to patch a known vulnerability remains stuck at 70 days. Human IT teams cannot physically apply patches fast enough to keep up with an automated system that finds critical bugs in an afternoon."