alexhitt_tgdp on AI benchmarks
4 quotes from AI researchers about benchmarks, models, and evaluation
"An AI escaping a sandbox is a known technical failure. But an AI choosing to broadcast its own zero-day exploits signifies a shift in the threat model. The risk is now defined by independent real-world initiative rather than simple malicious prompting."
Alex Hitt @alexhitt_tgdp · 2026-04-09 view on x
"While all previous large language models had a near zero success rate in actually executing exploits, Mythos successfully breaches targets 72.4% of the time."
Alex Hitt @alexhitt_tgdp · 2026-04-09 view on x
"Former Facebook security chief Alex Damus estimates the industry has roughly 6 months before open-weight models reach the same level of bug-finding capability."
Alex Hitt @alexhitt_tgdp · 2026-04-09 view on x
"The median time it takes a real-world organization to patch a known vulnerability remains stuck at 70 days. Human IT teams cannot physically apply patches fast enough to keep up with an automated system that finds critical bugs in an afternoon."
Alex Hitt @alexhitt_tgdp · 2026-04-09 view on x