youtube on AI benchmarks
4 quotes from AI researchers about benchmarks, models, and evaluation
"In SWE-bench Pro for example, Claude Mythos beats out Opus 4.6 by 25%."
AI Explained @youtube · 2026-04-08 view on x
"Claude Mythos getting 83% beats out Gemini 3.1 Pro 82% and GPT 5.4 Pro at 80%. But on the subset remix, Claude Mythos gets the same score as Gemini 3.1 Pro and slightly underperforms GPT 5.4 Pro, which gets 88%."
AI Explained @youtube · 2026-04-08 view on x
"The geometric mean productivity uplift according to technical staff surveyed within Anthropic was 4x, four times the productivity when using Mythos."
AI Explained @youtube · 2026-04-08 view on x
"Humanity's Last Exam designed to test topics so obscure that it would indeed be the last exam that AI would saturate. Well, when allowed some tools, Claude Mythos gets almost two-thirds of those questions right compared to around 50% for other Frontier models."
AI Explained @youtube · 2026-04-08 view on x