"When taking away some of the AI's complex reasoning parts, known as mixture of experts, MoE, then replacing it with the pantry actually makes the AI smarter. Lower is better here on the loss curves. And not just a little, this is significantly smarter."