"Melanie Mitchell pointed out that if they change the encoding from numbers to other symbols, accuracy goes down. The numbers representing colors in the input can be used by LLMs to find unintended arithmetic patterns that can lead to accidental correct solutions. It does remind us that even within a benchmark, how you set up the question matters."