ARC-AGI-3

ARC-AGI-3 throws a 2D game at an AI with no instructions and asks it to figure out the rules by itself and beat the game. Just like a person encountering a new arcade game for the first time has to press buttons and learn the rules through trial and error, an AI only scores points by working out the game's logic the same way.

What makes this test special is that it doesn't just check whether the AI gives one good answer — it looks at how steadily the AI can sustain long chains of judgment across many steps, sometimes spanning days. Because of this, the same AI model can score very differently depending on how the software scaffolding wrapped around it is built. In one experiment, simply changing the scaffolding while keeping the model the same pushed the score from the 30s up to 100.

In other words, ARC-AGI-3 serves as a proving ground that reveals not just how smart an AI model is on its own, but how much the way it's operated and managed affects its real-world problem-solving ability.

Article 1 describes a case where "using Claude Opus 5 as-is scored 30% on ARC-AGI-3... but wrapping it in a software shell raised the score to 100%." The easy mistake here is thinking the score jump came from the model itself getting smarter — it actually came from changing the operating scaffold (harness) around the model. Article 2 notes that NVIDIA's new framework, NOOA, was evaluated using ARC-AGI-3 alongside SWE-bench Verified and CyberGym L1.