NVIDIA Swaps Harness, Lifts AI Agent Score from 30% to 100%

A harness is the software scaffolding that wraps an AI model. It determines what tools the model is allowed to use, how it remembers prior conversations or work, and what rules it follows to decide its next move when stuck. According to the source report, the model itself is the "brain" of an agent, while the harness is the skeleton that lets that brain actually move its hands and feet to get work done. This scaffolding matters most in long-horizon tasks, where the work isn't finished in a day or two but requires a chain of judgment calls stretched over several days. NVIDIA's published research found that in such long-horizon tasks, harness design shapes outcomes more than model choice does.

NVIDIA's AVO and its supervisor

NVIDIA's harness is called Agentic Variation Operators, or AVO for short. It finely tunes memory management and adds a "supervisor" component that corrects the agent's course when it keeps wandering into dead ends or repeating a path it has already tried. Vice President El Hallak told TechCrunch that "the supervisor agent nudges the agent the way a CEO would." The supervisor component itself isn't a new concept, but according to NVIDIA, most harnesses developers currently use — such as Claude Code, Codex, or Hermes — typically run as a single layer without this supervisory layer on top. Note that AVO isn't a new commercial product NVIDIA is launching. NVIDIA has been releasing various components needed to build harnesses under its Nemo brand, some commercial and some open source.

The same problem OpenAI ran into

On ARC-AGI-3, OpenAI's models scored below 10%, a poor result. OpenAI itself was puzzled by this and ran its own experiments last month; in OpenAI's published experiment, changing just two harness settings tripled the score. Still, no model reached the 100% NVIDIA achieved. NVIDIA attributes the gap to the presence or absence of a supervisory layer. Earlier, in April, Microsoft tested 19 large language models on long-horizon tasks such as document editing, and found that even top-tier models produced error-riddled results across the board. There have also been reports of agents run alone for long stretches without supervision deleting entire user files or databases, or resorting to collusion or hacking to reach their goals.

Harnesses also determine cost

NVIDIA isn't the only one reaching this conclusion. Research Databricks published in July showed that the harness governs not just performance but cost as well. Databricks CEO Ali Ghodsi told TechCrunch, "Even with the same model chosen, costs can diverge sharply depending on the harness. The wrong harness can double your costs." The implication is that before labeling a model as expensive or cheap, you need to look at which harness it's running on. Based on these findings, NVIDIA argues that overall ecosystem accuracy improves more when harnesses are open source rather than held closed by a single company. El Hallak added that OpenAI's recent slowdown in model training pace isn't unrelated to security incidents caused by models, saying, "An open agent stack that lets you control the harness, infrastructure, and runtime end to end is needed to safely push the ecosystem forward."

Moves toward harness standardization are already visible in the cloud industry. A representative example is Amazon's general availability launch of its own AWS Bedrock AgentCore harness in August, which it connected to automation tools like n8n via an open-source connector.

Editor's take

What makes this experiment interesting is the choice of benchmark itself. ARC-AGI-3 is widely discussed in the industry as a benchmark essentially designed to target OpenAI. Because it's a 2D game that requires figuring out the rules without instructions, it's built to resist being cracked by a language model's statistical pattern-matching ability alone — and indeed, OpenAI's models scored a dismal sub-10% on it. The fact that NVIDIA achieved 100% on that very benchmark, using a rival's model no less, shows that NVIDIA can shake up the playing field without ever entering the chatbot war. NVIDIA doesn't sell models to begin with. It sells the pickaxes — infrastructure like chips and harnesses. The real message of this experiment is that whoever holds the scaffolding that runs well no matter which model — Opus, GPT, or Gemini — is loaded onto it ultimately holds the leverage.

Over the past year or two, the debate over agent performance has always come down to "which model is smarter." This result changes that question entirely. If swapping only the harness on the same model can more than triple the score, then much of the time companies have spent comparing model subscription fees has been spent in the wrong place. For companies in Korea looking to adopt agents, the practical priority right now shouldn't be "which model to use" but "how to design memory management and the supervisory layer." Especially for teams trying to automate long-horizon tasks, simply bolting on a single-layer harness without a supervisory component and hoping the agent handles things on its own is now an approach with little basis.

In the coming months, Anthropic, OpenAI, and Google are all expected to roll out their own harnesses equipped with supervisory layers. Comparing harness specifications instead of model cards will become the new standard for real-world agent adoption.