Prime Intellect publishes technical report on Prime Agent, ARC-AGI-3 score jumps from 30% to 95.5%

A harness is the execution layer that sits outside a language model when you hook that model up to real work. Calling tools, reading and writing files, running code, recovering from errors, deciding how much conversation history to keep — all of that happens here, not inside the model. The report's introduction frames a language model as a "bounded sequential processor": it can only decide its next move based on its weights and whatever information is currently sitting in its open context. The paper's starting premise is that the harness is what fills the gap between what a single computer can do and what a single model can do.

That means benchmark scores have always been a joint product of model and harness. If one tool call misfires and kills the session, the scoreboard just records it as a failure. The report states that Prime Agent "prevents harness failures from masquerading as model failures" (Prime Agent technical report abstract). The goal is to push the measured number closer to the model's actual ceiling.

Released August 5, revised August 24

Prime Agent itself launched on August 5. Prime Intellect introduced it then as a harness for coding and long-running autonomous tasks, built around programmatic tool calls, context handled as variables, multi-agent messaging, and self-repairing harness state — but no benchmark numbers came with that launch. This new report fills that gap. The paper's cover page lists an original publication date of August 5 and a current version dated August 24.

There are 11 authors, with affiliations listed as Princeton University, Prime Intellect, and MIT. Two corresponding addresses appear: seth@primeintellect.ai and altzhang@mit.edu. According to the company's post, the paper expands on an earlier blog post and puts the question of how a harness should be designed and evaluated at the center of the discussion.

Four components

The abstract breaks the architecture into four pieces.

Component	What it does
Persistent IPython REPL	A Python execution environment that stays alive for the whole session, treating context as a program and running test-time computation, following the recursive language model (RLM) abstraction
Continual Harness	Preserves conversation history, memory, skills, prompts, and subagent specifications across multiple task trajectories
Recursive subagents	Agents collaborate by talking directly to each other with no intermediary
Agents View	An interface that lets a human observe and manage sessions running as background daemons

The key point is that context gets handled as a variable a program manipulates, not as a blob of text. Instead of stuffing an entire long log into the model, the REPL trims it, summarizes it, and pulls out only the pieces that are needed. The design principle: execution, recovery, verification, and resource accounting are standardized inside the harness, while strategy is left to the model.

The X post numbered off four innovation points, but only two came through cleanly — agentic context management, and swarm/depth-n+ RLMs — while a third, "Verifiers support," cut off mid-sentence. It's worth checking the full paper for the rest of the list.

Where it was measured — from games to kernels

The list of test domains is unusual: long-context coding, GPU kernel generation, emulator building, and autonomous nanoGPT speedruns. That last one is a race to train a small GPT model to a target performance level as fast as possible, with the agent running the whole thing unsupervised. The abstract says Prime Agent matched or beat each model's native harness, as well as commonly used harnesses, across these domains.

Games made the list too. In Factorio, a factory-building game, the system could push through the tech tree without getting stuck, using iterative refinement, and dedicated subagents let it split work in parallel. The implication is that this kind of environment shows off an agent's ability to break a multi-hour goal into pieces and push on several fronts at once — something coding benchmarks don't capture as well.

Trying it yourself

The code is open-sourced at the PrimeIntellect-ai/prime-agent repository on GitHub, the address the paper's abstract points to. Because a harness is a swappable layer sitting on top of a model, you can run the same task you're already running with your current coding agent but swap in this harness, and see where the difference actually comes from. A practical way to try it: pull the repo, spin up a REPL session, throw a familiar task at it, and use Agents View to see exactly where the session stalls.

Editor's take

Whether or not you take the 30%-to-95.5% number at face value, what this report really disturbs is our confidence in a year's worth of agent benchmark scores. We've been reading "model X scored Y% on SWE-bench" as a statement about the model's raw capability. But if swapping the same model into a different shell can move the score this much, then that announcement was never really a model score — it was a score for a model-plus-harness combination. That's exactly why the paper puts the harness itself under evaluation. We've also been tracking recent attempts elsewhere to quantify infrastructure noise in agentic coding evaluations, and separate discussions about splitting the "brain" from the "hands" to scale managed agents. Multiple teams are clearly digging into the same problem at the same time.

The practical takeaway is that fixing your harness should come before swapping your model. Teams that have deployed coding agents tend to get stuck at the same point, over and over. It's usually not that the model doesn't know the answer — it's that a tool call goes wrong, a long log eats up the context window, or there's no way to revive a session that died halfway through, and the task just never finishes. Switching to a more expensive model at that point tends to double or triple your token bill without meaningfully improving your completion rate. The four things Prime Agent calls out for standardization — execution, recovery, verification, and resource accounting — hit exactly that pain point. Resource accounting in particular, tracking how many tokens each subagent burned, doesn't show up in a flashy demo, but it's exactly what determines your monthly bill in production.

The piece most immediately useful for teams here is the Continual Harness concept. Right now, most agent pipelines lose everything — conversation history and any tricks the agent picked up along the way — the moment a task ends. A design that carries history, memory, skills, prompts, and subagent specs across tasks pays off a lot when you attach it to repetitive internal work. If you're generating the same kind of report every week, your instructions get shorter starting the second week. On the flip side, swarms still feel early. Running multiple subagents in parallel works well in environments like Factorio, where the goal is clear and verification is automatic — but in work where the only verification is a human eyeball, parallelizing just means more review burden.

In the coming weeks, expect to see evaluations that hold the harness fixed and compare models, alongside ones that hold the model fixed and compare harnesses. And expect it to become standard practice for model announcements to specify which harness was used to produce a given score. Whether Prime Agent ends up setting that standard is a separate question — but an announcement that reports a score without naming the shell around it now reads as only half the story.