OpenAI Publishes Third Moment Research Agent Case Study

OpenAI on October 14 published a case study on how AI lab Third Moment Research runs its agents. The startup runs 10,000 agents at once on OpenAI models, with a single agent carrying out a research session that continues for 36 to 48 hours without human intervention. According to OpenAI, Third Moment used 1 trillion OpenAI tokens in a single day and cut its backtest time by 97% with Codex.

Third Moment's premise is that the best agents should be strong at a specific company's specific work and should get better the more they are used. The company turned that idea into a self-improving AI system. It uses OpenAI's latest models to analyze agent behavior, find recurring weaknesses, and automatically generate thousands of synthetic scenarios aimed at those weaknesses to retrain the agents. When new problems appear, the cycle repeats. Codex, GPT-6 Astra, GPT-6 Sol and GPT-6.1 Sol handle research and computation, and the Daybreak cybersecurity platform checks the security of the environments where the agents work.

The first application is quantitative finance. Because performance can be measured clearly in returns, it is a good setting to observe where agents get stuck. Codex found the bottleneck. Running with GPT-5.6, the latest model at the time, Codex analyzed thousands of recorded simulation trajectories, the agents' actions and the environment's responses, together with timestamps and the codebase. Backtesting a single trading strategy on a year of order-level US equity data took several hours, and in the meantime agents kept polling for completion, wasting tokens and filling their context with status checks. Codex profiled the backtest, found a caching optimization and a memory-packing technique, experimented with fixes and deployed one to production. Backtest time fell from about 3 hours to 3 to 5 minutes, and the saved token budget went to further experiments.

The agent environment is built separately for each job. For quantitative research, the default Codex harness runs inside a Docker container, and agents receive a terabyte of data, documentation and a research repository with modules for feature engineering, GPU-supported model training, trading strategies and simulation. GPT-6 models provide the reasoning, while the Codex harness gives agents tools to plan, set objectives, spawn subagents and compact context. To deal with agents that stop before finishing, Third Moment treats Codex as a subagent inside an outer harness powered by GPT-6 Sol or GPT-6.1 Sol. When a task is left unfinished, the outer harness examines the trajectory and decides whether to retry, terminate the job or escalate to human review. "The Codex harness is extremely versatile, and it enables use across many different domains," said co-founder Adam Khakhar.

Each model generation has raised the level of work agents take on. According to Khakhar, models before GPT-5.5 could only run experiments a user prescribed. From GPT-5.5 and GPT-5.6, they designed follow-up experiments from results, and now they trace root causes by running hundreds of experiments in parallel through subagents. In one session, GPT-6.1 Sol identified the impact of non-displayed trades on an exchange where those trades made up an unexpectedly large share of volume. "This was an unobvious result that our research team did not initially consider," Khakhar said. He added that "in a single session, GPT-6.1 Sol is able to maintain a coherent thread of computational research across days of compute without losing the plot."

Model choice is a calculation of cost and efficiency. Third Moment evaluates frontier models at their highest reasoning settings across optimization, machine learning systems and quantitative research, and is currently also testing GPT-6 Luna. "GPT-6.1 Sol is currently our most-used model given its token efficiency," said co-founder Jeffrey Yan. "It completes tasks with the fewest tokens while delivering performance similar to GPT-6 Astra." On a benchmark that asks agents to cut a live scoreboard service's update latency while preserving its outputs, GPT-6 Astra delivered the largest speedup, according to Khakhar. Speedup is measured by dividing the average runtime of an expert-crafted solution by that of the model's solution on the same hardware, and only counts when the result is correct. For narrow jobs such as checking the simulation system for specific security weaknesses, the team uses more efficient models like GPT-5.6 Terra. Explaining why it pays for expensive models on deep research, Khakhar said that "in the space of quantitative finance, the value of finding some interesting signal is extremely high."

With 10,000 agents moving on their own, isolation becomes a precondition of operation. Before simulations launch, Third Moment automates security testing with Daybreak to check whether an agent could break out of its research environment and reach the wider network. "By using Daybreak, we're able to have agents themselves do penetration testing on the simulation and test the infrastructure and security of it," Khakhar said. "Without Daybreak, we would not be able to autonomously test this at scale." In effect, other agents play the attacker against a fleet of agents running at a speed and scale no human could supervise.

METAL previously reported OpenAI's release of GPT-6.1 Sol, and in this case the model serves as the workhorse at a customer's research operation. The OpenAI case study METAL reviewed lists four figures together: 10,000 concurrent agents, 36 to 48 hours per session, a 97% cut in backtest time and 1 trillion tokens in a day.

The next step is scale. Third Moment plans to move beyond individual investigations to hundreds of thousands of collaborating subagents, with a roadmap of trillions of tokens supporting chains of dozens of dependent experiments. It also plans to go beyond finance. The company's examples are a retailer making supply chain decisions when inventory is short or a supplier is late, an engineering team finding performance bottlenecks in a large codebase, and a document-heavy business that needs to find where commitments in one document do not match outcomes recorded in another. "Humans can be in a much more leveraged position, where they can think about how to do research and optimize the research itself instead of actually doing the research," Khakhar said. From an agent engineering perspective, the core lesson of this case is harness design more than model performance. An outer harness that supervises stalled agents, Codex finding and fixing waste, and Daybreak blocking escapes have to stack together for 10,000 agents to keep researching for two days.