Open-source Ornith-1.5 claims scores on par with Claude Opus

All of these figures were self-measured and self-reported by the development team. The scores for competing models listed for comparison in the same table were also re-measured by the team using its own methodology, and no third-party reproduction has been carried out yet. The announcement page notes that the figures are averages of five independent runs.

Ornith was built by the DeepReinforce team. On August 19, the team published the new model "Ornith-1.5" on its own site along with these results, and uploaded the weights to the "ornith-ai" organization on Hugging Face.

From a model that prepares itself to a model that improves itself

Ornith-1.5 is not the team's first model. Its predecessor, Ornith-1.0, was built on Alibaba's Qwen 3.5 and Google's Gemma 4, with additional continued pretraining (CPT), mid-training, and post-training layered on top. That earlier release introduced the concept of "self-scaffolding," in which the model builds, on its own, the tools and procedures — the scaffold — needed to solve a problem, then solves it on top of that scaffold.

Ornith-1.5 takes this a step further. Rather than stopping at building scaffolds, it completes a loop in which the model generates the tasks to be solved, builds scaffolds suited to those tasks, and generates the solution rollouts, using all of this as material for reinforcement learning. This means the model doesn't rely on a predetermined problem set or hand-built grading tools; instead, it continually finds gaps in its own capabilities and generates problems that fill those gaps.

Report card across three weight classes, from 397B to 9B

Ornith-1.5 comes in three sizes: a 397-billion-parameter (397B) MoE (mixture of experts), a 35-billion-parameter (35B) MoE, and a 9-billion-parameter (9B) dense model. MoE architectures select only a subset of "expert" parameters out of the total for each query, requiring less computation than a dense model of the same size.

The table released by the development team is reproduced below.

Model	Parameter architecture	Terminal-Bench 2.1	Other benchmarks
Ornith-1.5-397B	MoE	86.1	DeepSWE 56.0
Ornith-1.5-35B	MoE (3B active per token)	68.5	SWE-Bench Verified 79.0
Ornith-1.5-9B	Dense	47.0	SWE-Bench Verified 70.6

The scores for competing models listed for comparison in the same table are as follows. These figures were also measured by the development team in its own evaluation environment.

Model	Terminal-Bench 2.1	Other benchmarks
Claude Opus 4.8	85.0	DeepSWE 59.0
GLM-5.2	82.7	DeepSWE 46.2
DeepSeek-V4-Flash-0731	82.7	DeepSWE 54.4
Gemma 4-31B	43.4	SWE-Bench Verified 52.0
Muse Glimmer-30B	51.7	SWE-Bench Verified 76.0

Ornith said its 35B model, despite activating only 3 billion parameters per token, outperformed not only Qwen 3.6-35B in the same weight class but also larger dense models such as Gemma4-31B and Meta's Muse Glimmer-30B by a wide margin on agentic coding benchmarks.

A 9B model that fits on a smartphone

The 9-billion-parameter model was also released as a downsized quantized version, "Ornith-1.5-9B-Mobile," designed to run directly on iPhone and Android devices. The team reported that even this small model scored 47.0 on Terminal-Bench 2.1 and 70.6 on SWE-Bench Verified, matching or exceeding the much larger Gemma4-31B and Qwen 3.6-35B.

Hugging Face hosts the original weights for all three sizes, along with 4-bit, 6-bit, and 8-bit quantized variants in GGUF and MLX formats — ready to download and run directly via MLX on Mac or GGUF on PC and Linux.

How tasks, scaffolds, and solutions are trained in a single loop

One training cycle runs in three stages. First, the model proposes a harder task on its own, based on its history of tasks solved so far, targeting areas it has not yet mastered. Next, it builds or refines a scaffold suited to that task — the instructions, tools, and solution strategy needed to approach the problem. Finally, it generates an actual solution rollout on top of that scaffold.

Each of these three stages is rewarded according to three criteria: whether the task and scaffold are actually gradeable (validity), whether the difficulty is right at the edge of what the model can currently solve (frontier difficulty), and whether the task doesn't overlap with ones used in previous training (novelty). The difficulty criterion in particular targets a success rate of 0.2, giving the highest reward to tasks the model succeeds at roughly once in five attempts. As the model grows accustomed to a task and its success rate rises, the reward naturally decreases, pushing the task generator toward harder problems. Task generation, scaffold generation, and solution generation are all jointly optimized using a reinforcement learning method called GRPO.

Editor's view

This announcement arrives amid a summer in which "recursive self-improvement" (RSI) has become one of the most frequently cited terms in the AI industry. Following reports that Google co-founder Sergey Brin has been pushing DeepMind to shift resources toward RSI, Sakana AI said it would apply recursive self-improvement to the physical world of moving robots, and Oumi released a platform that feeds production logs back into training data automatically. Ornith extends this trend, using open-source benchmark figures to back the idea that "a model can build its own problem set."

The point worth noting across generations is the difference between scaffolding and self-improvement. Where the previous generation of models simply solved problems on top of tool sets built by humans, this generation rewrites those tool sets itself while also adjusting task difficulty. It has been repeatedly noted in agentic coding that models relying on fixed scaffolds tend to perform well on familiar patterns but break down on unfamiliar repository structures. If a structure that co-evolves tasks and scaffolds actually closes this gap, the distance between open-weight and closed models in coding-agent tasks could narrow considerably.

Two points stand out for domestic development teams. One is that a model small enough to run on-device — like the 9-billion-parameter mobile version — has reportedly reached scores in the 70-point range on SWE-Bench Verified. This is a weight class worth watching for teams looking to add coding-assist features directly on-device without server costs. The other is the training method itself. Since it reduces the cost of humans having to build problem sets, organizations without the resources to design their own evaluation tasks tailored to a specific codebase may find room to apply a similar approach.

Still, all the evidence currently available comes from a single announcement by the development team. Since its release, the news has drawn over 200 votes on Hacker News and been picked up by overseas tech outlets, but those reports also rely on the same announcement. Since the scores were produced through a self-improvement loop, verification will only be complete once the community loads the weights into an inference engine and reproduces the same numbers independently.