Multiverse Computing shrinks a model to 4-bit and gets one smarter than the original

The standard playbook for making an AI model lighter runs in two steps. First comes structural compression — trimming the number of layers or neurons. Then comes quantization, dropping the remaining weights from 16-bit down to 4-bit. Both steps cut memory and compute significantly, but they usually cost you the capabilities people actually care about: reasoning, math, code generation. That's why real-world deployment pipelines add a third step after compression and quantization — a recovery phase often called "healing." Multiverse Computing, a company focused on model compression research, introduced a redesigned version of this healing step in a paper posted to the Hugging Face blog on August 25. They call it Quantization-Aware Healing (QAH).

Where the two existing healing methods fall short

Until now, two healing methods have dominated the field. The first is Quantization-Aware Training (QAT), which inserts fake quantization operations into the forward pass and keeps fine-tuning against a task loss function. The problem is that you're running a model that's already been through expensive pretraining, supervised fine-tuning, and RLHF back through a noisy, low-precision forward pass all over again. That's costly, and if you push training past the optimal point, it can actually become unstable.

The other method, Quantization-Aware Distillation (QAD), skips that repeated training. Instead, it transfers the output distribution of a frozen, full-precision teacher model to the student using a KL-divergence loss — a measure of how different two probability distributions are. This works fine when only quantization changes. But once structural compression is added — when the number of layers, heads, or neurons themselves shrinks — it runs into trouble. There simply is no independent full-precision model at the exact same size as the compressed architecture. The only teacher left is the bfloat16 checkpoint recovered after compression, which is itself an approximation of the original. That creates a ceiling: the student model's accuracy can never exceed what that recovered checkpoint achieves.

Learning straight from the original

QAH changes exactly one thing: instead of distilling from the recovered checkpoint, it transfers knowledge directly from the original, pre-compression model. Teacher and student don't even share an architecture. The teacher is the full-precision model at its original size; the student runs at half that size in MXFP4. According to Multiverse Computing, because the teacher's output distribution is defined independently of its structure, differences in size or shape don't get in the way of knowledge transfer. The student never sees ground-truth labels — it trains purely by matching the teacher's output distribution via KL divergence. This reframes what quantization even means in this pipeline. Where it used to be a loss-inducing step tacked on after healing was done, in QAH it becomes a second distillation pass run directly against the original teacher.

Benchmarks: 4-bit beats 16-bit

Multiverse Computing tested this on GPT-OSS 120B, OpenAI's open-weight model. They structurally compressed it down to 60B parameters, recovered it in bfloat16, then applied QAH to quantize it further down to MXFP4. The comparison point was a bfloat16 checkpoint at that same 60B architecture — the best full-precision version that architecture could possibly produce.

Benchmark	QAH 4-bit vs. bfloat16 60B	Notes
AA-LCR (long context)	+7.4 points	Largest gain
AIME 2025 (math)	+5.6 points	Second-largest gain
MMLU-Pro	Behind by under 1.5 points	One of two losses
SciCode	Slightly behind	One of two losses
Overall	Ahead on 7 of 9	Beats the bfloat16 original

The comparison against the pre-compression 120B teacher model is just as striking. Despite having half the teacher's parameter count and roughly a quarter of its weight memory, the QAH model scored 66.5 on LiveCodeBench, edging out the teacher's 66.0, and came within 1.6 points of the teacher on GPQA Diamond (67.4 vs. 69.0). The biggest remaining gap was on AA-LCR, which tests extremely long context handling — and according to the company, that's precisely the hardest kind of capacity to recover once compression takes it away.

Seven times faster than QAT, and it doesn't fall apart afterward

To isolate the effect of the loss function itself, the team ran a head-to-head test between QAH and QAT under matched conditions, quantizing a GPT-OSS 9B model to MXFP4. They tracked average scores across MMLU-Pro, LiveCodeBench, and GPQA Diamond over the course of training.

Method	Step to peak	Peak score	What happens after
QAH	~100 steps	54.9 (peak)	Stays within 2 points of peak
QAT	~700 steps	54.6 (peak)	Drops nearly 19 points by step 1,200

The two methods' peak scores are essentially identical — 54.9 versus 54.6. The difference is how fast they get there, and what happens afterward. QAH hits its peak in about 100 steps, nearly 7 times faster than QAT, and holds that level steady. QAT, by contrast, starts collapsing the moment it passes its peak, falling into the mid-30s by step 1,200. In practice, that means deploying a QAT checkpoint requires a separate validation process just to find the exact moment to stop training before performance breaks down. QAH avoids that problem: once the student has caught up to the teacher's distribution, there's no incentive left for it to keep drifting, so a fully trained checkpoint can be shipped to production as-is, according to Multiverse Computing.

How much memory and compute does it actually save

The gains aren't limited to accuracy — the efficiency that was the whole point of compression in the first place holds up too. The 4-bit QAH model uses roughly a quarter of the weight memory of a student model trained in bfloat16. Since the parameter count is also half that of the 120B teacher, per-token compute drops by roughly half as well. The company says that applying this approach to other model families currently served in bfloat16 could cut per-token compute by up to 8x, once the reductions from smaller parameter count and lower precision are combined.

This isn't the first attempt to shrink models while preserving performance. A similar concern shows up in the article Liquid AI open-sources Pipette, an on-device benchmark.

Editor's take

The paper's core claim — that quantization isn't a tax you pay in lost accuracy but a second chance to teach the model — reframes how the whole compression pipeline should be understood. The industry has long treated the three-step process of compress, quantize, and recover as damage control: a way to minimize a loss that's assumed to be unavoidable. QAH redefines the recovery step as a second training opportunity, one where the model learns directly from its original teacher again. If this idea generalizes to other model families, beating the original on a benchmark table might stop being the exception for compressed models and start being the norm.

If you've ever deployed a small open-weight model in production, you'll know why this result stands out. Cutting parameters in half and dropping to 4-bit used to cause visible degradation on long-context tasks and math problems, every time. Yet in this case, those are exactly the two areas — AA-LCR and AIME 2025 — where the QAH model improved the most. The recovery process ended up targeting precisely the capabilities that compression tends to damage most.

For teams in Korea running their own sLLMs or serving models on-premise, this result is worth reading as a price tag. If the GPU memory needed for the same performance drops to a quarter, and per-token compute falls by more than half, that translates directly into fewer inference servers for the same workload — or more traffic handled on the same budget. The catch is that QAH requires precomputing logits from the pre-compression original model to use as the teacher, so teams without access to that original model can't easily replicate this on their own. It's a method best suited, for now, to teams already working with open-weight models.

QAT has been the industry standard for a long time, so it's likely that other open-weight model families will see reproduction attempts using QAH within the next few months. If those results come out similar to what we're seeing here with GPT-OSS, the assumption that "quantization always costs you performance" may not stay in the industry playbook much longer.