MXFP4

MXFP4 is a way of squeezing the huge number of values inside an AI model into a very small number of digits. Normally, when a model is built, each number is stored with around 16 digits of precision, but MXFP4 shrinks that down to just 4 digits. It's similar to how a high-resolution photo takes up a lot of space, but converting it to a compressed file shrinks the size while losing fine detail.

However, MXFP4 doesn't just uniformly cut down the digits. Instead, it groups similar numbers together, sets a single shared scale (a common reference point) for that group, and then represents each individual number only within that scale. This loses less information than simply shrinking every digit evenly across the board. Doing this greatly reduces the memory and compute needed to run the model, making it possible to run bigger models on the same hardware or operate services more cheaply.

The catch is that squeezing numbers down this way usually also reduces the model's judgment — its ability to reason or calculate. So after converting to MXFP4, a separate recovery step to restore performance is often needed.

Articles have described things like "the student model is half the size and runs on MXFP4" or "verified by quantizing further down to MXFP4 with QAH." A common misunderstanding here is assuming that shrinking to MXFP4 automatically improves performance. In reality, cutting precision down to 4 bits usually degrades performance, and the results reported in such articles are exceptional outcomes achieved only because a new recovery technique was applied alongside the quantization.