Softmax Normalization

Softmax normalization is a way of taking several scores and turning them into proportions that show how plausible each one is compared to the others.

Say you graded several exam answers with points. The raw points are hard to compare directly, but if you rescale everything so the total equals 100% and express each answer's share of that total, you can compare them at a glance. Softmax normalization works the same way: it stretches larger scores to stand out more and shrinks smaller scores further down, then scales everything so the total always comes out to 100%.

AI models often use this when they have to pick one option among several. For example, a model judging whether a sentence is risky might score the two candidates "yes" and "no," then run softmax normalization on those scores to get two numbers that always add up to 100%, such as 83% for yes and 17% for no. The model then compares this probability against a threshold to make its final call.

The article explains that Mistral AI's safety classifier, Ships trail, applies softmax normalization to the "yes" and "no" tokens to produce its scores. A common misunderstanding is that this step doesn't make the judgment more accurate—it's simply the final step that dresses up already-computed scores to look like probabilities.