AA-Omniscience

AA-Omniscience isn't a test of how much a model knows — it's a test of how honestly it can tell the difference between knowing and not knowing. It asks 600 questions spanning six knowledge domains: correct answers earn points, confidently wrong answers lose points, and admitting "I don't know" (by not answering) costs nothing.

In an ordinary exam, leaving a question blank and answering it wrong both get zero points. This test works differently. It punishes confidently wrong answers the most. Because of this design, a model can see a big score jump just by learning to stay silent on questions it's unsure about, even if its actual knowledge hasn't grown at all.

That's why this score shouldn't be read on its own — you also need to check the rate at which the model actually attempted to answer. If that attempt rate drops sharply, a higher score may not reflect broader knowledge at all, but rather a shift toward staying quiet when unsure.

The Upstage Solar Pro 4 coverage reports that its AA-Omniscience score rose from -53 to -1, but notes this improvement came not from knowing more but from the answer-attempt rate falling from 92% to 41%, which in turn lowered the hallucination rate (88%→24%). Taken at face value, the score alone might suggest the model "got smarter," but the real story is closer to "it learned to hold its tongue when unsure." In an article comparing search APIs, AA-Omniscience appears as one of three benchmarks, serving as a baseline for how much a model knows even without search.