Menti-Bench

Menti-Bench is like a test that asks an AI, "What does this person think is happening right now?" If a cup is secretly moved from the table to a cupboard while someone is out of the room, that person still believes the cup is on the table. Menti-Bench gathers 448 scenes like this to check whether an AI picks the correct answer by reasoning about a character's beliefs, intentions, and emotions -- not just the physical situation as it actually is.

The scenes are split into 320 text-only cases, 100 that combine images with a story, and 28 that mix audio and video, and most involve two or more characters. Each item comes with six answer choices and a human-written explanation for the correct answer, so you can compare not only whether a model gets the answer right but why.

The benchmark was built because existing world-simulation techniques tend to render object positions and movements accurately while missing what the people inside those scenes actually believe or want. If a robot or an assistant program can't read these mental states, it may look convincing on the surface yet act in the wrong way in real situations.

Articles introduce it with phrasing like "we built Menti-Bench," describing it as a newly created test set for evaluating world-simulation approaches that account for mental states. The scores reported for OpenAI and Anthropic models on it show that this is a benchmark released by a research team, not a product tied to any single company. One point that's easy to miss: Menti-Bench scores came out much higher for methods that jointly reason about mental and physical states than for methods that simply sample multiple answers and pick the majority vote.