Adding a 'mind' variable to world models boosted accuracy from 63 to 88

The paper points out that existing world models like Sora, Genie 3, JEPA, and Marble only encode physical layers as state — objects, positions, movement, occlusion. What a person in that world believes, wants, or considers socially appropriate never enters the state representation at all. That's a serious problem for service robots, medical assistants, or collaborative agents, since most human behavior flows from mental states that aren't directly visible. A scene where a cup gets moved to the cupboard can be reproduced flawlessly on the physical level, but only a model that separately tracks that person's "belief" would know they'll still go looking for the cup on the table.

MWM: a framework that adds the 'mind' variable

The framework the researchers released on GitHub is called, fittingly, "Mental World Modeling (MWM)." It layers variables like belief, attention, goals, intention, emotion, norms, and social relationships on top of existing world models. The structure has the observed agent seeing only a partial, first-person slice of the scene, while the world model itself holds the full state. The paper splits every action into a physical vehicle (speaking, pointing, grasping) and a mental content (comforting, deceiving, refusing). The same physical act — nudging a cup across a table — could be an apology, a trick, or an act of kindness, and only the mental variable can tell those apart. The researchers aren't claiming to simulate consciousness; they treat mental states as hypotheses inferred from behavior and context, and insist the model should keep that uncertainty visible rather than paper over it.

A six-stage pipeline and the results

To test the theory, the researchers built MENTIS, a modular pipeline that runs without any additional training. It interprets the scene, renders a first-person viewpoint, splits candidate actions into physical and mental components, and simulates both states in parallel. Each branch is scored on three criteria — physical plausibility, mental consistency, and social appropriateness — before a single outcome is deterministically selected at the end. Every stage leaves behind a machine-readable intermediate result, so it's possible to trace back exactly where things went wrong.

For evaluation, they built a new benchmark called Menti-Bench, made up of 448 decision scenes: 320 text scenarios, 100 image-based stories, and 28 audio-video clips. Seventy-eight percent of the scenes involve two or more characters interacting. Each scene comes with six answer choices and a human-written explanation of the correct answer.

Method	Description	F1 score
Direct answer	Model answers in one shot	63.3
Self-consistency	Same question asked 6 times, majority vote	77.9
MWM pipeline	Combined mental-physical state simulation	87.9
Human	Same protocol applied to humans	98.5

The researchers tested eight language models in total, including OpenAI's GPT-5.6-Sol and GPT-4.1, along with Anthropic's Claude Fable 5, Claude Opus 4.8, and Claude Haiku 4.5. The weakest model, GPT-4.1, scored 84.9 with MWM applied — beating even the strongest model, GPT-5.6-Sol, using self-consistency (83.6). In other words, simply sampling more answers and taking a majority vote can't close this gap.

What the ablations revealed

The researchers removed pieces of the pipeline one at a time to see how much each actually contributed.

Component removed	Score drop
Mental channel removed	-12.1
Physical channel removed	-16.5
Physical and mental transitions predicted independently (not coupled)	-6.4

Mental modeling had its biggest payoff exactly where the theory predicted it would. In scenes where characters interact with each other, F1 scores rose by 26.4 points, compared with just 14.0 points in object-centric scenes. Weaker base models also gained more from this structured approach: GPT-4.1 showed a 28-point gap between MWM and direct answering, versus 21 points for GPT-5.6-Sol.

Where the gap with humans comes from

To trace where the remaining gap with human performance was coming from, the researchers swapped out intermediate pipeline stages one by one with human-written ground truth. The biggest gain came from perfect state-transition prediction (+3.5 points), followed by perfect initial state (+2.8) and perfect observation (+1.7). Replacing every intermediate stage with ground truth pushed the pipeline up to 97 points. The researchers concluded that roughly 80% of the remaining gap comes from prediction errors in the intermediate stages — particularly in simulating how the combined physical-mental state transitions over time. Describing the current state is comparatively easy; the hard part is predicting how a combined physical-mental world actually changes.

Where this fits in the world-model debate

Demis Hassabis, who recently stepped down from his operational leadership role at Google DeepMind, now spends most of his research time on world models and has predicted a "ChatGPT moment" for the field. Startups like Odyssey have already pulled in hundreds of millions of dollars from investors. But what counts as a "world model" is still far from settled. A recent international research team led by Peking University proposed a narrower definition that would exclude text-to-video models like Sora entirely, on the grounds that they lack any feedback loop with the real world. Yann LeCun has long argued that generative approaches are a dead end and championed abstract representation learning instead (JEPA) — yet this paper groups Sora, Genie, and JEPA together and levels the same criticism at all three: none of them account for mental states.

The challenge of handling mental states also touches on an area where language models have long struggled. Research teams from Meta FAIR, the University of Washington, and Carnegie Mellon have already shown that language models fail on demanding theory-of-mind tests and are weaker at inferring beliefs than at tracking the state of the physical world. The MWM framework imposes these mental states externally through a preprocessing pipeline — but there are also signs that something similar may be emerging naturally inside the models themselves. Anthropic has reported finding what amounts to an internal scratchpad in Claude, containing word-like thoughts that never surface in the output, and that multi-step reasoning breaks down without it.

To look into this yourself

Because the MWM framework and the MENTIS pipeline sit on top of existing language models without requiring any separate training run, researchers can pull the code from the github.com/mental-world/Mentis repository and apply it directly to whatever model they're already using.

Editor's take

What makes this paper interesting is that a better procedure beat a bigger model. The fact that adding a mental-state variable to the weakest model outperformed repeated sampling on the strongest one raises a real question about an industry that's currently pouring money into model size and inference-time repetition to push benchmark scores up. For teams here building robots or counseling agents, layering a structured preprocessing step like this on top of the model they already have could be a faster, cheaper win than waiting for the next model upgrade.

At the same time, there's something striking about how this research lumps in Yann LeCun's own JEPA with the models it critiques. His case for abstract representation learning — built on the argument that generative models are a dead end — gets flagged for the exact same blind spot: neither approach accounts for what's going on in a person's mind. A field that's been fixated on physical-simulation accuracy now looks like it's shifting toward a completely different axis of competition: who can best model what people are thinking.

The number worth watching practically is the gap analysis. If 80% of the remaining error comes from predicting state transitions, then the next edge in this field won't go to whoever describes the current state best — it'll go to whoever best predicts the moment that state changes. Expect follow-up papers and benchmarks in the coming months to dig specifically into state-transition prediction itself.