Entity tracking emerges in sub-billion parameter language models and exceeds human performance in naturalistic narratives

arXiv:2608.180832026-08-20

Understanding language requires tracking entities across discourse - i.e., knowing where things are and how they change, even when not explicitly stated. Whether language models perform such tracking in a human-like fashion remains unclear, in part because existing evaluations rely on artificial tasks, far removed from natural language comprehension, and lack comparisons to humans. Here, we evaluate entity tracking in both language models and human

Authors · Karolina Dro\.zd\.z, Micha Heilbron