이미지: @allen_ai (X)
Summary
- A Georgia Tech research team used Allen Institute for AI's fully open model Olmo 3 and its training corpus Dolma 3 to trace which types of training documents drove benchmark-specific performance
- SocialIQA (social reasoning) scores turned out to depend more heavily on narrative texts like literature and everyday dialogue, while ARC and MMLU relied more on expository texts such as technical documents and scientific writing
- When the team removed the literature-category documents most strongly linked to social reasoning from the model, SocialIQA scores dropped more sharply than when an equivalent amount was removed at random, confirming a causal relationship
- 연구자
- Glenn Matlin, Chandreyi(Zini) Chakraborty (조지아공대)
- 사용 모델·도구
- 올모3(Olmo 3), 돌마3(Dolma 3), WebOrganizer, OlmoEval, OLMES
- 분석 방법
- 영향함수(influence functions)로 개별 학습 문서와 벤치마크 답변의 연관성 추정
- 표본 규모
- 돌마3 문서 약 12억6000만개 중 576개 범주에서 568만개 표본 추출
- 테스트 벤치마크
- SocialIQA, ARC-Challenge, MMLU 사회과학/STEM
- 핵심 발견
- SocialIQA만 문학·고객상담·Q&A 등 서사형 글에 크게 의존, 나머지는 기술문서·과학 글에 의존
- 인과 검증
- 문학 범주 중 영향력 큰 문서를 삭제하자 무작위 삭제보다 SocialIQA 점수가 더 하락
- 발행
- Ai2, 2026년 8월 21일
Removing literature broke social reasoning
The ability to read human emotions, intentions, and everyday moral choices—so-called social reasoning—depends on what texts a language model was trained on, according to a Georgia Tech research team. The team identified the documents in the literature category that had the greatest influence on the model's training, then removed them. The result: scores on SocialIQA, a benchmark measuring understanding of social situations, dropped more sharply than when an equivalent amount of data was removed at random. By deleting the data and observing the effect, the team confirmed that what a model learns actually shapes what it can do.
The experiment was led by Glenn Matlin, a computer science PhD student at Georgia Tech, together with co-author Chandreyi (Zini) Chakraborty. The two have spent years tracking where a language model's social reasoning comes from. Regarding the study they designed, Chakraborty said, "There was one question: where does [that capability] come from."
An experiment only possible with a fully open model
The method the team used is called influence functions—a technique that estimates how much each individual training document contributed to a model's answer on a specific benchmark question. Summing these estimates by document type reveals which kinds of text, whether literature or technical documentation, underpin a given capability. But this method only works if it can be confirmed that the model actually trained on those documents.
Most language models offer no way to verify this. Even models commonly labeled "open source" often release only the finished weights while withholding what data was used to train them and how. Olmo 3, built by the Allen Institute for AI, is different: its training data, Dolma 3, along with intermediate checkpoints and evaluation tools, are all fully public. That's why the team chose it. As Matlin explained, "My research goal is to draw a map that traces a model's behavior back to the training data that produced it."
Splitting four benchmarks across 5.68 million documents
The team drew on five open tools from the Olmo ecosystem: Olmo 3 itself, the Dolma 3 training data, WebOrganizer (a labeling system that classifies data by topic and format), OlmoEval (which supplies the benchmarks), and OLMES (which standardizes scoring criteria). Since Dolma 3 contains roughly 1.26 billion documents, analyzing every single one was impossible. Instead, the team sampled 5.68 million documents across 576 document categories and aggregated each document's influence on benchmark answers into category-level scores.
They applied this process to four benchmarks—SocialIQA, ARC-Challenge (reasoning ability), and the social science and STEM subsets of MMLU (factual knowledge)—generating a unique "influence profile" for each.
Not "social vs. science" but "narrative vs. expository"
Comparing the four profiles produced a less straightforward result than expected. The team initially anticipated a split between social knowledge and scientific knowledge, but in practice, MMLU's social science knowledge behaved more like STEM knowledge and reasoning ability than like social reasoning. What stood out instead was SocialIQA. Answers on this benchmark relied heavily on Dolma categories with strong narrative and conversational qualities—literature, everyday social life, customer service, and Q&A forums. The other three benchmarks leaned more on expository text such as technical documents and scientific writing.
"Categories rich in dialogue and social or emotional language influenced social reasoning far more strongly than they did any other benchmark," Chakraborty said. This pattern wasn't limited to social reasoning. Conversational, relationship-centered text showed greater influence on both reasoning benchmarks (SocialIQA and ARC) than on the knowledge benchmarks—suggesting conversational data may contribute not just to teaching social situations but to reasoning ability itself.
The team was careful to note that this doesn't reduce to a simple formula like "add more literature, get better social reasoning." Rather, once it's confirmed which data actually underpins which capability, that finding can be used to test how the model changes when that data is scaled up or down.
Why this research is possible now
On August 6, the Allen Institute for AI announced in Ai2 Expands Collaboration with Hugging Face to Accelerate Open Science that it had tripled its storage capacity on the Hugging Face Hub to about 2 petabytes. The move removed download speed limits on large datasets and multi-checkpoint models. The models and datasets the Allen Institute for AI has released are reported to have been downloaded more than 50 million times since spring 2024. This Georgia Tech study is a direct product of that open infrastructure.
"AI safety researchers need to challenge conventional wisdom and try entirely new auditing methods, and that would be impossible if researchers had to depend solely on the model assets of private labs," Matlin said. "The Allen Institute for AI didn't just release a model—they opened up the entire scientific stack needed for outside teams to investigate that model and reproduce it from scratch."
Checking the code and data directly
The team has posted its experiment code and detailed results in the github.com/HCAI-LAB-gt/capabilibara repository and on the capabilibara space on huggingface.co. The paper detailing the methodology is available via the study design posted on arXiv, and questions can be directed to the team's Discord server. At this stage, the release consists of code and results without a dedicated app or web UI—reproduction requires downloading the repository directly.
Editor's view
What makes this study compelling isn't the finding itself but the process required to reach it. Asking "what data produced this answer" is, in principle, impossible for closed models like ChatGPT or Claude—since their training data isn't public, all one can do is guess. Olmo 3, by contrast, has its weights, training documents, intermediate checkpoints, and scoring criteria all open, allowing two outside graduate students to independently carry out an audit on par with what a major corporate lab might do. That's a different kind of contribution than simply building a good model.
In practical terms, the real value of this research lies not in the conclusion that "adding literature improves sociability" but in the methodology itself. Teams doing fine-tuning or data curation domestically can now, before blindly scaling up data volume to boost a specific benchmark score, first check which types of text are actually linked to that capability. That said, the method demands considerable computation—sampling 5.68 million documents and recalculating influence for each benchmark. It's better suited as a methodology for organizations training their own foundation models to inform data composition strategy, rather than something to apply directly to small-scale fine-tuning jobs involving just a few gigabytes of data.
More studies auditing open models in similar ways are likely to emerge in coming months. Combined with the Allen Institute for AI's recent move to triple its Hugging Face Hub storage, a trend appears to be taking hold in which fully open models become the standard benchmark for academic auditing.
Comments