Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

arXiv:2608.196892026-08-21

AI模仿人类做选择时，只看它是否'猜中标准答案'是一种错误的评判方式

用大语言模型（LLM）模拟人类社会行为时，以往的做法是检查模型选的答案是否和某个真人当时选的答案完全一致。但人的行为本身就带有随机性，同一个人在同样情境下也可能做出不同选择，所以记录到的那一个答案只是背后一整个概率分布中随机抽到的一次结果。研究者从数学上证明了旧方法为何失效，并提出了新训练方法SALT和配套评测基准SubjSim。

他们做了什么

提出了'主观性系数'这一基于熵（衡量不确定性的指标）的数值，用来区分编程等只有唯一正确答案的客观任务，和社会模拟等答案本身就会因人而异的主观任务
从理论上证明了现行做法——按准确率评测模型是否选中人类记录的唯一答案、并训练模型去复现这一个答案——随着任务主观性增强会越来越不可靠
提出SALT（主观性自适应软标签训练）：把语义上相近情境的真实答案汇总成一个概率分布式的软标签，并根据每个情境估计出的主观程度，自动调整汇总邻域的大小
构建了新基准SubjSim，由193名标注者回答100道主观问卷题，共产生19300条情境-答案数据；训练时模型和现实场景一样只看到单一答案，但评测时会与完整的概率分布对比
以Qwen3-8B为基础模型的实验显示，相比标准监督微调（SFT），SALT将KL散度降低77.6%、JSD降低45.9%、TVD降低31.2%、MMD降低52.7%，且在主观性越高的问题上提升幅度越大，最高达96.9%

Figure 1: Subjectivity-stratified comparison between SFT and SALT on SubjSim, over low-, mid-, and high-subjectivity questions. All metrics are divergences (lower is better), and percentages give SALT’s relative change over SFT.

Figure 2: Ablation over neighbor selection. Top-N varies N (teal, bottom axis); fixed Keff substitutes a constant for K^eff(x) (navy, top axis). Both share the y axis and reference lines; SFT lies far above the plotted range and is reported as text.

Table 1: Main results on SubjSim. All metrics are divergences (lower is better). Best values are in bold on a blue background; second-best values are underlined on a sand background.
	Economy	Political	Technology
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.6285	0.2851	0.5735	0.6346	4.1453	0.3044	0.6206	0.6586	4.2015	0.2547	0.5335	0.5502
SFT	1.3543	0.1542	0.3951	0.3283	1.2865	0.1568	0.4145	0.3167	1.3409	0.1537	0.3956	0.3215
DPO	6.9962	0.2752	0.5573	0.6049	7.2926	0.2834	0.5816	0.5861	6.9174	0.2580	0.5305	0.5450
PPO	4.0161	0.2672	0.5495	0.5863	4.4698	0.2726	0.5724	0.5684	4.2959	0.2502	0.5251	0.5310
DSA	3.2032	0.2332	0.4920	0.4337	3.4679	0.2442	0.5161	0.4436	3.4744	0.2337	0.4888	0.4275
SALT (Ours)	0.3129	0.0892	0.2839	0.1640	0.2535	0.0724	0.2473	0.1247	0.2817	0.0815	0.2736	0.1466
	Social	Culture	Health
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.2286	0.2720	0.5562	0.6079	3.0456	0.2445	0.5175	0.5646	4.7412	0.2822	0.5737	0.6467
SFT	1.3259	0.1588	0.3977	0.3354	0.8578	0.1209	0.3459	0.2739	1.5169	0.1566	0.3886	0.3286
DPO	6.8092	0.2608	0.5289	0.5629	5.6298	0.2385	0.5050	0.5398	6.5058	0.2501	0.5152	0.5366
PPO	4.2388	0.2524	0.5210	0.5471	3.5294	0.2278	0.4909	0.5180	4.4975	0.2409	0.5059	0.5215
DSA	3.2049	0.2334	0.4963	0.4537	4.1157	0.2623	0.5259	0.5762	3.8155	0.2433	0.4889	0.4561
SALT (Ours)	0.3168	0.0905	0.2863	0.1656	0.2593	0.0746	0.2652	0.1548	0.2995	0.0859	0.2793	0.1601
	Environment	Education	ALL
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	5.3499	0.3091	0.6199	0.6506	6.1097	0.3662	0.6842	0.8295	4.2801	0.2851	0.5745	0.6350
SFT	0.8800	0.1373	0.3837	0.2685	2.0897	0.2324	0.5246	0.4818	1.2871	0.1524	0.3953	0.3194
DPO	7.2403	0.3099	0.6163	0.6573	8.4950	0.3276	0.6291	0.7074	6.8819	0.2690	0.5497	0.5774
PPO	3.5411	0.2914	0.6035	0.6206	5.1310	0.3337	0.6447	0.7245	4.1393	0.2588	0.5403	0.5578
DSA	3.0895	0.2116	0.4613	0.3496	2.7515	0.2260	0.4870	0.4152	3.3904	0.2343	0.4937	0.4418
SALT (Ours)	0.2983	0.0870	0.2705	0.1407	0.3866	0.1095	0.3229	0.1947	0.2880	0.0825	0.2720	0.1510

Figure 3: Sensitivity of SALT to the radius scale C (divergences, lower is better).

Figure 4: Per-domain results on the Economy domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 2: Mapping between theoretical concepts, notation, and their SubjSim implementation.
Concept	Notation	SubjSim realization	Protocol role
Decision context	x=(u,s)	Annotator persona paired with a survey question; each persona has 30 demographic attributes.	Input to training and evaluation.
Action space	𝒜⁡(x)	Candidate response options for the survey question.	Common support for distributional evaluation within each question.
Latent response-propensity target	ϕ∗(⋅∣x)	Probability-ball empirical distribution ϕ^uball(⋅∣x), used as an elicited distributional proxy rather than a repeated-choice frequency.	Hidden during training; used only as the evaluation target.
Hard observation	a⁡(x)	Modal response under ϕ^uball(⋅∣x).	Hard-label proxy for SFT, DPO, PPO, and SALT targets.
Action-space group	𝒢g	Persona-question contexts from the same survey question.	Restricts aggregation to comparable candidate options.
SALT neighborhood	𝒩⁡(x)	Nearest contexts within the adaptive embedding radius, using Qwen3-embedding-8b embeddings.	Defines which hard observations SALT pools.
Effective action count (oracle)	Keff∗(x)	Effective action count of the true response distribution at the context; never observed.	Appears in the bound of Theorem 1 and in the oracle radius.
Effective action count (model-based)	K^eff(x)	The same functional applied to the model’s own distribution, Keff(ϕθ(⋅∣x)).	Sets the adaptive radius without using the hidden target.
Global baseline	Global-Freq	Question-level frequency of hard labels.	Ablation that removes persona conditioning.

Table 3: Main training and method-specific hyperparameters for all methods.
		Policy models	PPO aux.
Group	Parameter	SALT	SFT	DPO	PPO	DSA	RM
General	Epochs	4	5	4	4	4	4
Global batch size	128	128	96	128	128	128
Learning rate	5e-6	5e-6	5e-7	1e-6	5e-6	1e-5
LR schedule	Cosine	Cosine	Cosine	Cosine	Cosine	Cosine
Warmup ratio	0.1	0.1	0.1	0.1	0.1	0.1
Optimizer	AdamW	AdamW	AdamW	AdamW	AdamW	AdamW
Max sequence length	1024	1024	1024	1024	1024	1024
DPO	β	—	—	0.07	—	—	—
Loss type	—	—	Sigmoid	—	—	—
Label smoothing	—	—	0.0	—	—	—
FTX coefficient	—	—	0.0	—	—	—
PPO	ϵclip	—	—	—	0.2	—	—
Target KL	—	—	—	6.0	—	—
Initial KL coeff.	—	—	—	0.05	—	—
Sampling temperature	—	—	—	0.7	—	—
Top-p	—	—	—	0.9	—	—
SALT	Scale factor C	0.2	—	—	—	—	—
PCA variance threshold	0.90	—	—	—	—	—
Softmax temperature	1.0	—	—	—	—	—

Figure 5: Per-domain results on the Political domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 4: Point estimates and 95% paired-bootstrap confidence intervals on the full test set (1,000 resamples over contexts). All metrics are divergences (lower is better).
Method	KL	JSD	TVD	MMD
Pretrained	4.2801[4.1628,4.3949]	0.2851[0.2785,0.2912]	0.5745[0.5646,0.5833]	0.6350[0.6161,0.6524]
SFT	1.2871[1.2281,1.3439]	0.1524[0.1474,0.1573]	0.3953[0.3869,0.4034]	0.3194[0.3060,0.3324]
DPO	6.8819[6.7226,7.0539]	0.2690[0.2625,0.2752]	0.5497[0.5396,0.5590]	0.5774[0.5595,0.5935]
PPO	4.1393[4.0256,4.2464]	0.2588[0.2524,0.2647]	0.5403[0.5302,0.5490]	0.5578[0.5407,0.5743]
DSA	3.3904[3.2932,3.4851]	0.2343[0.2284,0.2403]	0.4937[0.4857,0.5019]	0.4418[0.4281,0.4567]
SALT (Ours)	0.2880[0.2749,0.3009]	0.0825[0.0788,0.0863]	0.2720[0.2653,0.2787]	0.1510[0.1440,0.1582]
Top-N (N=3)	0.3653[0.3497,0.3809]	0.0925[0.0886,0.0964]	0.3078[0.3011,0.3144]	0.1921[0.1833,0.2014]
Fixed Keff (K=2)	0.3152[0.3017,0.3290]	0.0900[0.0861,0.0939]	0.2839[0.2769,0.2909]	0.1656[0.1586,0.1734]
Global-Freq	0.3879[0.3713,0.4036]	0.0933[0.0894,0.0970]	0.3065[0.2993,0.3130]	0.1886[0.1798,0.1973]

Figure 6: Per-domain results on the Technology domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 5: Source distribution of the 100 subjective questions in SubjSim.
Source	Count	Proportion
ATP	35	35.0%
GSS	19	19.0%
WVS	14	14.0%
CGSS	9	9.0%
ANES	9	9.0%
ESS	9	9.0%
ISSP	5	5.0%
Total	100	100%

Table 6: Examples of Demographic Attributes and Subjective Questions
Domain	Question	Response Options
Demographic Attributes
Demographic	What is your current academic year?	1. Freshman/Sophomore; 2. Junior/Senior; 3. Master’s; 4. PhD; 5. Not a student
	Are you an only child?	1. Yes; 2. No
	What is your current employment status?	1. Full-time; 2. Part-time; 3. Self-employed; 4. Retired; 5. Homemaker; 6. Student; 7. Unemployed; 8. Other
Subjective Questions
Political System	Over the next 30 years, which social trend concerns you the most?	1. AI replacing jobs; 2. Social stratification; 3. Misinformation; 4. Weakening family structures
	Among occupational groups, which do you trust the most?	1. Healthcare/education; 2. Law enforcement; 3. Business; 4. Non-profits
Education	Who should ensure young people acquire skills for good jobs?	1. Government; 2. Employers; 3. Education system; 4. Individuals
	Which quality is most important for children to learn?	1. Socially adept; 2. Obedient; 3. Hard work; 4. Helpful; 5. Independent thinking
Social Relations	What do you most often do in free time?	1. Social entertainment; 2. Leisure; 3. Self-improvement; 4. Exercise
	Which is more important: considerate or proper behavior?	1. Considerate; 2. Proper
Health Well-being	What is your view on vaccination?	1. Mandatory; 2. Personal choice; 3. Cautious; 4. Not mandatory
	What is the biggest problem with the healthcare system?	1. Over-prescription; 2. High cost; 3. Drug safety; 4. Uneven resources
Economy Labor	Do you prefer full-time employment?	1. Yes; 2. No
	What is the biggest challenge if changing jobs?	1. Salary/benefits; 2. Skill competitiveness; 3. Few opportunities; 4. Not difficult
Values Culture	Do you believe in life after death?	1. Yes; 2. No
	Which value should society prioritize?	1. Equal opportunity; 2. Individual freedom; 3. Social order; 4. Tradition
Technology & Society	What is your view on genetically modified foods?	1. Healthier; 2. More harmful; 3. No difference
	Who should protect personal information online?	1. Companies; 2. Individuals; 3. Public institutions
Environment & Energy	Which position do you lean toward on climate change?	1. Existential crisis; 2. Politicized; 3. Long-term issue; 4. Natural cycles
	Regarding energy, what are you most concerned about?	1. Prices; 2. Outages; 3. Fossil fuel reliance; 4. Natural disasters

为什么重要

随着LLM被越来越多地用于模拟民意、消费选择、政治态度等具有个体差异的行为，这项研究表明当前普遍采用的'对答案'式评测和训练方法可能从根本上就是错的。任何想用LLM做社会模拟、市场调研或民意建模的人，都需要重新思考评测标准和训练方式是否真的反映了人类行为的多样性。

本文术语

LLM · 大语言模型，一种能理解和生成文本的人工智能系统
主观性系数 · 衡量某个问题下人们答案分散程度的数值，基于信息熵计算
SALT · 本文提出的新训练方法，通过汇总相似情境的答案生成概率分布式软标签
SubjSim · 本文构建的新基准数据集，193人回答100道主观问题，并记录了完整的概率分布用于评测
KL散度/JSD/TVD/MMD · 几种衡量两个概率分布之间差异程度的数学方法

论文原文摘要（英文）

LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evaluate the fidelity of LLM-simulated human behavior and optimize LLMs toward it. Prevailing practice evaluates by accuracy, checking whether the model selects the single response observed from a human, and trains the LLM to reproduce this hard label. However, human behavior is inherently subjective: the same person in the same situation may reasonably act differently, so an observed response is only one draw from an underlying response distribution, rendering accuracy-based evaluation unreliable and hard-label training misleading. To address these problems, we first introduce the subjectivity coefficient, an entropy-based quantity distinguishing objective tasks such as coding from subjective ones such as social simulation, and use it to systematically analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows. Based on the subjectivity coefficient, we propose Subjectivity-Adaptive soft-Label Training (SALT): it pools observed outputs from semantically nearby inputs into soft distributional labels, with an aggregation radius adapted to the estimated subjectivity of each input; in the near-objective limit the neighborhood shrinks, so SALT naturally falls back to standard single-label training. Moreover, since existing datasets record only single observed responses and cannot support distributional evaluation, we construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions. Since real-world data typically provide only a single observation per input, our experiments train models from single observed outputs while evaluating them against the full response distributions, verifying feasibility in realistic settings. Results on SUBJSIM demonstrate the advantages of our method.

作者 · Pei Wang, Xu Chen, Ji-Rong Wen

在 arXiv 阅读

METAL LAB 最新报道

图片来源： Pei Wang et al., arXiv:2608.19689, CC BY 4.0

Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

他们做了什么

为什么重要

本文术语

最新论文

METAL LAB 最新报道