Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

arXiv:2608.196892026-08-21

When AI mimics human survey answers, checking if it 'got the answer right' is the wrong way to grade it

LLMs used to simulate human social behavior have been evaluated by whether they pick the exact same single answer a real person gave. But people are inherently inconsistent: the same person in the same situation could reasonably answer differently, so one recorded answer is just one random draw from a wider spread of possible answers. The researchers prove mathematically why this old method breaks down, and propose SALT, a training method that builds soft probability-distribution targets from similar contexts, tested on a new benchmark called SubjSim.

What they did

Introduced a 'subjectivity coefficient,' a number based on entropy (a measure of unpredictability) that separates objective tasks like coding (one correct answer) from subjective ones like social simulation (many reasonable answers)
Formally proved that the standard practice — grading models by accuracy against one recorded human answer, and training them to reproduce that single answer — gets increasingly wrong as a question becomes more subjective
Proposed SALT (Subjectivity-Adaptive soft-Label Training), which pools answers from semantically similar contexts into a probability distribution, automatically widening or narrowing the pool of neighbors depending on how subjective each context is estimated to be
Built SubjSim, a new benchmark with 193 annotators answering 100 subjective survey questions, producing 19,300 context-answer pairs; models were trained on single answers (as in real-world data) but evaluated against the full probability distributions
Using Qwen3-8B as the base model, SALT cut KL divergence by 77.6%, JSD by 45.9%, TVD by 31.2%, and MMD by 52.7% compared to standard supervised fine-tuning (SFT), with the biggest gains (up to 96.9%) on the most subjective questions

Figure 1: Subjectivity-stratified comparison between SFT and SALT on SubjSim, over low-, mid-, and high-subjectivity questions. All metrics are divergences (lower is better), and percentages give SALT’s relative change over SFT.

Figure 2: Ablation over neighbor selection. Top-N varies N (teal, bottom axis); fixed Keff substitutes a constant for K^eff(x) (navy, top axis). Both share the y axis and reference lines; SFT lies far above the plotted range and is reported as text.

Table 1: Main results on SubjSim. All metrics are divergences (lower is better). Best values are in bold on a blue background; second-best values are underlined on a sand background.
	Economy	Political	Technology
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.6285	0.2851	0.5735	0.6346	4.1453	0.3044	0.6206	0.6586	4.2015	0.2547	0.5335	0.5502
SFT	1.3543	0.1542	0.3951	0.3283	1.2865	0.1568	0.4145	0.3167	1.3409	0.1537	0.3956	0.3215
DPO	6.9962	0.2752	0.5573	0.6049	7.2926	0.2834	0.5816	0.5861	6.9174	0.2580	0.5305	0.5450
PPO	4.0161	0.2672	0.5495	0.5863	4.4698	0.2726	0.5724	0.5684	4.2959	0.2502	0.5251	0.5310
DSA	3.2032	0.2332	0.4920	0.4337	3.4679	0.2442	0.5161	0.4436	3.4744	0.2337	0.4888	0.4275
SALT (Ours)	0.3129	0.0892	0.2839	0.1640	0.2535	0.0724	0.2473	0.1247	0.2817	0.0815	0.2736	0.1466
	Social	Culture	Health
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.2286	0.2720	0.5562	0.6079	3.0456	0.2445	0.5175	0.5646	4.7412	0.2822	0.5737	0.6467
SFT	1.3259	0.1588	0.3977	0.3354	0.8578	0.1209	0.3459	0.2739	1.5169	0.1566	0.3886	0.3286
DPO	6.8092	0.2608	0.5289	0.5629	5.6298	0.2385	0.5050	0.5398	6.5058	0.2501	0.5152	0.5366
PPO	4.2388	0.2524	0.5210	0.5471	3.5294	0.2278	0.4909	0.5180	4.4975	0.2409	0.5059	0.5215
DSA	3.2049	0.2334	0.4963	0.4537	4.1157	0.2623	0.5259	0.5762	3.8155	0.2433	0.4889	0.4561
SALT (Ours)	0.3168	0.0905	0.2863	0.1656	0.2593	0.0746	0.2652	0.1548	0.2995	0.0859	0.2793	0.1601
	Environment	Education	ALL
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	5.3499	0.3091	0.6199	0.6506	6.1097	0.3662	0.6842	0.8295	4.2801	0.2851	0.5745	0.6350
SFT	0.8800	0.1373	0.3837	0.2685	2.0897	0.2324	0.5246	0.4818	1.2871	0.1524	0.3953	0.3194
DPO	7.2403	0.3099	0.6163	0.6573	8.4950	0.3276	0.6291	0.7074	6.8819	0.2690	0.5497	0.5774
PPO	3.5411	0.2914	0.6035	0.6206	5.1310	0.3337	0.6447	0.7245	4.1393	0.2588	0.5403	0.5578
DSA	3.0895	0.2116	0.4613	0.3496	2.7515	0.2260	0.4870	0.4152	3.3904	0.2343	0.4937	0.4418
SALT (Ours)	0.2983	0.0870	0.2705	0.1407	0.3866	0.1095	0.3229	0.1947	0.2880	0.0825	0.2720	0.1510

Figure 3: Sensitivity of SALT to the radius scale C (divergences, lower is better).

Figure 4: Per-domain results on the Economy domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 2: Mapping between theoretical concepts, notation, and their SubjSim implementation.
Concept	Notation	SubjSim realization	Protocol role
Decision context	x=(u,s)	Annotator persona paired with a survey question; each persona has 30 demographic attributes.	Input to training and evaluation.
Action space	𝒜⁡(x)	Candidate response options for the survey question.	Common support for distributional evaluation within each question.
Latent response-propensity target	ϕ∗(⋅∣x)	Probability-ball empirical distribution ϕ^uball(⋅∣x), used as an elicited distributional proxy rather than a repeated-choice frequency.	Hidden during training; used only as the evaluation target.
Hard observation	a⁡(x)	Modal response under ϕ^uball(⋅∣x).	Hard-label proxy for SFT, DPO, PPO, and SALT targets.
Action-space group	𝒢g	Persona-question contexts from the same survey question.	Restricts aggregation to comparable candidate options.
SALT neighborhood	𝒩⁡(x)	Nearest contexts within the adaptive embedding radius, using Qwen3-embedding-8b embeddings.	Defines which hard observations SALT pools.
Effective action count (oracle)	Keff∗(x)	Effective action count of the true response distribution at the context; never observed.	Appears in the bound of Theorem 1 and in the oracle radius.
Effective action count (model-based)	K^eff(x)	The same functional applied to the model’s own distribution, Keff(ϕθ(⋅∣x)).	Sets the adaptive radius without using the hidden target.
Global baseline	Global-Freq	Question-level frequency of hard labels.	Ablation that removes persona conditioning.

Table 3: Main training and method-specific hyperparameters for all methods.
		Policy models	PPO aux.
Group	Parameter	SALT	SFT	DPO	PPO	DSA	RM
General	Epochs	4	5	4	4	4	4
Global batch size	128	128	96	128	128	128
Learning rate	5e-6	5e-6	5e-7	1e-6	5e-6	1e-5
LR schedule	Cosine	Cosine	Cosine	Cosine	Cosine	Cosine
Warmup ratio	0.1	0.1	0.1	0.1	0.1	0.1
Optimizer	AdamW	AdamW	AdamW	AdamW	AdamW	AdamW
Max sequence length	1024	1024	1024	1024	1024	1024
DPO	β	—	—	0.07	—	—	—
Loss type	—	—	Sigmoid	—	—	—
Label smoothing	—	—	0.0	—	—	—
FTX coefficient	—	—	0.0	—	—	—
PPO	ϵclip	—	—	—	0.2	—	—
Target KL	—	—	—	6.0	—	—
Initial KL coeff.	—	—	—	0.05	—	—
Sampling temperature	—	—	—	0.7	—	—
Top-p	—	—	—	0.9	—	—
SALT	Scale factor C	0.2	—	—	—	—	—
PCA variance threshold	0.90	—	—	—	—	—
Softmax temperature	1.0	—	—	—	—	—

Figure 5: Per-domain results on the Political domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 4: Point estimates and 95% paired-bootstrap confidence intervals on the full test set (1,000 resamples over contexts). All metrics are divergences (lower is better).
Method	KL	JSD	TVD	MMD
Pretrained	4.2801[4.1628,4.3949]	0.2851[0.2785,0.2912]	0.5745[0.5646,0.5833]	0.6350[0.6161,0.6524]
SFT	1.2871[1.2281,1.3439]	0.1524[0.1474,0.1573]	0.3953[0.3869,0.4034]	0.3194[0.3060,0.3324]
DPO	6.8819[6.7226,7.0539]	0.2690[0.2625,0.2752]	0.5497[0.5396,0.5590]	0.5774[0.5595,0.5935]
PPO	4.1393[4.0256,4.2464]	0.2588[0.2524,0.2647]	0.5403[0.5302,0.5490]	0.5578[0.5407,0.5743]
DSA	3.3904[3.2932,3.4851]	0.2343[0.2284,0.2403]	0.4937[0.4857,0.5019]	0.4418[0.4281,0.4567]
SALT (Ours)	0.2880[0.2749,0.3009]	0.0825[0.0788,0.0863]	0.2720[0.2653,0.2787]	0.1510[0.1440,0.1582]
Top-N (N=3)	0.3653[0.3497,0.3809]	0.0925[0.0886,0.0964]	0.3078[0.3011,0.3144]	0.1921[0.1833,0.2014]
Fixed Keff (K=2)	0.3152[0.3017,0.3290]	0.0900[0.0861,0.0939]	0.2839[0.2769,0.2909]	0.1656[0.1586,0.1734]
Global-Freq	0.3879[0.3713,0.4036]	0.0933[0.0894,0.0970]	0.3065[0.2993,0.3130]	0.1886[0.1798,0.1973]

Figure 6: Per-domain results on the Technology domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 5: Source distribution of the 100 subjective questions in SubjSim.
Source	Count	Proportion
ATP	35	35.0%
GSS	19	19.0%
WVS	14	14.0%
CGSS	9	9.0%
ANES	9	9.0%
ESS	9	9.0%
ISSP	5	5.0%
Total	100	100%

Table 6: Examples of Demographic Attributes and Subjective Questions
Domain	Question	Response Options
Demographic Attributes
Demographic	What is your current academic year?	1. Freshman/Sophomore; 2. Junior/Senior; 3. Master’s; 4. PhD; 5. Not a student
	Are you an only child?	1. Yes; 2. No
	What is your current employment status?	1. Full-time; 2. Part-time; 3. Self-employed; 4. Retired; 5. Homemaker; 6. Student; 7. Unemployed; 8. Other
Subjective Questions
Political System	Over the next 30 years, which social trend concerns you the most?	1. AI replacing jobs; 2. Social stratification; 3. Misinformation; 4. Weakening family structures
	Among occupational groups, which do you trust the most?	1. Healthcare/education; 2. Law enforcement; 3. Business; 4. Non-profits
Education	Who should ensure young people acquire skills for good jobs?	1. Government; 2. Employers; 3. Education system; 4. Individuals
	Which quality is most important for children to learn?	1. Socially adept; 2. Obedient; 3. Hard work; 4. Helpful; 5. Independent thinking
Social Relations	What do you most often do in free time?	1. Social entertainment; 2. Leisure; 3. Self-improvement; 4. Exercise
	Which is more important: considerate or proper behavior?	1. Considerate; 2. Proper
Health Well-being	What is your view on vaccination?	1. Mandatory; 2. Personal choice; 3. Cautious; 4. Not mandatory
	What is the biggest problem with the healthcare system?	1. Over-prescription; 2. High cost; 3. Drug safety; 4. Uneven resources
Economy Labor	Do you prefer full-time employment?	1. Yes; 2. No
	What is the biggest challenge if changing jobs?	1. Salary/benefits; 2. Skill competitiveness; 3. Few opportunities; 4. Not difficult
Values Culture	Do you believe in life after death?	1. Yes; 2. No
	Which value should society prioritize?	1. Equal opportunity; 2. Individual freedom; 3. Social order; 4. Tradition
Technology & Society	What is your view on genetically modified foods?	1. Healthier; 2. More harmful; 3. No difference
	Who should protect personal information online?	1. Companies; 2. Individuals; 3. Public institutions
Environment & Energy	Which position do you lean toward on climate change?	1. Existential crisis; 2. Politicized; 3. Long-term issue; 4. Natural cycles
	Regarding energy, what are you most concerned about?	1. Prices; 2. Outages; 3. Fossil fuel reliance; 4. Natural disasters

Why it matters

As LLMs get used more to simulate public opinion, consumer choices, and political behavior, this work shows the standard way of grading and training these models on a single 'correct' answer may be fundamentally flawed. Anyone building AI-based survey simulation, market research, or opinion modeling tools needs to rethink how they measure and train for realistic human variability.

Terms in this paper

LLM · Large Language Model, an AI system trained to understand and generate text
subjectivity coefficient · a number measuring how much people's answers to a given question naturally vary, based on entropy
SALT · the paper's new training method that pools answers from similar contexts into soft probability targets
SubjSim · a new benchmark dataset with 193 people's answers to 100 subjective questions, including full probability distributions for evaluation
KL divergence / JSD / TVD / MMD · different mathematical ways of measuring how different two probability distributions are

Original abstract (English)

LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evaluate the fidelity of LLM-simulated human behavior and optimize LLMs toward it. Prevailing practice evaluates by accuracy, checking whether the model selects the single response observed from a human, and trains the LLM to reproduce this hard label. However, human behavior is inherently subjective: the same person in the same situation may reasonably act differently, so an observed response is only one draw from an underlying response distribution, rendering accuracy-based evaluation unreliable and hard-label training misleading. To address these problems, we first introduce the subjectivity coefficient, an entropy-based quantity distinguishing objective tasks such as coding from subjective ones such as social simulation, and use it to systematically analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows. Based on the subjectivity coefficient, we propose Subjectivity-Adaptive soft-Label Training (SALT): it pools observed outputs from semantically nearby inputs into soft distributional labels, with an aggregation radius adapted to the estimated subjectivity of each input; in the near-objective limit the neighborhood shrinks, so SALT naturally falls back to standard single-label training. Moreover, since existing datasets record only single observed responses and cannot support distributional evaluation, we construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions. Since real-world data typically provide only a single observation per input, our experiments train models from single observed outputs while evaluating them against the full response distributions, verifying feasibility in realistic settings. Results on SUBJSIM demonstrate the advantages of our method.

Authors · Pei Wang, Xu Chen, Ji-Rong Wen

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Pei Wang et al., arXiv:2608.19689, CC BY 4.0