Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

arXiv:2608.196892026-08-21

AI가 사람 흉내를 낼 때, '정답 하나 맞히기'는 애초에 잘못된 시험 방식이다

설문조사처럼 사람의 행동을 흉내 내는 AI(LLM)를 평가할 때 지금까지는 '사람이 실제로 고른 답 하나'를 맞혔는지만 봤다. 하지만 사람은 같은 상황에서도 기분에 따라 다른 선택을 할 수 있어서, 관찰된 답 하나는 그 사람의 '가능한 선택지 확률 분포' 중 딱 한 번 뽑힌 결과일 뿐이다. 연구팀은 이 문제를 수학적으로 증명하고, 비슷한 상황들의 답을 모아 확률 분포 형태로 학습시키는 SALT라는 새 학습법과, 이를 검증할 벤치마크 SubjSim을 만들었다.

무엇을 했나

질문이나 상황이 얼마나 '주관적'인지를 숫자(주관성 계수)로 정의해서, 코딩처럼 답이 하나인 객관적 문제와 설문조사처럼 답이 여러 개일 수 있는 주관적 문제를 구분했다
기존 방식(정확도로 평가하고, 관찰된 답 하나만 정답으로 학습)이 주관성이 높아질수록 왜, 얼마나 틀리는지를 수식으로 증명했다
SALT는 각 상황과 비슷한 다른 사람들의 답을 모아 '이 상황에서는 이런 비율로 답이 갈린다'는 확률 분포를 만들고, 상황이 얼마나 주관적인지에 따라 얼마나 넓게 이웃을 모을지 자동으로 조절한다
193명이 100개의 주관적 설문 문항에 답한 19,300개의 데이터로 SubjSim 벤치마크를 새로 만들었다. 훈련 때는 기존 방식처럼 답 하나만 주고, 평가할 때만 전체 확률 분포와 비교해 실제 상황에서도 통하는지 확인했다
Qwen3-8B 모델로 실험한 결과 기존 지도학습(SFT) 대비 SALT는 KL divergence를 77.6%, JSD를 45.9%, TVD를 31.2%, MMD를 52.7% 줄였고, 특히 주관성이 높은 문제일수록 개선폭이 컸다(최고 96.9%)

Figure 1: Subjectivity-stratified comparison between SFT and SALT on SubjSim, over low-, mid-, and high-subjectivity questions. All metrics are divergences (lower is better), and percentages give SALT’s relative change over SFT.

Figure 2: Ablation over neighbor selection. Top-N varies N (teal, bottom axis); fixed Keff substitutes a constant for K^eff(x) (navy, top axis). Both share the y axis and reference lines; SFT lies far above the plotted range and is reported as text.

Table 1: Main results on SubjSim. All metrics are divergences (lower is better). Best values are in bold on a blue background; second-best values are underlined on a sand background.
	Economy	Political	Technology
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.6285	0.2851	0.5735	0.6346	4.1453	0.3044	0.6206	0.6586	4.2015	0.2547	0.5335	0.5502
SFT	1.3543	0.1542	0.3951	0.3283	1.2865	0.1568	0.4145	0.3167	1.3409	0.1537	0.3956	0.3215
DPO	6.9962	0.2752	0.5573	0.6049	7.2926	0.2834	0.5816	0.5861	6.9174	0.2580	0.5305	0.5450
PPO	4.0161	0.2672	0.5495	0.5863	4.4698	0.2726	0.5724	0.5684	4.2959	0.2502	0.5251	0.5310
DSA	3.2032	0.2332	0.4920	0.4337	3.4679	0.2442	0.5161	0.4436	3.4744	0.2337	0.4888	0.4275
SALT (Ours)	0.3129	0.0892	0.2839	0.1640	0.2535	0.0724	0.2473	0.1247	0.2817	0.0815	0.2736	0.1466
	Social	Culture	Health
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	4.2286	0.2720	0.5562	0.6079	3.0456	0.2445	0.5175	0.5646	4.7412	0.2822	0.5737	0.6467
SFT	1.3259	0.1588	0.3977	0.3354	0.8578	0.1209	0.3459	0.2739	1.5169	0.1566	0.3886	0.3286
DPO	6.8092	0.2608	0.5289	0.5629	5.6298	0.2385	0.5050	0.5398	6.5058	0.2501	0.5152	0.5366
PPO	4.2388	0.2524	0.5210	0.5471	3.5294	0.2278	0.4909	0.5180	4.4975	0.2409	0.5059	0.5215
DSA	3.2049	0.2334	0.4963	0.4537	4.1157	0.2623	0.5259	0.5762	3.8155	0.2433	0.4889	0.4561
SALT (Ours)	0.3168	0.0905	0.2863	0.1656	0.2593	0.0746	0.2652	0.1548	0.2995	0.0859	0.2793	0.1601
	Environment	Education	ALL
Method	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓	KL↓	JSD↓	TVD↓	MMD↓
Pretrained	5.3499	0.3091	0.6199	0.6506	6.1097	0.3662	0.6842	0.8295	4.2801	0.2851	0.5745	0.6350
SFT	0.8800	0.1373	0.3837	0.2685	2.0897	0.2324	0.5246	0.4818	1.2871	0.1524	0.3953	0.3194
DPO	7.2403	0.3099	0.6163	0.6573	8.4950	0.3276	0.6291	0.7074	6.8819	0.2690	0.5497	0.5774
PPO	3.5411	0.2914	0.6035	0.6206	5.1310	0.3337	0.6447	0.7245	4.1393	0.2588	0.5403	0.5578
DSA	3.0895	0.2116	0.4613	0.3496	2.7515	0.2260	0.4870	0.4152	3.3904	0.2343	0.4937	0.4418
SALT (Ours)	0.2983	0.0870	0.2705	0.1407	0.3866	0.1095	0.3229	0.1947	0.2880	0.0825	0.2720	0.1510

Figure 3: Sensitivity of SALT to the radius scale C (divergences, lower is better).

Figure 4: Per-domain results on the Economy domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 2: Mapping between theoretical concepts, notation, and their SubjSim implementation.
Concept	Notation	SubjSim realization	Protocol role
Decision context	x=(u,s)	Annotator persona paired with a survey question; each persona has 30 demographic attributes.	Input to training and evaluation.
Action space	𝒜⁡(x)	Candidate response options for the survey question.	Common support for distributional evaluation within each question.
Latent response-propensity target	ϕ∗(⋅∣x)	Probability-ball empirical distribution ϕ^uball(⋅∣x), used as an elicited distributional proxy rather than a repeated-choice frequency.	Hidden during training; used only as the evaluation target.
Hard observation	a⁡(x)	Modal response under ϕ^uball(⋅∣x).	Hard-label proxy for SFT, DPO, PPO, and SALT targets.
Action-space group	𝒢g	Persona-question contexts from the same survey question.	Restricts aggregation to comparable candidate options.
SALT neighborhood	𝒩⁡(x)	Nearest contexts within the adaptive embedding radius, using Qwen3-embedding-8b embeddings.	Defines which hard observations SALT pools.
Effective action count (oracle)	Keff∗(x)	Effective action count of the true response distribution at the context; never observed.	Appears in the bound of Theorem 1 and in the oracle radius.
Effective action count (model-based)	K^eff(x)	The same functional applied to the model’s own distribution, Keff(ϕθ(⋅∣x)).	Sets the adaptive radius without using the hidden target.
Global baseline	Global-Freq	Question-level frequency of hard labels.	Ablation that removes persona conditioning.

Table 3: Main training and method-specific hyperparameters for all methods.
		Policy models	PPO aux.
Group	Parameter	SALT	SFT	DPO	PPO	DSA	RM
General	Epochs	4	5	4	4	4	4
Global batch size	128	128	96	128	128	128
Learning rate	5e-6	5e-6	5e-7	1e-6	5e-6	1e-5
LR schedule	Cosine	Cosine	Cosine	Cosine	Cosine	Cosine
Warmup ratio	0.1	0.1	0.1	0.1	0.1	0.1
Optimizer	AdamW	AdamW	AdamW	AdamW	AdamW	AdamW
Max sequence length	1024	1024	1024	1024	1024	1024
DPO	β	—	—	0.07	—	—	—
Loss type	—	—	Sigmoid	—	—	—
Label smoothing	—	—	0.0	—	—	—
FTX coefficient	—	—	0.0	—	—	—
PPO	ϵclip	—	—	—	0.2	—	—
Target KL	—	—	—	6.0	—	—
Initial KL coeff.	—	—	—	0.05	—	—
Sampling temperature	—	—	—	0.7	—	—
Top-p	—	—	—	0.9	—	—
SALT	Scale factor C	0.2	—	—	—	—	—
PCA variance threshold	0.90	—	—	—	—	—
Softmax temperature	1.0	—	—	—	—	—

Figure 5: Per-domain results on the Political domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 4: Point estimates and 95% paired-bootstrap confidence intervals on the full test set (1,000 resamples over contexts). All metrics are divergences (lower is better).
Method	KL	JSD	TVD	MMD
Pretrained	4.2801[4.1628,4.3949]	0.2851[0.2785,0.2912]	0.5745[0.5646,0.5833]	0.6350[0.6161,0.6524]
SFT	1.2871[1.2281,1.3439]	0.1524[0.1474,0.1573]	0.3953[0.3869,0.4034]	0.3194[0.3060,0.3324]
DPO	6.8819[6.7226,7.0539]	0.2690[0.2625,0.2752]	0.5497[0.5396,0.5590]	0.5774[0.5595,0.5935]
PPO	4.1393[4.0256,4.2464]	0.2588[0.2524,0.2647]	0.5403[0.5302,0.5490]	0.5578[0.5407,0.5743]
DSA	3.3904[3.2932,3.4851]	0.2343[0.2284,0.2403]	0.4937[0.4857,0.5019]	0.4418[0.4281,0.4567]
SALT (Ours)	0.2880[0.2749,0.3009]	0.0825[0.0788,0.0863]	0.2720[0.2653,0.2787]	0.1510[0.1440,0.1582]
Top-N (N=3)	0.3653[0.3497,0.3809]	0.0925[0.0886,0.0964]	0.3078[0.3011,0.3144]	0.1921[0.1833,0.2014]
Fixed Keff (K=2)	0.3152[0.3017,0.3290]	0.0900[0.0861,0.0939]	0.2839[0.2769,0.2909]	0.1656[0.1586,0.1734]
Global-Freq	0.3879[0.3713,0.4036]	0.0933[0.0894,0.0970]	0.3065[0.2993,0.3130]	0.1886[0.1798,0.1973]

Figure 6: Per-domain results on the Technology domain, mirroring Figure 1, Figure 2, and Figure 3 of the main text. All metrics are divergences (lower is better).

Table 5: Source distribution of the 100 subjective questions in SubjSim.
Source	Count	Proportion
ATP	35	35.0%
GSS	19	19.0%
WVS	14	14.0%
CGSS	9	9.0%
ANES	9	9.0%
ESS	9	9.0%
ISSP	5	5.0%
Total	100	100%

Table 6: Examples of Demographic Attributes and Subjective Questions
Domain	Question	Response Options
Demographic Attributes
Demographic	What is your current academic year?	1. Freshman/Sophomore; 2. Junior/Senior; 3. Master’s; 4. PhD; 5. Not a student
	Are you an only child?	1. Yes; 2. No
	What is your current employment status?	1. Full-time; 2. Part-time; 3. Self-employed; 4. Retired; 5. Homemaker; 6. Student; 7. Unemployed; 8. Other
Subjective Questions
Political System	Over the next 30 years, which social trend concerns you the most?	1. AI replacing jobs; 2. Social stratification; 3. Misinformation; 4. Weakening family structures
	Among occupational groups, which do you trust the most?	1. Healthcare/education; 2. Law enforcement; 3. Business; 4. Non-profits
Education	Who should ensure young people acquire skills for good jobs?	1. Government; 2. Employers; 3. Education system; 4. Individuals
	Which quality is most important for children to learn?	1. Socially adept; 2. Obedient; 3. Hard work; 4. Helpful; 5. Independent thinking
Social Relations	What do you most often do in free time?	1. Social entertainment; 2. Leisure; 3. Self-improvement; 4. Exercise
	Which is more important: considerate or proper behavior?	1. Considerate; 2. Proper
Health Well-being	What is your view on vaccination?	1. Mandatory; 2. Personal choice; 3. Cautious; 4. Not mandatory
	What is the biggest problem with the healthcare system?	1. Over-prescription; 2. High cost; 3. Drug safety; 4. Uneven resources
Economy Labor	Do you prefer full-time employment?	1. Yes; 2. No
	What is the biggest challenge if changing jobs?	1. Salary/benefits; 2. Skill competitiveness; 3. Few opportunities; 4. Not difficult
Values Culture	Do you believe in life after death?	1. Yes; 2. No
	Which value should society prioritize?	1. Equal opportunity; 2. Individual freedom; 3. Social order; 4. Tradition
Technology & Society	What is your view on genetically modified foods?	1. Healthier; 2. More harmful; 3. No difference
	Who should protect personal information online?	1. Companies; 2. Individuals; 3. Public institutions
Environment & Energy	Which position do you lean toward on climate change?	1. Existential crisis; 2. Politicized; 3. Long-term issue; 4. Natural cycles
	Regarding energy, what are you most concerned about?	1. Prices; 2. Outages; 3. Fossil fuel reliance; 4. Natural disasters

왜 중요한가

정치 성향, 소비 선택, 여론 조사처럼 사람마다 답이 갈리는 문제를 AI로 시뮬레이션하는 연구와 서비스가 늘고 있는데, 지금까지 쓰던 '정답 맞히기' 평가와 학습 방식 자체가 근본적으로 잘못됐을 수 있다는 것을 보여준다. AI 기반 사회 시뮬레이션, 여론조사 대체, 마케팅 리서치 등에 LLM을 쓰려는 사람이라면 평가 기준과 학습 방식을 다시 점검해야 한다는 신호다.

이 논문의 용어

LLM · 거대언어모델. 텍스트를 이해하고 생성하는 AI
주관성 계수 · 어떤 질문/상황에서 사람들의 답이 얼마나 갈리는지를 엔트로피로 측정한 수치
SALT · 비슷한 상황들의 답을 모아 확률 분포로 만들어 학습시키는 이 논문의 새 학습법
SubjSim · 193명이 100개 주관식 문항에 답한, 확률 분포까지 기록된 새 벤치마크 데이터셋
KL divergence / JSD / TVD / MMD · 두 확률 분포가 얼마나 다른지를 재는 여러 가지 수학적 거리 측정 방법들

논문 원문 초록 (영문)

LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evaluate the fidelity of LLM-simulated human behavior and optimize LLMs toward it. Prevailing practice evaluates by accuracy, checking whether the model selects the single response observed from a human, and trains the LLM to reproduce this hard label. However, human behavior is inherently subjective: the same person in the same situation may reasonably act differently, so an observed response is only one draw from an underlying response distribution, rendering accuracy-based evaluation unreliable and hard-label training misleading. To address these problems, we first introduce the subjectivity coefficient, an entropy-based quantity distinguishing objective tasks such as coding from subjective ones such as social simulation, and use it to systematically analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows. Based on the subjectivity coefficient, we propose Subjectivity-Adaptive soft-Label Training (SALT): it pools observed outputs from semantically nearby inputs into soft distributional labels, with an aggregation radius adapted to the estimated subjectivity of each input; in the near-objective limit the neighborhood shrinks, so SALT naturally falls back to standard single-label training. Moreover, since existing datasets record only single observed responses and cannot support distributional evaluation, we construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions. Since real-world data typically provide only a single observation per input, our experiments train models from single observed outputs while evaluating them against the full response distributions, verifying feasibility in realistic settings. Results on SUBJSIM demonstrate the advantages of our method.

저자 · Pei Wang, Xu Chen, Ji-Rong Wen

arXiv에서 원문 보기

METAL LAB 최신 기사

그림 출처: Pei Wang et al., arXiv:2608.19689, CC BY 4.0

Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

무엇을 했나

왜 중요한가

이 논문의 용어

최신 논문

METAL LAB 최신 기사