Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See
arXiv:2608.177442026-08-18
그리스어로 생각하는 AI 모델을 만들어보니, 정확도 점수는 거의 의미가 없었다
연구자는 Alibaba, OpenAI, NVIDIA가 만든 세 개의 대형 AI 모델을 그리스어로 추론하도록 미세조정했다. 그런데 정확도 점수만 보면 별 차이가 없었고, 심지어 무작위 시드값만 바꿔도 점수가 7.7점이나 흔들려서 그동안 측정한 어떤 데이터·훈련법 효과보다 컸다. 진짜 변화는 정확도로는 안 보이는 곳, 즉 모델이 어떤 언어로 생각하는지, 얼마나 효율적으로 생각하는지에 있었다.
무엇을 했나
원래 모델들은 그리스어 질문을 받아도 속으로는 전혀 그리스어로 생각하지 않았다(1000개 중 0개). 답은 맞히지만 사용자가 읽거나 검증할 수 없는 형태로 사고 과정이 진행된 셈이다.
지도미세조정(SFT, 정답이 붙은 예시로 모델을 추가 학습시키는 방법)을 거치자 네 개 모델 모두 약 98%의 문제에서 질문 언어(그리스어)로 사고하게 되었고, 문법 품질도 좋아졌으며 다른 능력은 거의 잃지 않았다.
SFT가 못 고치는 결함도 있었다: 답의 4분의 1은 요청된 형식을 건너뛰었고, 답이 사고 과정 칸으로 새어나갔으며, '영어로 생각해'라는 명시적 지시도 절반 이하로만 지켜졌다.
검증 가능한 보상을 이용한 강화학습(RLVR)을 훈련 전에 미리 설계해 적용하자 형식 무시 문제(24%→2.5%)와 답 누출 문제(3.5%→0%)는 확실히 고쳐졌고, 영어 지시 순응도도 개선됐다. 다만 그리스어로 생각하는 습관은 정확도만 노리는 학습으로는 바뀌지 않았다.
연구팀은 다섯 개의 모델 체크포인트를 공개했고, 이 측정·검증 방법은 그리스어뿐 아니라 다른 자원 부족 언어에도 적용 가능하다.
Figure 1: fallback% (the rate at which a model never emits the requested answer line) by recipe and by language-matched checkpoint. It separates “cannot reason” from “will not answer in the requested form”: every recipe failure in the paper is visible here before it is visible anywhere else. The Qwen recipes are the side experiment of §7; the Gpt-OSS language-matched arm’s pre-repair Greek-lane rate (70%) is the family defect at its largest (its traces are present and on-language; the model simply does not close with the line the scorer is told to read), and the released checkpoint’s format-repair dose cuts it to 26% (§11), still the highest of the releases. The dashed reference is a base model on the English control (3.4%), showing the failure is not inherent to the benchmark.
Table 1: Total and routed-expert parameters counted from the released tensors; active-per-token is non-routed + k/E of routed. Gpt-OSS ships MXFP4-packed experts, so its figures are vendor-reported. NemotronH (Nano) and Nemotron-3.5-Lightning are Mamba/MoE hybrids (Mamba, MoE and attention layers of 52; the state-space line of Gu and Dao 2023, Dao and Gu 2024 and the vendor’s Nemotron-H report [21]); the other two are MoE transformers. The right-most column names the fine-tuned Sophea reasoning release each base produces; all four SFT releases are language-matched, and a fifth release, Sophea-Qwen3.6-v1.1, is the RLVR refinement of the Qwen row (§15.1). Throughout, base refers to the vendor checkpoint and fine-tuned to its Sophea reasoning model. Lightning is the next-generation release in the Nemotron line; Nano and Lightning share the 31.6B / 6-of-128 routing profile and are reported together as the Nemotron family (Nano is the same-generation member of that family).
lab
base model
total
active
routing
fine-tuned release
Alibaba
Qwen3.6-35B-A3B
36.0B
3.97B
8 of 256
Sophea-Qwen3.6-v1
OpenAI
Gpt-OSS-20B
20.9B
3.60B
4 of 32
Sophea-OSS-v1
NVIDIA
NemotronH-30B-A3B
31.6B
3.58B
6 of 128
Sophea-Nemo-3-Nano-v1
NVIDIA
Nemotron-3.5-Lightning-30B-A3B
31.6B
3.58B
6 of 128
Sophea-Nemo-3.5-Lightning-v1
Figure 2: Left: one configuration, three seeds, nothing else changed. Right: every accuracy effect we measured over the project, against that range. All of them fit inside it.
Table 2: The reasoning half is math-heavy and thin on the axes we evaluate hardest: commonsense is 6.6% and logic 8.7% of it.
domain (reasoning half)
rows
share
mathematics
22,608
38.2%
science
11,064
18.7%
world knowledge
5,884
10.0%
deductive logic
5,151
8.7%
medical
4,898
8.3%
commonsense
3,904
6.6%
reading comprehension
3,828
6.5%
physics
1,770
3.0%
Figure 3: Every accuracy effect we measured, against the 7.7-point seed-to-seed band (shaded). Attribute-level effects (data selection and corpus versions) sit inside the band we would have been tempted to narrate. The only effect that survives is the replicated phase-2 comparison of §7 (+6.9 pp, 15 arms, permutation p=0.0008): it is inside the band geometrically, but it is estimated across 15 independently trained arms rather than read off a single run, which is precisely the difference the floor teaches. The language-matched accuracy cost (−6.9 pp) is drawn as an open (unfilled) marker on purpose: it is a single-run measurement inside the floor, disclosed rather than interpreted, and not a recipe property. The equal magnitudes of the +6.9 and the −6.9 are a coincidence: different lanes, different arms.
Table 3: Two-Phase continues the same adapter from Reasoning onto the hybrid mix; One-Phase never separates the two halves. The three recipes fail in different ways (§7).
name
trained on
rows
Base
nothing (the released model)
—
Reasoning
reasoning rows only
59,107
Subset
a subset of the same rows
15,607
Two-Phase
reasoning rows, then + the direct half
59,107+
One-Phase
one pass over both halves at once
59,107+
Figure 4: The same two checkpoints on four dimensions, the base and the structure-selected reasoning-only arm (Subset, the “Reason-16k” of Figure 11), on the 1,000-item probe lane. Only the first is accuracy.
Table 4: sd=4.4 pp, range =7.7 pp. Note the last column.
seed
mean
logic
fallback
trace Greek
42
76.2
56.2
10%
1.00
43
68.7
35.6
41%
1.00
44
76.4
54.0
3%
1.00
Figure 5: Median trace length in tokens (each family’s own tokenizer, 5,156 matched items), base against the fine-tuned release. The word-level shortening must repay Greek’s 2.3–2.5× token fertility before it becomes a serving saving: Qwen’s does, NemotronH’s breaks even, Gpt-OSS’s does not.
Table 5: Selection buys −0.5 pp (0.31σ); a 4× smaller pool costs nothing. The 62,562 is the reasoning count of the pre-gate pool the experiment was run against (62,562 reasoning, 60,214 direct); the 59,107 quoted everywhere else is the post-gate train half (59,107 reasoning, 58,985 direct), so the table’s all row and the paper’s reasoning-half count differ by the 3,455 rows the decontamination/dedup gate and the val reserve removed.
phase-1 rows
selection
mean
62,562
all
69.5
15,607
top-by-structure
69.5
15,607
uniform random
69.0
Figure 6: Overrun = trace ≥3× the median trace for that item across arms. Easy = at least 80% of arms answer correctly. Cells pool the arms of each column; hard-item think-mode rates are compressed by generation-cap truncation (§5), so the easy row carries the section’s claims.
Table 6: Per-domain results on the Greek think lane; here and in every table, ↑ marks metrics where higher is better and ↓ where lower is better (math = the two math slices, commonsense = HellaSwag+Winogrande, logic = ProofWriter macro-recall). Rows are grouped by family, each release indented under its own base, so every legitimate accuracy delta is a within-block comparison and the cross-base reading (the one an earlier draft of this paper fell for) is structurally discouraged. Every release now carries its own base row: the NemotronH base dump is the same vintage as the Qwen base dump, and the Nemotron-3.5 base lane, absent until this revision, was generated with the identical lane settings and scored with the same scorer as every other row. The same base is also measured on the NLU retention suite (Table 14, Greek macro 57.5), a direct-mode instrument whose numbers are not comparable to this think-lane table. The Nano release also appears in the retention suite (Table 13), and its fallback shape is the mirror of Gpt-OSS’s: negligible on math and commonsense (1.7/1.8%) but 39.9% on logic, so its 9.8 logic macro-recall is substantially a format floor on the one domain where its answer form breaks. Accuracy and fallback move together: every release keeps the base’s domain ordering, and the Sophea-OSS format-repair dose pulls the commonsense fallback back toward math’s level (83.5→25.7) while cutting logic’s by a third (77.4→50.9). The pre-repair row is reference only; the release is the repaired arm. Conditioning on rows that emit the requested answer line (excluding the fallback-scored path) shrinks the apparent deficits sharply: the Qwen release reads 95.8/78.8/39.5 against its base’s 96.0/82.8/42.7, and the same condition makes Gpt-OSS pre-repair 74.6→92.2 / 53.7→60.2 / 25.2→36.5 and Sophea-OSS-v1 78.6→94.4 / 53.3→58.2 / 26.5→30.0 (math/commonsense/logic), so a substantial share of every raw gap is format compliance, not reasoning.
anchored accuracy
fallback %
arm
math
cs
logic
math
cs
logic
↑ higher is better
↓ lower is better
Qwen base (Greek think)
92.9
82.4
42.6
3.3
0.5
0.2
Sophea-Qwen3.6-v1
83.3
75.8
37.3
18.2
22.9
46.3
Gpt-OSS base (Greek think)
90.7
63.2
32.2
2.4
5.2
0.7
before repair (reference)
74.6
53.7
25.2
34.9
83.5
77.4
Sophea-OSS-v1 (repaired)
78.6
53.3
26.5
17.2
25.7
50.9
NemotronH base (Greek think)
86.6
49.2
26.8
8.3
6.2
6.6
Sophea-Nemo-3-Nano-v1
71.5
39.7
9.8
1.7
1.8
39.9
Nemotron-3.5 base (Greek think)
76.9
46.3
25.0
1.4
1.9
4.4
Sophea-Nemo-3.5-Lightning-v1
75.6
40.9
23.8
16.1
14.7
5.3
Figure 7: Each point is an independently trained arm. Corpus version varies within both groups.
Table 7: Language-matching across the three families: all four released checkpoints, the two measurable bases, and the one-directional reference arm. Greek fid.: Greek-trace fidelity on the Greek benchmark; both bases sit at exactly 0.0 (0/5,156 traces reach ratio 0.9), and all four language-matched checkpoints land at the same level (97.4–98.1%). EN ctrl: English-control compliance, 100% for every arm. Switches: in-question language switches per 100 sentences on the Greek lane; the bases switch 16.4 (Qwen) and 9.7 (Gpt-OSS) times, every fine-tuned arm (the one-directional reference included) zero. The base fidelity and switch cells are one instrument run: both base dumps rescored with the current scorer on the same day (an earlier draft printed 19.0 for the Qwen base from an earlier dump vintage). Leak: answer-channel leak, the one place families differ (10.3% Sophea-OSS-v1, 5.2% Lightning, 3.5% Qwen, 0.0% Nano, the only arm at zero; base leak is measured on the English control, fine-tuned leak on the Greek think lane). †Arms of the earlier one-directional recipe, shown for reference (a different recipe, not releases); the Qwen one-directional row is the recipe-evolution comparison: it locks against explicit language instructions (Table 10), not against English questions. §Measured on the 1,900-item paired probe (99.3% English, 0.0% Greek, 0.7% too short to score); this arm was never run on the 1,100-item control the other EN-ctrl cells use. ‡Measured on the pre-repair language-matched arm: the repair dose adds 2% trace-less answer-format rows and does not touch trace language, but the released arm’s own English control has not been generatively re-run. The recipe generalises to every family tried; the leak’s size does not.
Greek fid.
EN ctrl
switches
leak
arm
(%)↑
(%)↑
/100 sent.↓
(%)↓
Qwen base
0.0
100
16.4
15.6
Qwen lang-matched
98.0
100
0.0
3.5
Qwen one-dir.†
98.7
99.3§
0.0
1.5
Gpt-OSS base
0.0
100
9.7
0.5
Gpt-OSS lang-matched
98.1
100‡
0.0
10.3
Nemotron-3.5 lang-matched
98.1
100
0.0
5.2
NemotronH lang-matched
97.4
100
0.0
0.0
NemotronH one-dir.†
98.4
100
0.0
11.3
Figure 8: The four withdrawn findings of this section, drawn: the effect as first believed (open circle) against the same quantity after its control (filled). Three flip sign; the un-anchored scorer artifact (+29.8 pp, 18σ, one probe) collapses entirely: its endpoint is drawn at zero because the prose reports the artifact removed by anchored scoring, not as a measured residual. Rows are different instruments and lanes, so magnitudes compare within a row, not across rows; the other pairs’ numbers appear in this section’s prose. The section’s fifth failure (§8.5) is a scorer defect rather than an effect-size withdrawal, so it has no before/after pair to draw.
Table 8: single-phase = one pass over both halves; two-phase = reasoning-only then hybrid; reasoning-only = the reasoning half alone. Empty-trace rate is measured in think mode, fallback% in direct mode; means are over all arms of each recipe. Fallback ranges are over the individual arms of each recipe (2–12% across the three reasoning-only arms; 33–40% across the two-phase arms); the three seed replicates of §4 widen it further (3–41% across seeds 42–44, all three of which are among the six phase-1-only arms above), so the fuller picture is 2–41% when seed noise is admitted.
recipe
mean
empty
fallback
One-Phase
66.1
23.6%
12%
Two-Phase
64.4–69.9
0.0–1.3%
33–40%
reasoning-only
73.6
0.0%
2–12%
Figure 9: Trace-language composition per lane, Qwen line: the share of traces that are Greek (character ratio ≥0.9), mixed, or English (≤0.1), on the same instrument as Table 7’s fidelity column. Greek-question rows are the full 5,156-item benchmark; English-question rows are the 1,100-item control (base) and the 1,900-item probe (matched). Two honesty notes drawn rather than hidden: the base’s Greek-question traces are not mostly pure English: only 4.2% are; the mass is mixed script with English scaffolding (median ratio 0.33, §5); and the one-directional arm’s Greek-question lane decomposes as 98.7 Greek / 1.3 mixed / 0.0 English on the same instrument (n=5,156, uninstructed); its English-question lane, generated later on the same 1,900-item probe, measures 99.3% English (Table 7). Uninstructed defaults only; the instructed override is Table 10.
Table 9: The base is ∼5 pp worse on our translations; every fine-tune is slightly better.
arm
human
machine
Δ
Base
95.6
90.7
−4.9 (−3.14σ)
Subset
94.0
95.1
+1.1
Reasoning
93.6
95.3
+1.7
Two-Phase
87.6
90.4
+2.8
Figure 10: Every per-benchmark retention delta behind Table 14: fine-tuned release minus its own base (Lightning against the Lightning base; Sophea-OSS-v1 is the format-repaired release), Titan-1 suite, non-reasoning mode. Per-benchmark binomial sigma applies and a delta is a direction, not a test (Table 13); the value is the pattern: the Qwen column is near-white in both languages, the Sophea-OSS-v1 residual concentrates on the knowledge-heavy benchmarks (MMLU-el −10.1) while HellaSwag gains in both languages on every family, and MMLU is the largest English loss for all three non-Qwen releases.
Table 10: The one-directional fine-tune’s traces are identical under both instructions, and it does not matter how the instruction is phrased. Rows classified as neither/mixed scripts (28% of the base under the Greek instruction) are omitted, so rows need not sum to 100. The matched ft rows, measured after the language-matched re-training of §10, show the override partially returning; its mixed share under the English instruction is 30.4%.
model, instruction
English traces
Greek traces
base, “reason in English”
100.0%
0.0%
base, “reason in Greek”
72.0%
0.0%
fine-tune, “reason in English”
0.0%
98.7%
fine-tune, “reason in Greek”
0.0%
98.7%
matched ft, “reason in English”
44.8%
24.8%
matched ft, “reason in Greek”
0.0%
98.5%
Figure 11: Left: accuracy per axis on the 5,156-item benchmark for the Qwen recipe arms (the cross-family releases are Table 6), with the seed-to-seed noise floor shaded around the Qwen base. Right: the same arms’ cost in words per correct answer (token-denominated cost is Figure 5). Math uses the 250 human-translated items only (§8); logic is macro-recall (§3).
Table 11: Instructed-override compliance on the five released checkpoints: the share of traces in the instructed language when the instruction contradicts the question (n=1,000 Greek-question and n=1,100 English-question items; same scorer as Table 10). Every release obeys “switch to Greek”; only the Qwen line and Gpt-OSS obey “switch to English”, and the RLVR refinement widens Qwen’s opening (44.8→53.9%, §15.1). Two instrument notes, both checked: the Nemotron dumps do not separate trace from answer, so the mandatory Greek answer line falls inside the scored text: re-scoring with that line stripped leaves both Nemotron arms at 0.0% (the Qwen control moves 44.8→49.5%, the expected direction); and the Nemotron arms return 5–9% of rows too short to score against Qwen’s 0%, consistent with the low reasoning ceiling reported in §13.
released checkpoint
“in English”↑
“in Greek”↑
(Greek Q)
(English Q)
Sophea-Qwen3.6-v1
44.8%
83.7%
Sophea-Qwen3.6-v1.1 (RLVR)
53.9%
85.7%
Sophea-OSS-v1
62.5%
93.3%
Sophea-Nemo-3.5-Lightning-v1
0.0%
92.7%
Sophea-Nemo-3-Nano-v1
0.0%
87.5%
one-directional (reference)
0.0%
91.4%
Table 12: English questions get English traces from every arm measured. The fine-tunes in the first three rows are the dual-mode / reasoning-recipe checkpoints (§7), kept deliberately: they are the recipe-class evidence that one-directional training does not lock English questions. The last two rows are the language-matched checkpoints: the released Qwen checkpoint, re-run on this exact control (94.4, +0.4 over its base, 100% English traces, consistent with its 1,900-item probe above), and the Gpt-OSS language-matched arm evaluated after merging (the pre-repair arm; the released, format-repaired checkpoint’s control has not been generatively re-run, Table 7). Together the rows confirm the language-matching result is not Qwen-specific; none of this retests the hard one-directional lock of Table 10, which remains a property of training in a single output direction.
arm (n=1,100)
accuracy
reasons in EN
Qwen base / ft
94.0 / 96.2
100% / 100%
Gpt-OSS base / ft
95.5 / 96.1
100% / 100%
NemotronH base / ft
92.7 / 93.7
100% / 100%
Qwen language-matched
94.4
100%
Gpt-OSS language-matched
91.9
100%
Table 13: Forgetting on the Titan-1 suite. Top: Gpt-OSS on the nine Greek non-reasoning benchmarks, base against the language-matched fine-tune, before and after the format-repair dose: the language-matched fine-tune initially costs −7.3 points, and the repair dose recovers 4.1 of them, so the residual is −3.2 not zero. Below: the macro deltas this paper reports per family, fine-tuned arm vs its own base; the format-repaired Gpt-OSS is the fine-tuned arm we stand behind. Qwen is flat in both languages; Nano gains Greek (+3.8) against the family’s low 0.49 Nano base (mostly catch-up, not forgetting); Lightning is read against its own base, measured separately (Greek macro 57.5, well above Nano’s 48.6), and gains +1.7; an earlier +10.6 reading against the shared Nano base is superseded by that measurement. Per benchmark against binomial standard error; a macro delta is a direction, not a test. †On English the pre-repair language-matched Gpt-OSS lost −7.7 points (75.7→68.0); re-measured on the repaired release, the English macro is 76.7, +1.0 above base: the repair dose recovered both lanes, so the pre-repair loss was format behaviour throughout. Macro deltas are computed on unrounded macros; Table 14 prints one decimal, so its Qwen columns round to 75.7→75.6 Greek / 85.1→85.2 English.
Greek macro
base
language-matched
+ format repair
Gpt-OSS
61.2
53.9
58.0
macro deltas vs base (points):
Sophea-Qwen3.6-v1
−0.01 Greek / +0.08 English
Sophea-OSS-v1
−3.2 Greek / +1.0 English†
Sophea-Nemo-3-Nano-v1
+3.8 Greek / −2.0 English
Sophea-Nemo-3.5-Lightning-v1
+1.7 Greek / −1.1 English
Table 14: Every benchmark behind the macros of Table 13 (Titan-1 suite, non-reasoning mode; accuracy %). Qwen ft is Sophea-Qwen3.6-v1 (language-matched); Gpt-OSS ft is Sophea-OSS-v1, the format-repaired release (its pre-repair intermediate is in Table 13); Nano ft is Sophea-Nemo-3-Nano-v1, the language-matched Nano release; Lightning ft is Sophea-Nemo-3.5-Lightning-v1 (language-matched); the two Nemotron generations share the same 31.6B / 6-of-128 routing profile (Table 1) and each is read against its own base column. (The earlier one-directional recipe arm is reported only in §13’s recipe history, not as a release.) The per-benchmark view localises the Gpt-OSS story: before the repair the loss concentrated on the instruction-format-heavy reading benchmarks (Belebele-el −17.7, ARC-Challenge −13.1 el / −19.1 en, MMLU-en −16.0) while HellaSwag and Winogrande gained in both languages, and the repair dose recovers most where the loss was format (Belebele-el regains +13.0 of the 17.7). English retention on the released Gpt-OSS checkpoint recovers to +1.0 above its base; Lightning retains English within 1.1 points of its own base.
Qwen
Gpt-OSS
Nemotron
benchmark
base
ft
base
ft
Nano base
Nano ft
Ltng. base
Ltng. ft
Greek (nine benchmarks, the Greek macro of Table 13):
ARC-Challenge-el
91.8
92.0
77.1
68.6
54.2
59.0
68.7
72.8
ARC-Easy-el
96.9
97.1
85.4
79.9
62.3
70.1
75.8
81.4
Belebele-el
92.7
93.7
86.0
81.3
67.0
78.0
81.3
83.3
GreekMMLU
83.8
83.0
66.3
61.3
58.8
53.3
64.2
62.2
HellaSwag-el
57.6
59.4
42.9
52.1
39.2
47.6
44.5
51.2
Medical-MCQA-el
81.2
80.3
42.1
38.7
27.8
31.2
41.7
38.2
TruthfulQA-el
39.1
36.5
39.7
35.5
31.7
33.2
34.2
33.7
Winogrande-el
61.0
62.5
55.4
59.6
53.3
58.3
55.8
60.1
MMLU-el
76.9
76.2
55.5
45.4
43.2
40.8
51.3
49.6
Greek macro
75.7
75.6
61.2
58.0
48.6
52.4
57.5
59.2
English retention (five benchmarks):
ARC-Challenge
96.1
95.7
90.0
86.8
88.3
82.2
92.0
89.1
ARC-Easy
99.1
99.1
96.0
95.7
96.2
93.4
97.8
96.4
HellaSwag
73.2
74.2
57.0
68.4
67.9
73.1
72.7
75.9
Winogrande
73.2
74.2
64.6
66.7
69.9
72.9
72.0
73.3
MMLU
83.9
82.6
70.8
65.8
69.9
60.5
75.8
69.7
English macro
85.1
85.2
75.7
76.7
78.4
76.4
82.0
80.9
Table 15: Supported and unsupported configuration choices, each with its evidential basis.
Supported by our evidence
basis
Drop the hybrid phase 2; train reasoning-only
+6.9 pp over 15 runs, p=0.0008 (§7); and the direct mode it sacrifices costs nothing measurable on direct-mode NLU (E3, §11)
Generate exploratory traces, not explanations
structure 0.27→0.68 on the same questions
Forbid formulaic openings in the generator
24/24 identical openings without it
Report fallback% beside accuracy
separates reasoning from format compliance
Run the seed control before any ablation
7.7 pp (§4)
Probe trace-language steerability per checkpoint
same recipe, four checkpoints, 62.5/44.8/0/0 (§10.1)
Keep a human-translated benchmark subset
caught a 3.1σ artifact (§8)
Test numeric normalisation in the target language’s own convention
5 of 6 Greek-thousands answers scored wrong while right (§8.5)
Gate metrics on length correlation
removed 3 of our 7
Fix format and leak defects with verifiable-reward RL, not more SFT data
fallback →2.5%, leak →0%, control flat, ∼78 GPU-h/arm (§15.1)
Run a random-reward control arm with any RLVR claim
it reproduced our baseline on every axis; without it, every gain here would be unattributable (§15.1)
NOT supported
why
A specific corpus version
five expansions, all inside the noise floor
A data-selection method
selection ≈ random, −0.5 pp (0.31σ)
A LoRA rank, stride, or epoch count
never varied; no evidence either way
A per-domain model choice
two of three per-domain score spreads exceed the seed range itself (§12)
Table 16: The pre-registered RLVR round, scored on the held-out instruments (Greek think lane n=5,156; override lanes n=1,000/1,100), same scorer, same day. The before row is the released checkpoint’s frozen dump rescored with the current scorer; its full same-day regeneration reproduced all 5,156 responses bit-identically (greedy decoding is deterministic here), and the plain-mode lanes to the decimal (44.8/83.7): the before column is not a provenance caveat but a verified constant. The control row is the entire admissibility argument: on every axis the random-reward arm reproduces the baseline, so nothing below can be elicitation. Accuracy is shown for completeness only: +3.3–3.5 is inside the ±7.7 seed floor of §4 and is claimed by nobody.
Greek
fall-
override
reverse
arm
fid. %↑
acc↑
back %↓
leak %↓
el→EN %↑
en→EL %↑
SFT checkpoint (before)
97.98
73.7
24.1
3.53
44.8
83.7
random reward (control)
98.06
74.0
22.1
3.61
44.1
83.8
correct+format+term.
98.22
77.2
2.5
0.00
48.8
84.7
+ language
98.02
77.2
5.6
0.04
45.9
84.5
+ override
98.27
77.0
2.8
0.02
53.9
85.7
왜 중요한가
저자원 언어로 AI를 서비스하려는 회사나 커뮤니티에게, '정확도 점수가 좋아졌다'는 흔한 주장이 실제로는 거의 근거가 없을 수 있다는 경고다. 대신 모델이 어떤 언어로 사고하는지, 토큰을 얼마나 쓰는지, 형식을 잘 지키는지 같은 구체적 행동 지표를 봐야 진짜 개선을 확인할 수 있다.
이 논문의 용어
SFT (지도미세조정) · 정답이 달린 예시 데이터로 이미 학습된 모델을 추가로 훈련시켜 특정 행동을 하도록 만드는 방법
RLVR (검증 가능한 보상 기반 강화학습) · 정답 여부를 자동으로 확인할 수 있는 보상 신호를 이용해 모델을 추가로 훈련시키는 강화학습 방식
MoE (전문가 혼합 모델) · 전체 파라미터 중 일부 '전문가' 부분만 골라 쓰는 구조로, 큰 모델을 싸게 서비스할 수 있게 해준다
시드(seed) · 훈련 시작 시 난수를 결정하는 값으로, 같은 설정이라도 시드가 다르면 결과가 달라질 수 있다
언어 충실도(language fidelity) · 모델이 사고 과정을 실제로 요청된 언어로 작성하는 정도
논문 원문 초록 (영문)
Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.
