每天早上一封邮件,把昨天的 AI 梳理好订阅邮件

METAL LAB

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

arXiv:2608.177442026-08-18

让AI用希腊语思考的实验发现,准确率几乎没变,真正的变化藏在准确率看不到的地方

研究者对来自阿里巴巴、OpenAI和NVIDIA的三个前沿AI模型进行微调,让它们用希腊语这种资源较少的语言进行推理。结果发现准确率几乎没有变化,而且噪音很大,仅仅改变随机种子就能让分数波动7.7分,比任何实际训练改动的影响都大。真正的变化发生在准确率无法衡量的地方:模型用什么语言思考,以及思考效率如何。

他们做了什么

  1. 原始模型即使收到希腊语问题,内部思考过程也从不使用希腊语(1000次中0次),也就是说模型给出正确答案,但用户完全看不懂也无法核查其思考过程。
  2. 经过监督微调(SFT,用带答案的示例数据对模型进行额外训练)后,四个模型都在约98%的问题上用希腊语进行思考,语法质量提升,且几乎没有损失其他能力。
  3. SFT无法解决所有问题:四分之一的答案没有按要求的格式给出,部分答案泄漏进了思考过程文本中,而明确要求'用英语思考'的指令被遵守的比例不到一半。
  4. 研究者预先设计了一种基于可验证奖励的强化学习方法(RLVR),用它训练后,格式不合规问题(24%降到2.5%)和答案泄漏问题(3.5%降到0%)都得到明显改善,遵守英语指令的比例也有提升,但用希腊语思考的习惯即使用纯准确率驱动的训练也无法改变。
  5. 研究团队公开发布了五个微调后的模型版本,其测量方法也适用于其他资源匮乏语言。
Figure 1: fallback% (the rate at which a model never emits the requested answer line) by recipe and by language-matched checkpoint. It separates “cannot reason” from “will not answer in the requested form”: every recipe failure in the paper is visible here before it is visible anywhere else. The Qwen recipes are the side experiment of §7; the Gpt-OSS language-matched arm’s pre-repair Greek-lane rate (70%) is the family defect at its largest (its traces are present and on-language; the model simply does not close with the line the scorer is told to read), and the released checkpoint’s format-repair dose cuts it to 26% (§11), still the highest of the releases. The dashed reference is a base model on the English control (3.4%), showing the failure is not inherent to the benchmark.
Figure 1: fallback% (the rate at which a model never emits the requested answer line) by recipe and by language-matched checkpoint. It separates “cannot reason” from “will not answer in the requested form”: every recipe failure in the paper is visible here before it is visible anywhere else. The Qwen recipes are the side experiment of §7; the Gpt-OSS language-matched arm’s pre-repair Greek-lane rate (70%) is the family defect at its largest (its traces are present and on-language; the model simply does not close with the line the scorer is told to read), and the released checkpoint’s format-repair dose cuts it to 26% (§11), still the highest of the releases. The dashed reference is a base model on the English control (3.4%), showing the failure is not inherent to the benchmark.
Table 1: Total and routed-expert parameters counted from the released tensors; active-per-token is non-routed + k/E of routed. Gpt-OSS ships MXFP4-packed experts, so its figures are vendor-reported. NemotronH (Nano) and Nemotron-3.5-Lightning are Mamba/MoE hybrids (Mamba, MoE and attention layers of 52; the state-space line of Gu and Dao 2023, Dao and Gu 2024 and the vendor’s Nemotron-H report [21]); the other two are MoE transformers. The right-most column names the fine-tuned Sophea reasoning release each base produces; all four SFT releases are language-matched, and a fifth release, Sophea-Qwen3.6-v1.1, is the RLVR refinement of the Qwen row (§15.1). Throughout, base refers to the vendor checkpoint and fine-tuned to its Sophea reasoning model. Lightning is the next-generation release in the Nemotron line; Nano and Lightning share the 31.6B / 6-of-128 routing profile and are reported together as the Nemotron family (Nano is the same-generation member of that family).
labbase modeltotalactiveroutingfine-tuned release
AlibabaQwen3.6-35B-A3B36.0B3.97B8 of 256Sophea-Qwen3.6-v1
OpenAIGpt-OSS-20B20.9B3.60B4 of 32Sophea-OSS-v1
NVIDIANemotronH-30B-A3B31.6B3.58B6 of 128Sophea-Nemo-3-Nano-v1
NVIDIANemotron-3.5-Lightning-30B-A3B31.6B3.58B6 of 128Sophea-Nemo-3.5-Lightning-v1
Figure 2: Left: one configuration, three seeds, nothing else changed. Right: every accuracy effect we measured over the project, against that range. All of them fit inside it.
Figure 2: Left: one configuration, three seeds, nothing else changed. Right: every accuracy effect we measured over the project, against that range. All of them fit inside it.
Table 2: The reasoning half is math-heavy and thin on the axes we evaluate hardest: commonsense is 6.6% and logic 8.7% of it.
domain (reasoning half)rowsshare
mathematics22,60838.2%
science11,06418.7%
world knowledge5,88410.0%
deductive logic5,1518.7%
medical4,8988.3%
commonsense3,9046.6%
reading comprehension3,8286.5%
physics1,7703.0%
Figure 3: Every accuracy effect we measured, against the 7.7-point seed-to-seed band (shaded). Attribute-level effects (data selection and corpus versions) sit inside the band we would have been tempted to narrate. The only effect that survives is the replicated phase-2 comparison of §7 (+6.9 pp, 15 arms, permutation p=0.0008): it is inside the band geometrically, but it is estimated across 15 independently trained arms rather than read off a single run, which is precisely the difference the floor teaches. The language-matched accuracy cost (−6.9 pp) is drawn as an open (unfilled) marker on purpose: it is a single-run measurement inside the floor, disclosed rather than interpreted, and not a recipe property. The equal magnitudes of the +6.9 and the −6.9 are a coincidence: different lanes, different arms.
Figure 3: Every accuracy effect we measured, against the 7.7-point seed-to-seed band (shaded). Attribute-level effects (data selection and corpus versions) sit inside the band we would have been tempted to narrate. The only effect that survives is the replicated phase-2 comparison of §7 (+6.9 pp, 15 arms, permutation p=0.0008): it is inside the band geometrically, but it is estimated across 15 independently trained arms rather than read off a single run, which is precisely the difference the floor teaches. The language-matched accuracy cost (−6.9 pp) is drawn as an open (unfilled) marker on purpose: it is a single-run measurement inside the floor, disclosed rather than interpreted, and not a recipe property. The equal magnitudes of the +6.9 and the −6.9 are a coincidence: different lanes, different arms.
Table 3: Two-Phase continues the same adapter from Reasoning onto the hybrid mix; One-Phase never separates the two halves. The three recipes fail in different ways (§7).
nametrained onrows
Basenothing (the released model)
Reasoningreasoning rows only59,107
Subseta subset of the same rows15,607
Two-Phasereasoning rows, then + the direct half59,107+
One-Phaseone pass over both halves at once59,107+
Figure 4: The same two checkpoints on four dimensions, the base and the structure-selected reasoning-only arm (Subset, the “Reason-16k” of Figure 11), on the 1,000-item probe lane. Only the first is accuracy.
Figure 4: The same two checkpoints on four dimensions, the base and the structure-selected reasoning-only arm (Subset, the “Reason-16k” of Figure 11), on the 1,000-item probe lane. Only the first is accuracy.
Table 4: sd=4.4 pp, range =7.7 pp. Note the last column.
seedmeanlogicfallbacktrace Greek
4276.256.210%1.00
4368.735.641%1.00
4476.454.03%1.00
Figure 5: Median trace length in tokens (each family’s own tokenizer, 5,156 matched items), base against the fine-tuned release. The word-level shortening must repay Greek’s 2.3–2.5× token fertility before it becomes a serving saving: Qwen’s does, NemotronH’s breaks even, Gpt-OSS’s does not.
Figure 5: Median trace length in tokens (each family’s own tokenizer, 5,156 matched items), base against the fine-tuned release. The word-level shortening must repay Greek’s 2.3–2.5× token fertility before it becomes a serving saving: Qwen’s does, NemotronH’s breaks even, Gpt-OSS’s does not.
Table 5: Selection buys −0.5 pp (0.31​σ); a 4× smaller pool costs nothing. The 62,562 is the reasoning count of the pre-gate pool the experiment was run against (62,562 reasoning, 60,214 direct); the 59,107 quoted everywhere else is the post-gate train half (59,107 reasoning, 58,985 direct), so the table’s all row and the paper’s reasoning-half count differ by the 3,455 rows the decontamination/dedup gate and the val reserve removed.
phase-1 rowsselectionmean
62,562all69.5
15,607top-by-structure69.5
15,607uniform random69.0
Figure 6: Overrun = trace ≥3× the median trace for that item across arms. Easy = at least 80% of arms answer correctly. Cells pool the arms of each column; hard-item think-mode rates are compressed by generation-cap truncation (§5), so the easy row carries the section’s claims.
Figure 6: Overrun = trace ≥3× the median trace for that item across arms. Easy = at least 80% of arms answer correctly. Cells pool the arms of each column; hard-item think-mode rates are compressed by generation-cap truncation (§5), so the easy row carries the section’s claims.
Table 6: Per-domain results on the Greek think lane; here and in every table, ↑ marks metrics where higher is better and ↓ where lower is better (math = the two math slices, commonsense = HellaSwag+Winogrande, logic = ProofWriter macro-recall). Rows are grouped by family, each release indented under its own base, so every legitimate accuracy delta is a within-block comparison and the cross-base reading (the one an earlier draft of this paper fell for) is structurally discouraged. Every release now carries its own base row: the NemotronH base dump is the same vintage as the Qwen base dump, and the Nemotron-3.5 base lane, absent until this revision, was generated with the identical lane settings and scored with the same scorer as every other row. The same base is also measured on the NLU retention suite (Table 14, Greek macro 57.5), a direct-mode instrument whose numbers are not comparable to this think-lane table. The Nano release also appears in the retention suite (Table 13), and its fallback shape is the mirror of Gpt-OSS’s: negligible on math and commonsense (1.7/1.8%) but 39.9% on logic, so its 9.8 logic macro-recall is substantially a format floor on the one domain where its answer form breaks. Accuracy and fallback move together: every release keeps the base’s domain ordering, and the Sophea-OSS format-repair dose pulls the commonsense fallback back toward math’s level (83.5→25.7) while cutting logic’s by a third (77.4→50.9). The pre-repair row is reference only; the release is the repaired arm. Conditioning on rows that emit the requested answer line (excluding the fallback-scored path) shrinks the apparent deficits sharply: the Qwen release reads 95.8/78.8/39.5 against its base’s 96.0/82.8/42.7, and the same condition makes Gpt-OSS pre-repair 74.6→92.2 / 53.7→60.2 / 25.2→36.5 and Sophea-OSS-v1 78.6→94.4 / 53.3→58.2 / 26.5→30.0 (math/commonsense/logic), so a substantial share of every raw gap is format compliance, not reasoning.
anchored accuracyfallback %
armmathcslogicmathcslogic
↑ higher is better↓ lower is better
Qwen base (Greek think)92.982.442.63.30.50.2
Sophea-Qwen3.6-v183.375.837.318.222.946.3
Gpt-OSS base (Greek think)90.763.232.22.45.20.7
before repair (reference)74.653.725.234.983.577.4
Sophea-OSS-v1 (repaired)78.653.326.517.225.750.9
NemotronH base (Greek think)86.649.226.88.36.26.6
Sophea-Nemo-3-Nano-v171.539.79.81.71.839.9
Nemotron-3.5 base (Greek think)76.946.325.01.41.94.4
Sophea-Nemo-3.5-Lightning-v175.640.923.816.114.75.3
Figure 7: Each point is an independently trained arm. Corpus version varies within both groups.
Figure 7: Each point is an independently trained arm. Corpus version varies within both groups.
Table 7: Language-matching across the three families: all four released checkpoints, the two measurable bases, and the one-directional reference arm. Greek fid.: Greek-trace fidelity on the Greek benchmark; both bases sit at exactly 0.0 (0/5,156 traces reach ratio 0.9), and all four language-matched checkpoints land at the same level (97.4–98.1%). EN ctrl: English-control compliance, 100% for every arm. Switches: in-question language switches per 100 sentences on the Greek lane; the bases switch 16.4 (Qwen) and 9.7 (Gpt-OSS) times, every fine-tuned arm (the one-directional reference included) zero. The base fidelity and switch cells are one instrument run: both base dumps rescored with the current scorer on the same day (an earlier draft printed 19.0 for the Qwen base from an earlier dump vintage). Leak: answer-channel leak, the one place families differ (10.3% Sophea-OSS-v1, 5.2% Lightning, 3.5% Qwen, 0.0% Nano, the only arm at zero; base leak is measured on the English control, fine-tuned leak on the Greek think lane). †Arms of the earlier one-directional recipe, shown for reference (a different recipe, not releases); the Qwen one-directional row is the recipe-evolution comparison: it locks against explicit language instructions (Table 10), not against English questions. §Measured on the 1,900-item paired probe (99.3% English, 0.0% Greek, 0.7% too short to score); this arm was never run on the 1,100-item control the other EN-ctrl cells use. ‡Measured on the pre-repair language-matched arm: the repair dose adds 2% trace-less answer-format rows and does not touch trace language, but the released arm’s own English control has not been generatively re-run. The recipe generalises to every family tried; the leak’s size does not.
Greek fid.EN ctrlswitchesleak
arm(%)↑(%)↑/100 sent.↓(%)↓
Qwen base0.010016.415.6
Qwen lang-matched98.01000.03.5
Qwen one-dir.†98.799.3§0.01.5
Gpt-OSS base0.01009.70.5
Gpt-OSS lang-matched98.1100‡0.010.3
Nemotron-3.5 lang-matched98.11000.05.2
NemotronH lang-matched97.41000.00.0
NemotronH one-dir.†98.41000.011.3
Figure 8: The four withdrawn findings of this section, drawn: the effect as first believed (open circle) against the same quantity after its control (filled). Three flip sign; the un-anchored scorer artifact (+29.8 pp, 18​σ, one probe) collapses entirely: its endpoint is drawn at zero because the prose reports the artifact removed by anchored scoring, not as a measured residual. Rows are different instruments and lanes, so magnitudes compare within a row, not across rows; the other pairs’ numbers appear in this section’s prose. The section’s fifth failure (§8.5) is a scorer defect rather than an effect-size withdrawal, so it has no before/after pair to draw.
Figure 8: The four withdrawn findings of this section, drawn: the effect as first believed (open circle) against the same quantity after its control (filled). Three flip sign; the un-anchored scorer artifact (+29.8 pp, 18​σ, one probe) collapses entirely: its endpoint is drawn at zero because the prose reports the artifact removed by anchored scoring, not as a measured residual. Rows are different instruments and lanes, so magnitudes compare within a row, not across rows; the other pairs’ numbers appear in this section’s prose. The section’s fifth failure (§8.5) is a scorer defect rather than an effect-size withdrawal, so it has no before/after pair to draw.
Table 8: single-phase = one pass over both halves; two-phase = reasoning-only then hybrid; reasoning-only = the reasoning half alone. Empty-trace rate is measured in think mode, fallback% in direct mode; means are over all arms of each recipe. Fallback ranges are over the individual arms of each recipe (2–12% across the three reasoning-only arms; 33–40% across the two-phase arms); the three seed replicates of §4 widen it further (3–41% across seeds 42–44, all three of which are among the six phase-1-only arms above), so the fuller picture is 2–41% when seed noise is admitted.
recipemeanemptyfallback
One-Phase66.123.6%12%
Two-Phase64.4–69.90.0–1.3%33–40%
reasoning-only73.60.0%2–12%
Figure 9: Trace-language composition per lane, Qwen line: the share of traces that are Greek (character ratio ≥0.9), mixed, or English (≤0.1), on the same instrument as Table 7’s fidelity column. Greek-question rows are the full 5,156-item benchmark; English-question rows are the 1,100-item control (base) and the 1,900-item probe (matched). Two honesty notes drawn rather than hidden: the base’s Greek-question traces are not mostly pure English: only 4.2% are; the mass is mixed script with English scaffolding (median ratio 0.33, §5); and the one-directional arm’s Greek-question lane decomposes as 98.7 Greek / 1.3 mixed / 0.0 English on the same instrument (n=5,156, uninstructed); its English-question lane, generated later on the same 1,900-item probe, measures 99.3% English (Table 7). Uninstructed defaults only; the instructed override is Table 10.
Figure 9: Trace-language composition per lane, Qwen line: the share of traces that are Greek (character ratio ≥0.9), mixed, or English (≤0.1), on the same instrument as Table 7’s fidelity column. Greek-question rows are the full 5,156-item benchmark; English-question rows are the 1,100-item control (base) and the 1,900-item probe (matched). Two honesty notes drawn rather than hidden: the base’s Greek-question traces are not mostly pure English: only 4.2% are; the mass is mixed script with English scaffolding (median ratio 0.33, §5); and the one-directional arm’s Greek-question lane decomposes as 98.7 Greek / 1.3 mixed / 0.0 English on the same instrument (n=5,156, uninstructed); its English-question lane, generated later on the same 1,900-item probe, measures 99.3% English (Table 7). Uninstructed defaults only; the instructed override is Table 10.
Table 9: The base is ∼5 pp worse on our translations; every fine-tune is slightly better.
armhumanmachineΔ
Base95.690.7−4.9 (−3.14​σ)
Subset94.095.1+1.1
Reasoning93.695.3+1.7
Two-Phase87.690.4+2.8
Figure 10: Every per-benchmark retention delta behind Table 14: fine-tuned release minus its own base (Lightning against the Lightning base; Sophea-OSS-v1 is the format-repaired release), Titan-1 suite, non-reasoning mode. Per-benchmark binomial sigma applies and a delta is a direction, not a test (Table 13); the value is the pattern: the Qwen column is near-white in both languages, the Sophea-OSS-v1 residual concentrates on the knowledge-heavy benchmarks (MMLU-el −10.1) while HellaSwag gains in both languages on every family, and MMLU is the largest English loss for all three non-Qwen releases.
Figure 10: Every per-benchmark retention delta behind Table 14: fine-tuned release minus its own base (Lightning against the Lightning base; Sophea-OSS-v1 is the format-repaired release), Titan-1 suite, non-reasoning mode. Per-benchmark binomial sigma applies and a delta is a direction, not a test (Table 13); the value is the pattern: the Qwen column is near-white in both languages, the Sophea-OSS-v1 residual concentrates on the knowledge-heavy benchmarks (MMLU-el −10.1) while HellaSwag gains in both languages on every family, and MMLU is the largest English loss for all three non-Qwen releases.
Table 10: The one-directional fine-tune’s traces are identical under both instructions, and it does not matter how the instruction is phrased. Rows classified as neither/mixed scripts (28% of the base under the Greek instruction) are omitted, so rows need not sum to 100. The matched ft rows, measured after the language-matched re-training of §10, show the override partially returning; its mixed share under the English instruction is 30.4%.
model, instructionEnglish tracesGreek traces
base, “reason in English”100.0%0.0%
base, “reason in Greek”72.0%0.0%
fine-tune, “reason in English”0.0%98.7%
fine-tune, “reason in Greek”0.0%98.7%
matched ft, “reason in English”44.8%24.8%
matched ft, “reason in Greek”0.0%98.5%
Figure 11: Left: accuracy per axis on the 5,156-item benchmark for the Qwen recipe arms (the cross-family releases are Table 6), with the seed-to-seed noise floor shaded around the Qwen base. Right: the same arms’ cost in words per correct answer (token-denominated cost is Figure 5). Math uses the 250 human-translated items only (§8); logic is macro-recall (§3).
Figure 11: Left: accuracy per axis on the 5,156-item benchmark for the Qwen recipe arms (the cross-family releases are Table 6), with the seed-to-seed noise floor shaded around the Qwen base. Right: the same arms’ cost in words per correct answer (token-denominated cost is Figure 5). Math uses the 250 human-translated items only (§8); logic is macro-recall (§3).
Table 11: Instructed-override compliance on the five released checkpoints: the share of traces in the instructed language when the instruction contradicts the question (n=1,000 Greek-question and n=1,100 English-question items; same scorer as Table 10). Every release obeys “switch to Greek”; only the Qwen line and Gpt-OSS obey “switch to English”, and the RLVR refinement widens Qwen’s opening (44.8→53.9%, §15.1). Two instrument notes, both checked: the Nemotron dumps do not separate trace from answer, so the mandatory Greek answer line falls inside the scored text: re-scoring with that line stripped leaves both Nemotron arms at 0.0% (the Qwen control moves 44.8→49.5%, the expected direction); and the Nemotron arms return 5–9% of rows too short to score against Qwen’s 0%, consistent with the low reasoning ceiling reported in §13.
released checkpoint“in English”↑“in Greek”↑
(Greek Q)(English Q)
Sophea-Qwen3.6-v144.8%83.7%
Sophea-Qwen3.6-v1.1 (RLVR)53.9%85.7%
Sophea-OSS-v162.5%93.3%
Sophea-Nemo-3.5-Lightning-v10.0%92.7%
Sophea-Nemo-3-Nano-v10.0%87.5%
one-directional (reference)0.0%91.4%
Table 12: English questions get English traces from every arm measured. The fine-tunes in the first three rows are the dual-mode / reasoning-recipe checkpoints (§7), kept deliberately: they are the recipe-class evidence that one-directional training does not lock English questions. The last two rows are the language-matched checkpoints: the released Qwen checkpoint, re-run on this exact control (94.4, +0.4 over its base, 100% English traces, consistent with its 1,900-item probe above), and the Gpt-OSS language-matched arm evaluated after merging (the pre-repair arm; the released, format-repaired checkpoint’s control has not been generatively re-run, Table 7). Together the rows confirm the language-matching result is not Qwen-specific; none of this retests the hard one-directional lock of Table 10, which remains a property of training in a single output direction.
arm (n=1,100)accuracyreasons in EN
Qwen base / ft94.0 / 96.2100% / 100%
Gpt-OSS base / ft95.5 / 96.1100% / 100%
NemotronH base / ft92.7 / 93.7100% / 100%
Qwen language-matched94.4100%
Gpt-OSS language-matched91.9100%
Table 13: Forgetting on the Titan-1 suite. Top: Gpt-OSS on the nine Greek non-reasoning benchmarks, base against the language-matched fine-tune, before and after the format-repair dose: the language-matched fine-tune initially costs −7.3 points, and the repair dose recovers 4.1 of them, so the residual is −3.2 not zero. Below: the macro deltas this paper reports per family, fine-tuned arm vs its own base; the format-repaired Gpt-OSS is the fine-tuned arm we stand behind. Qwen is flat in both languages; Nano gains Greek (+3.8) against the family’s low 0.49 Nano base (mostly catch-up, not forgetting); Lightning is read against its own base, measured separately (Greek macro 57.5, well above Nano’s 48.6), and gains +1.7; an earlier +10.6 reading against the shared Nano base is superseded by that measurement. Per benchmark against binomial standard error; a macro delta is a direction, not a test. †On English the pre-repair language-matched Gpt-OSS lost −7.7 points (75.7→68.0); re-measured on the repaired release, the English macro is 76.7, +1.0 above base: the repair dose recovered both lanes, so the pre-repair loss was format behaviour throughout. Macro deltas are computed on unrounded macros; Table 14 prints one decimal, so its Qwen columns round to 75.7→75.6 Greek / 85.1→85.2 English.
Greek macrobaselanguage-matched+ format repair
Gpt-OSS61.253.958.0
macro deltas vs base (points):
Sophea-Qwen3.6-v1−0.01 Greek / +0.08 English
Sophea-OSS-v1−3.2 Greek / +1.0 English†
Sophea-Nemo-3-Nano-v1+3.8 Greek / −2.0 English
Sophea-Nemo-3.5-Lightning-v1+1.7 Greek / −1.1 English
Table 14: Every benchmark behind the macros of Table 13 (Titan-1 suite, non-reasoning mode; accuracy %). Qwen ft is Sophea-Qwen3.6-v1 (language-matched); Gpt-OSS ft is Sophea-OSS-v1, the format-repaired release (its pre-repair intermediate is in Table 13); Nano ft is Sophea-Nemo-3-Nano-v1, the language-matched Nano release; Lightning ft is Sophea-Nemo-3.5-Lightning-v1 (language-matched); the two Nemotron generations share the same 31.6B / 6-of-128 routing profile (Table 1) and each is read against its own base column. (The earlier one-directional recipe arm is reported only in §13’s recipe history, not as a release.) The per-benchmark view localises the Gpt-OSS story: before the repair the loss concentrated on the instruction-format-heavy reading benchmarks (Belebele-el −17.7, ARC-Challenge −13.1 el / −19.1 en, MMLU-en −16.0) while HellaSwag and Winogrande gained in both languages, and the repair dose recovers most where the loss was format (Belebele-el regains +13.0 of the 17.7). English retention on the released Gpt-OSS checkpoint recovers to +1.0 above its base; Lightning retains English within 1.1 points of its own base.
QwenGpt-OSSNemotron
benchmarkbaseftbaseftNano baseNano ftLtng. baseLtng. ft
Greek (nine benchmarks, the Greek macro of Table 13):
ARC-Challenge-el91.892.077.168.654.259.068.772.8
ARC-Easy-el96.997.185.479.962.370.175.881.4
Belebele-el92.793.786.081.367.078.081.383.3
GreekMMLU83.883.066.361.358.853.364.262.2
HellaSwag-el57.659.442.952.139.247.644.551.2
Medical-MCQA-el81.280.342.138.727.831.241.738.2
TruthfulQA-el39.136.539.735.531.733.234.233.7
Winogrande-el61.062.555.459.653.358.355.860.1
MMLU-el76.976.255.545.443.240.851.349.6
Greek macro75.775.661.258.048.652.457.559.2
English retention (five benchmarks):
ARC-Challenge96.195.790.086.888.382.292.089.1
ARC-Easy99.199.196.095.796.293.497.896.4
HellaSwag73.274.257.068.467.973.172.775.9
Winogrande73.274.264.666.769.972.972.073.3
MMLU83.982.670.865.869.960.575.869.7
English macro85.185.275.776.778.476.482.080.9
Table 15: Supported and unsupported configuration choices, each with its evidential basis.
Supported by our evidencebasis
Drop the hybrid phase 2; train reasoning-only+6.9 pp over 15 runs, p=0.0008 (§7); and the direct mode it sacrifices costs nothing measurable on direct-mode NLU (E3, §11)
Generate exploratory traces, not explanationsstructure 0.27→0.68 on the same questions
Forbid formulaic openings in the generator24/24 identical openings without it
Report fallback% beside accuracyseparates reasoning from format compliance
Run the seed control before any ablation7.7 pp (§4)
Probe trace-language steerability per checkpointsame recipe, four checkpoints, 62.5/44.8/0/0 (§10.1)
Keep a human-translated benchmark subsetcaught a 3.1​σ artifact (§8)
Test numeric normalisation in the target language’s own convention5 of 6 Greek-thousands answers scored wrong while right (§8.5)
Gate metrics on length correlationremoved 3 of our 7
Fix format and leak defects with verifiable-reward RL, not more SFT datafallback →2.5%, leak →0%, control flat, ∼78 GPU-h/arm (§15.1)
Run a random-reward control arm with any RLVR claimit reproduced our baseline on every axis; without it, every gain here would be unattributable (§15.1)
NOT supportedwhy
A specific corpus versionfive expansions, all inside the noise floor
A data-selection methodselection ≈ random, −0.5 pp (0.31​σ)
A LoRA rank, stride, or epoch countnever varied; no evidence either way
A per-domain model choicetwo of three per-domain score spreads exceed the seed range itself (§12)
Table 16: The pre-registered RLVR round, scored on the held-out instruments (Greek think lane n=5,156; override lanes n=1,000/1,100), same scorer, same day. The before row is the released checkpoint’s frozen dump rescored with the current scorer; its full same-day regeneration reproduced all 5,156 responses bit-identically (greedy decoding is deterministic here), and the plain-mode lanes to the decimal (44.8/83.7): the before column is not a provenance caveat but a verified constant. The control row is the entire admissibility argument: on every axis the random-reward arm reproduces the baseline, so nothing below can be elicitation. Accuracy is shown for completeness only: +3.3–3.5 is inside the ±7.7 seed floor of §4 and is claimed by nobody.
Greekfall-overridereverse
armfid. %↑acc↑back %↓leak %↓el→EN %↑en→EL %↑
SFT checkpoint (before)97.9873.724.13.5344.883.7
random reward (control)98.0674.022.13.6144.183.8
correct+format+term.98.2277.22.50.0048.884.7
+ language98.0277.25.60.0445.984.5
+ override98.2777.02.80.0253.985.7

为什么重要

对于想在资源匮乏语言上部署AI的团队来说,这项研究提醒人们:一个看似漂亮的准确率提升数字很可能只是噪音,而真正重要的改进或缺陷往往体现在标准测试完全测不到的行为细节上。这为评估微调效果提供了一套更可靠的具体方法。

本文术语

  • SFT(监督微调) · 用带标准答案的示例数据对已训练好的模型进行进一步训练,使其学会特定行为
  • RLVR(可验证奖励强化学习) · 利用可以自动核实对错的信号作为奖励,对模型进行强化学习训练的方法
  • MoE(混合专家模型) · 一种模型结构,每次只激活其中一小部分参数,从而让大模型的运行成本更低
  • 随机种子(seed) · 决定训练过程中随机性的数值,仅改变它就可能导致结果不同
  • 语言保真度(language fidelity) · 模型的思考文本实际使用所要求语言的一致程度

论文原文摘要(英文)

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.

作者 · Ayoub Kirouane

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源: Ayoub Kirouane et al., arXiv:2608.17744, arxiv-nonexclusive