LLMs Get Smarter from Targeted Synthetic Multilingual Data
arXiv:2608.159642026-08-15
Teaching an AI to write its own quiz questions so it stops failing in certain languages
Large language models often answer the same question correctly in English but incorrectly in Spanish or Arabic, a problem called language-specific competency (LSC). The authors built HOTFIXR, a framework that trains a question-generation model to find and target a student model's multilingual weak spots, then uses those questions to retrain the model. This improved performance on the trained tasks by 6.2% and cut performance loss on unseen tasks and languages by 3.7% and 7.1% respectively.
What they did
Language models can give different, sometimes wrong, answers to the same question depending on what language it's asked in, because the model represents meaning differently across languages internally.
Existing fixes either route everything through English (boosting accuracy but losing other languages' expressiveness) or train on balanced multilingual data (making performance even but weaker overall).
HOTFIXR trains a separate 'question generator' model using reinforcement learning (GRPO) to probe a target 'student' model and generate questions that expose its weaknesses.
Weaknesses are measured two ways: how uncertain the student model is even when reasoning in its strongest language (English), and how differently it represents its reasoning when solving the same question in English versus another language.
Trained on six languages (English, French, Spanish, Arabic, Portuguese, Italian) and tested on four unseen languages (German, Japanese, Russian, Chinese), HOTFIXR consistently outperformed prior methods with less forgetting of other skills.
Figure 1: Motivating results for LSC: the performance of Aya (with a more language-balanced pretraining dataset) versus Qwen (with a more English-dominant pretraining dataset) on multilingual HotPotQA (11). We see that although Aya performs more consistently across languages, Aya degrades in performance (see Qwen’s performance in English).
Table 1: HOTFIXR has the best in-distribution performance, the best out-of-distribution performance among the training-based baselines, and remains multilingually consistent. This table shows the cross-lingual consistency with respect to ID and OOD performance. Language spread is the standard deviation of performance across languages. Low spread indicates more consistent performance across languages. Note: the training-based baselines are SelectionGT, SelectionGEN, Filtered, Untrained, and DataEnvGym.
Performance
Language
Method
ID (↑)
OOD (↑)
Spread (↓)
Base
51.9
65.6
9.6
EngReason
50.8
65.4
8.8
SelectionGT
49.2
60.9
9.8
SelectionGEN
50.2
55.8
9.5
Filtered
48.9
58.2
10.6
Untrained
50.4
63.4
9.7
DataEnvGym
49.0
57.4
10.2
HOTFIXR (ours)
56.2
64.7
9.4
Figure 2: Intuition behind HOTFIXR. The “data space” represents the data samples generated by the question generator. Essentially, we want to hill climb through the data space to find samples that have high lingual deficit scores, as they will be most informative to the student model. By fine-tuning the question generation model to generate samples according to the student model’s lingual deficit, we can improve the information embedded within data, improving our models on downstream tasks.
Table 2: Average performance difference of HOTFIXR versus each baseline. The average is over nine settings: three students (Qwen 7B, Qwen 14B, Llama 8B) × three ID/OOD tasks. “Wins” counts the number of settings where HOTFIXR scores higher.
In-Distribution
Out-of-Distribution
Baseline
Δ
Wins
Δ
Wins
Base
+4.3
9/9
-0.9
3/9
EngReason
+5.4
9/9
-0.7
4/9
SelectionGT
+7.0
9/9
+3.8
9/9
SelectionGEN
+6.0
9/9
+8.9
8/9
Filtered
+7.3
9/9
+6.6
8/9
Untrained
+5.8
9/9
+1.4
7/9
DataEnvGym
+7.1
9/9
+7.3
9/9
Average
+6.2
9/9
+3.7
7/9
Figure 3: An illustration of how to train the question generation model in HOTFIXR, along with the reward design.
Table 3: OOD language generalization: performance differences relative to the Base model, averaged across all models and corresponding tasks. Averaging the difference in HOTFIXR and other methods, we can compute how much HOTFIXR avoids catastrophic forgetting. For example, averaged across all languages, HOTFIXR avoids catastrophic forgetting by 8.7% compared to DataEnvGym (10.1-1.4).
Method
De
Ja
Ru
Zh
Avg
SelectionGT
-3.3
-4.7
-9.5
-10.5
-7.0
SelectionGEN
-1.9
-4.0
-22.5
-21.5
-12.5
Filtered
-3.8
-4.8
-14.7
-16.3
-9.9
Untrained
-2.9
-2.0
-2.3
-5.3
-3.1
DataEnvGym
-6.8
-6.0
-12.2
-15.4
-10.1
HOTFIXR
-2.6
-0.7
-0.9
-1.3
-1.4
Figure 4: Performance of data curation methods in-distribution.
Table 4: Cross-lingual spread of student performance, averaged over three student models (Qwen2.5-7B, Qwen2.5-14B, Llama-3.1-8B) and three multilingual tasks (OPUS, MMMLU, mHotpotQA) evaluated with each task’s primary metric. Lower is more consistent for all dispersion columns; Bold marks the best value among data-generation methods. Range is the max − min performance across all languages. Trimmed Range is the best minus the 2nd worst language accuracy. Std is standard deviation of the performance. IQR is the Inter-Quartile Range (difference in the 25% and 75% quartile). CV is the coeffiecient of variation (the standard devation divided by the mean).
Method
Range
Trim. Range
Trim.
Range
Std
IQR
CV
Trim.
Range
Base
20.2
16.0
7.2
9.6
0.12
EngReason
20.7
15.4
7.2
8.8
0.12
SelectionGT
22.3
17.1
7.9
9.8
0.13
SelectionGEN
18.8
14.9
6.9
9.5
0.13
Filtered
22.6
17.1
8.0
10.6
0.14
Untrained
20.1
15.0
7.1
9.7
0.12
DataEnvGym
22.8
17.0
8.0
10.2
0.14
HOTFIXR
19.4
14.6
6.9
9.4
0.11
Figure 5: Performance of data curation methods for factual (MMMLU) queries, an out-of-distribution task.
Table 5: Reward-signal ablation. All rows include the format reward. “Nemo.” is Nemotron, “Trans.” is Translation, “Fact.” is Factual. Highlights the impact each reward has—there is more effect of the reward functions combined, than individually. LAI is the “Language Agnostic Incompetency” and LSI is the “Language Specific Incompetency”, as described in Section 3.
ID
OOD
Model
Config
Nemo.
Trans.
Fact.
RAG
Qwen 7B
Format
52.0
57.6
64.0
75.8
+ LAI
50.9
57.9
64.3
77.2
+ LSI
50.8
57.8
63.6
77.7
+ LAI + LSI
57.8
59.1
64.3
76.7
Llama 8B
Format
48.2
57.8
49.9
70.2
+ LAI
48.5
57.2
49.2
69.2
+ LSI
47.5
57.5
49.3
67.8
+ LAI + LSI
52.5
58.9
49.3
69.7
Qwen 14B
Format
49.3
58.0
71.5
79.8
+ LAI
50.9
58.6
66.3
76.0
+ LSI
48.8
57.2
62.3
76.8
+ LAI + LSI
58.3
58.2
67.9
78.3
Figure 6: Performance of data curation methods for RAG (multilingual HotPotQA) queries, an out-of-distribution task.
Table 6: Minimum and maximum standard deviation (across 3 runs) for the Nemotron task (STEM, MATH, and CHAT), per model and method. This explains the small error bars in Figure 8.
Qwen 7B
Qwen 14B
Llama 8B
Method
min
max
min
max
min
max
Base
0.30
1.50
0.10
0.40
0.20
2.00
EngReason
0.25
1.23
0.16
0.75
0.34
1.00
SelectionGT
0.20
3.40
0.10
1.00
0.40
2.20
SelectionGEN
0.30
3.10
0.00
0.90
0.10
2.20
Filtered
0.70
0.90
0.30
0.80
0.30
1.70
Untrained
0.40
1.90
0.00
1.50
0.30
0.70
DataEnvGym
0.40
1.20
0.10
0.70
0.20
1.70
HOTFIXR
0.10
0.90
0.20
0.40
0.05
0.50
Figure 7: Performance of data curation methods for translation (OPUS-100) queries, an out-of-distribution task.
Table 7: Minimum and maximum standard deviation (across 3 runs) for the Translation task (OPUS-100), per model and method. This explains the small error bars in Figure 10.
Qwen 7B
Qwen 14B
Llama 8B
Method
min
max
min
max
min
max
Base
0.43
0.85
0.41
1.55
0.58
1.56
EngReason
0.40
0.83
0.43
1.05
0.55
1.56
SelectionGT
0.10
1.40
0.20
1.00
0.40
1.60
SelectionGEN
0.00
0.80
0.00
1.50
0.10
1.50
Filtered
0.10
1.60
0.10
0.80
0.10
1.60
Untrained
0.20
0.50
0.00
0.70
0.30
1.70
DataEnvGym
0.00
1.10
0.20
0.70
0.20
0.80
HOTFIXR
0.00
0.90
0.00
0.80
0.20
1.10
Figure 8: Performance of data curation methods for Nemotron queries, per task.
Table 8: Minimum and maximum standard deviation (across 3 runs) for the Factual task (MMMLU), per model and method. This explains the small error bars in Figure 9.
Qwen 7B
Qwen 14B
Llama 8B
Method
min
max
min
max
min
max
Base
0.10
1.50
0.10
1.10
0.00
2.30
EngReason
0.09
1.27
0.28
1.09
0.33
1.73
SelectionGT
0.30
2.30
0.20
1.80
0.10
1.60
SelectionGEN
0.40
2.60
0.10
1.80
0.10
2.30
Filtered
0.10
1.30
0.20
1.20
0.00
1.20
Untrained
0.30
1.00
0.10
1.00
0.20
2.20
DataEnvGym
0.30
2.40
0.10
1.40
0.10
1.20
HOTFIXR
0.30
1.40
0.30
1.30
0.30
2.60
Figure 9: Performance of data curation methods for factual (MMMLU) queries, per language.
Table 9: Minimum and maximum standard deviation (across 3 runs) for the RAG task (mHotPotQA), per model and method. This explains the small error bars in Figure 11.
Qwen 7B
Qwen 14B
Llama 8B
Method
min
max
min
max
min
max
Base
0.10
0.90
0.10
0.80
0.80
2.50
EngReason
0.33
0.99
0.16
1.05
0.53
1.71
SelectionGT
0.30
1.20
0.10
0.80
0.30
1.80
SelectionGEN
0.80
1.40
0.30
1.50
0.00
2.50
Filtered
0.10
1.10
0.40
1.50
0.00
1.30
Untrained
0.20
1.10
0.30
1.10
0.10
1.30
DataEnvGym
0.20
1.40
0.30
0.60
0.30
1.20
HOTFIXR
0.00
0.60
0.20
1.50
0.00
1.40
Figure 10: Performance of data curation methods for translation (OPUS-100) queries, per language.
Table 10: Average performance per model, with the added Distillation method. Here, we are able to show that a lot of HOTFIXR’s empirical success comes from the targeted question generation, rather than having correct, distilled labels from a large model.
Method
Nemotron
Factual
Translation
RAG
Qwen2.5-7B-Instruct
Base
52.3
67.0
59.2
79.0
EngReason
51.2
68.2
59.0
79.3
SelectionGT
48.5
63.9
53.8
61.5
SelectionGEN
50.3
59.9
56.8
41.4
Filtered
48.6
61.7
56.6
66.5
mCOT
47.9
55.3
54.4
53.8
Untrained
51.2
59.2
57.1
72.3
DataEnvGym
49.3
46.2
56.9
47.5
Distillation (32B)
49.6
64.7
58.9
77.3
HOTFIXR
57.8
64.3
59.1
76.7
Qwen2.5-14B-Instruct
Base
55.2
73.7
59.4
79.9
EngReason
54.1
73.0
59.5
81.0
SelectionGT
51.4
67.3
54.2
73.8
SelectionGEN
52.1
70.3
57.4
42.8
Filtered
50.9
59.1
59.1
45.5
Untrained
52.1
71.0
57.9
77.5
DataEnvGym
50.3
62.6
57.4
70.4
Distillation (32B)
53.3
69.5
59.0
78.4
HOTFIXR
58.3
67.9
58.2
78.3
Llama-3.1-8B-Instruct
Base
48.1
48.5
56.3
67.7
EngReason
47.0
47.3
56.4
65.2
SelectionGT
47.6
48.8
57.6
67.2
SelectionGEN
48.1
48.6
57.6
67.2
Filtered
47.2
48.8
57.7
68.4
Untrained
47.7
49.5
57.1
68.6
DataEnvGym
47.6
48.8
57.5
69.1
Distillation (32B)
47.7
51.1
58.0
70.3
Figure 11: Performance of data curation methods for RAG (multilingual HotPotQA) queries, per language.
Why it matters
Many real-world AI products need to work reliably across languages, and this approach improves multilingual performance without the huge cost of retraining a model from scratch. It gives developers a concrete data-generation strategy to fix language-specific weak spots in an already-trained model.
Figure 12: The delta in performance of student models trained with two rounds of HOTFIXR versus one round. The improvement with two rounds is much more than the improvement with one round.
Terms in this paper
LLM (large language model) · an AI model trained on huge amounts of text to answer questions and hold conversations
language-specific competency (LSC) · the phenomenon where a model's accuracy on the same question changes depending on the prompt's language
student model · the language model being improved, which learns from the data HOTFIXR generates
GRPO · a reinforcement learning algorithm used here to optimize the question-generation model based on reward signals
out-of-distribution (OOD) tasks/languages · tasks or languages not seen during training, used to check whether improvements generalize
Original abstract (English)
Language-specific competency (LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses to the same semantic query when prompted in different languages. Prior work attributes this to an internal misalignment of semantic representation across languages. Currently, there are two main approaches to address LSC in the literature: (1) routing all queries through English, improving performance, but limiting language expressivity to English; or (2) training on language-balanced data, equalizing model performance across languages, but reducing overall performance. In this work, we take a data centric perspective and introduce HOTFIXR: Hardness Optimized Training data For Improving X-Lingual Reasoning. It is a data generation framework that uses models to probe and learn a student model's multilingual weaknesses, and generates data to mitigate them. HOTFIXR can generate multilingual synthetic training data that can improve multilingual performance. We evaluate on three in-distribution tasks, three out-of-distribution tasks, and four out-of-distribution languages. On average, HOTFIXR (1) improves in-distribution performance by 6.2%, (2) reduces catastrophic forgetting (induced by fine-tuning) on OOD tasks by 3.7%, and (3) on OOD languages by 7.1%. Overall, as many real-world applications requires multilingual LLMs, our work contributes to the efforts of making LLMs multilingually proficient. We will release code upon acceptance.
