Low-Resource Reasoning: What SFT Changes and RL Repairs

Introduction

Training a model to reason in a low-resource language is not the same as making it answer more questions correctly. A study using Greek as its test case argues that the most important effects of language adaptation may sit outside conventional accuracy metrics. Whether a model reasons in the language its user can read, follows the requested output format, and switches languages when instructed may matter just as much as the final answer.

The researchers adapted several mixture-of-experts models with roughly 3.6B–4.0B active parameters. They compared supervised fine-tuning (SFT) with reinforcement learning from verifiable rewards (RLVR), while also testing whether their behavioral instruments were accidentally measuring output length rather than the intended properties.

Key findings

• Accuracy was too noisy to carry the story. Overall benchmark accuracy barely moved, but changing only the random seed caused a 7.7-point swing. That was larger than every data and recipe effect measured in the study, making accuracy alone a weak signal for this type of adaptation.
• Base models did not reason in Greek. None of 1,000 sampled reasoning traces used Greek, even when the question was written in Greek. A model could still produce a correct answer, but the reasoning remained inaccessible to users who wanted to inspect or correct it.
• SFT changed the visible reasoning habit. After fine-tuning, released checkpoints reasoned in the question's language on about 98% of items. One model family achieved this with three times fewer tokens. Grammaticality improved across all four model families, while general ability stayed within a few points of the corresponding base models.
• SFT left protocol failures behind. About a quarter of answers ignored the requested format, some answers appeared in the reasoning channel, and an explicit instruction to think in English was followed less than half the time. Language fluency and reliable interaction control are therefore separate capabilities.
• RLVR repaired verifiable behaviors. In a pre-registered experiment, format fallback fell from 24% to 2.5%, and reasoning-channel leakage fell from 3.5% to 0%. Compliance with the English-thinking instruction increased by 9.1 percentage points. An accuracy-only training signal did not remove the learned Greek reasoning habit.

Why it matters

The study offers a more useful decomposition of multilingual reasoning. Evaluation should ask not only whether the answer is correct, but also whether the reasoning is expressed in the requested language, whether it is grammatical, whether the output follows the required structure, whether answer and reasoning channels remain separated, and whether the model can obey a language-switching instruction.

This is also a warning about measurement. The authors describe six ways their own instruments produced misleading results, with controls used to catch each failure. A metric that rises merely because a response becomes longer is not evidence of better reasoning. Future evaluations should therefore report length controls, seed sensitivity, general-capability retention, and instruction compliance alongside accuracy.

Greek is best understood as a measurable case study rather than the endpoint. For other low-resource languages, SFT may teach a model to express reasoning in the right language, while RL helps it act within the rules of an interactive protocol. Both layers are needed for reasoning that is readable, auditable, and controllable.

Source: Hugging Face Daily Papers