In-depth analysis · About 4 min read
AWS’s GPT-OSS-20B experiment separates language compliance from reasoning correctness
A small multilingual fine-tuning study shows where SFT and GRPO helped, while exposing a per-language regression and limits to generalization.
- Published here
- Source published
- Source checked
An October 20 AWS guide reports LoRA fine-tuning of GPT-OSS-20B on SageMaker HyperPod to make generated reasoning and final answers follow a requested language. This is a customization experiment, not a new base-model release. Its central result concerns the language of output, which must not be confused with correct mathematics, factual accuracy or faithful reasoning.
Listen to the news
Read this article aloud or keep listening to the news.
Preparing the reader…
Device voice
Speech is handled by your browser or device. AISubHub does not send the text to an audio API. Voice availability and playback with a locked screen vary by device.
Resuming may repeat a short phrase.
At a glance
- On 50 cases, generated reasoning followed the requested language in 11/50 base outputs and 50/50 after SFT. Final-answer language reached 31/50 after SFT and 35/50 after GRPO.
- The answer-language gain is 8 percentage points, or four cases; Italian fell from 60% to 40%, so improvement was not uniform across languages.
Define what the 50 cases actually measure
The task requests both generated reasoning text and the final answer in one of five languages: English, French, German, Spanish and Italian. The evaluation contains ten math/logic problems across those five languages, for 50 cases. The described training dataset has roughly 949 training and 50 validation examples. Two metrics ask whether reasoning text and answer text use the requested language. They do not score whether a solution is mathematically correct. Base reasoning-language compliance was 22%, including all English examples and one German example. Zero observed successes for French, Spanish or Italian is evidence of a behavior gap in this test, not proof that the model assigns the desired behavior exactly zero probability.
SFT teaches the observed behavior in stages
The guide’s supervised run uses LoRA rank 8 and alpha 16 for 1,000 steps, with 16 L40S 48GB GPUs across four ml.g6e.12xlarge instances, FSDP sharding, bf16 and gradient checkpointing. AWS reports reasoning-language compliance reaching 100% around step 200; the step-1000 evaluation still gives only 62% answer-language compliance. Saying the model learned the target language therefore needs two qualifications: which output segment and which evaluation set. SFT and the initial GPT-OSS GRPO run use TRL. The broader infrastructure includes HyperPod with EKS and FSx for Lustre; the later Qwen control uses a different training stack. These configurations are not interchangeable performance measurements.
GRPO refines the answer language, with a trade-off
Starting from the SFT checkpoint, the experiment merges its adapter and adds a new rank-8, alpha-16 LoRA, with eight sampled generations per prompt. Rewards emphasize answer language at ±5, reasoning language at ±1.5 and a smaller brevity term. GRPO compares sampled responses within a group and favors relatively better rewards; diversity in useful candidates matters. The selected result keeps reasoning-language compliance at 100% and raises answer-language compliance from 62% to 70%. French rises 60→70%, German 60→80%, Spanish 30→60%, English stays 100%, but Italian drops 60→40%. With ten cases per language, each case changes its rate by ten points. These numbers support a modest aggregate refinement, not a universal multilingual improvement.
The control experiment tests headroom, not a universal ceiling
AWS also reports an internal Qwen2.5-7B-Instruct comparison using veRL, Ray, KubeRay and vLLM. On its multilingual task, SFT-to-GRPO answer-language compliance rises 38→44%, while reasoning-language compliance rises 96→98%. On 200 GSM8K cases, the base and GRPO at step 30 both score 181/200, or 90.5%. The authors connect the lack of gain to sampled groups with little relative reward signal. This does not prove the model has reached a global quality ceiling or that every other GRPO setup must fail. Before spending training resources, inspect sample diversity, reward definition and target-language regressions, and use a separately defined acceptance set. AISubHub did not train a model, run these manifests or incur API charges to reproduce the results.
Why it matters
AISubHub analysis: choose an adaptation method against a defined behavior gap, then inspect each language and failure mode. An aggregate gain can coexist with a regression in a deployment’s most important language.
Limits and availability
These are AWS-reported experiments, not AISubHub measurements. The small language-compliance evaluation does not establish general reasoning quality, broad multilingual coverage or training economics. The separate Qwen control repository was not yet public according to the guide.
Sources and evidence
Source published: · Source checked:
AI-assisted editorial articles checked against linked primary sources. Vendor claims are attributed; inclusion is not endorsement.