From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
Olga Tsymboi, Dmitrii Stoianov, Ramil Latypov, Danil Taranets, Daniil Dryabin, Mikhail Gashkov, Viktor Zelenkovskiy, Aleksandr Fida and 6 others
T-Tech
Published September 1, 2026 · on the feed September 2, 2026 · ▲ 33 · arXiv 2609.01572
The abstract
Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
The Brief
2:16A short podcast summary of the paper. Written and voiced by AI.
Transcript
From Production Traffic to Post-Training. Building a Self-Hosted LLM That Covers the Corporate Request Mix. By Olga Tsymboi and thirteen others, from T-Tech. Published on the first of September. No code repository is linked to it.
Data-residency rules force enterprises to host their own language models. Each time a newer model is adopted without retiring the older ones, the serving fleet grows, and a finite GPU pool gets split into fragments. The aim here is the reverse: consolidate the traffic of more than two hundred internal applications onto one model. For that to work, the single model has to be good enough at everything those applications already ask of the fleet, and that means finding out where it falls short.
To get there, the team closes the quality gaps that production error analysis reveals, along three axes: instruction following, function-calling, and the company's own internal task distribution. Quality is tracked with offline benchmarks stratified to match production traffic, scored either by deterministic verifiers or by calibrated model judges.
Optimising all three objectives together causes reward interference across domains. So instead they train a separate GRPO expert for each axis, then merge the experts with a two-stage SLERP. Each expert's reward exposed its own failure mode. Semantic collapse in one, over-calling in another, verbosity hacking in the third. Each needed a fix specific to its domain. Each failure belonged to one expert's reward and was handled on that axis alone, before the merge brought the three experts together into a single model.
In non-reasoning mode, the result beats a baseline roughly seven times larger by total parameters. On the in-house Arena it scores sixty-nine point six against sixty-five point eight. On instruction following it scores zero point eight five against zero point eight three, and on function-calling zero point seven nine against zero point seven seven. General dialogue benchmarks go up as well.
The real test is the platform itself. In production, the model now takes half of the platform's traffic, a hundred and sixteen million requests a month, at a fraction of the serving cost.
The recipe starts from the traffic the company actually sees and the errors it actually makes, and it keeps the three repairs apart until the very end, because training them jointly made them interfere.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
Too many models, too few GPUs. Here is the wall.
Data-residency rules mean your company hosts its own language models. Each newer model is added without retiring the older ones, so the fleet keeps growing and a finite GPU pool gets split into fragments.
The road ahead
- 1 · The problem ~7 min A company that must host its own language models keeps adding models, splitting its GPUs, and one model is not yet good enough for everything.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Train one RL expert per quality axis on its own reward, fix each reward's hacking, then merge the experts into one model with two-stage SLERP.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Find gaps in production traffic, train one GRPO expert per gap, merge them with two-stage SLERP, then move platform traffic onto the merged model.
- 6 · The experiment ~6 min You’ll guess: After consolidation, what share of the corporate platform's traffic did the single self-hosted model end up absorbing?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.