CoTranslator: Reverse to Forward Chain-of-Thought Translator for Reasoning Distillation

CoTranslator. Reverse to Forward Chain-of-Thought Translator for Reasoning Distillation. By Zinan Tang and seven others, from Tsinghua University, Ant Group, and one other institution. Published on the fourteenth of October. No code repository is linked to it.

Distilling reasoning into a student needs chain-of-thought data that ends in correct answers, and getting that data efficiently is hard. There are two usual routes, each with a cost. Forward generation has the teacher answer from the question alone, then rejection sampling keeps the responses that check out. The result is verified, and it matches how the student will work, seeing only the question. But for difficult problems it can take many teacher calls. Reverse generation is cheaper, because it conditions on the known answer. That same access to the answer can produce post-hoc rationalization, and it creates a mismatch with the student, which will have to reason without the answer.

CoTranslator sits between the two. It learns to translate reverse chain-of-thought responses toward the distribution of verified forward responses. The cheap route supplies the volume, and the translator moves its output toward what the expensive route would have produced.

The translator is trained on a small set of pairs: a reverse response, and a rejection-sampled forward response. Once trained, it supports generating and verifying student training data at large scale.

Six student models were evaluated on six mathematics benchmarks. Against semantic suppression, CoTranslator improves average accuracy by up to five point eight two percentage points. For two students of four billion parameters, it keeps ninety-five percent of rejection sampling's average accuracy while using thirty-five point nine percent of its estimated compute.

The translator also transfers without retraining: to a new mathematics dataset, to a different teacher, and to STEM problems. On STEM, average accuracy rises by four point eight eight percentage points for a student of four billion parameters, and by four point three one points for one of eight billion.

The trade the method offers is plain in its numbers. Rejection sampling buys correctness and the right inference setting with many teacher calls. Reverse generation is cheap but leans on the answer. A translator trained on a small number of paired examples, for the smaller students, recovers nearly all of the first route's accuracy at roughly a third of its compute, and it keeps working when the data, the teacher, or the subject changes.