Sherpa: Teaching LLMs to Teach Adaptively

Sherpa. Teaching LLMs to Teach Adaptively. By Weixian Xu and four others, from Stanford University, Georgia Tech, and one other institution. Published on the sixth of October. Its code repository is worked out: about four hundred and fifty source files, with a readme and a licence.

Large language models have become capable problem solvers. But solving a problem is not the same as teaching it. Existing ways of training models as teachers rely on demonstrations, on preference data, or on predefined pedagogical criteria that spell out what good teaching is supposed to look like. The trouble is that these signals are often not grounded in how individual students actually learn, and the teaching strategy that works can vary a great deal from one learner to the next.

Sherpa grounds the training signal in learning outcomes. It is a multi-turn reinforcement learning framework in which the teacher is rewarded directly for how much its students learn. The students are simulated, and they are deliberately not all alike.

Sherpa instantiates several student archetypes, each an LLM conditioned on a distinct learning preference. A teacher model then interacts with them over multiple turns and is trained to adapt its instruction by directly maximizing their learning outcomes. Because the archetypes differ, a single fixed style of explanation is not enough. The teacher has to adjust to whoever it is teaching. Notice what is absent from this setup. There are no demonstrations of ideal tutoring to imitate, no preference labels comparing two lessons, and no checklist of pedagogical virtues. The reward is simply whether the simulated students end up learning, which is exactly the grounding the earlier signals were missing.

Teacher models trained with Sherpa improve the performance of the students they instruct across all archetypes, by an average of twenty point five percentage points. Under MathTutorBench's evaluation, the overall pedagogy score rises from fifty-two point five percent to seventy-nine point two percent, which the authors read as better teaching responses. And in human studies, the trained teacher is preferred over the base model in seventy-nine point six percent of pairwise comparisons.

The shift is from describing good teaching to measuring it. Instead of telling a model what a good lesson looks like, Sherpa lets varied simulated learners show whether a lesson worked. The authors see the result as teachers that adapt to diverse simulated students and line up better with human teachers, a step toward AI tutors for real ones.