DuoLM: Fast and Effective Personalized Recommendation with Language Models

DuoLM. Fast and Effective Personalized Recommendation with Language Models. By Jiang Wu and four others, from Independent Researcher, the University of California, Irvine, and two other institutions. Published on the twentieth of October. Its code repository is worked out: about seventy source files, with a readme and a licence.

Language models can recommend from rich material: the content of an item, a user's history, and the feedback that user has given. The cost shows up at serving time. Evaluating many candidates with full-text inputs means substantial online computation, and that limits how much throughput a recommender can deliver. Quality from reading everything in full is bought with speed.

DuoLM splits the work between two language models and pairs them with a serving runtime built for the job. An encoder LLM learns compact representations of items, and those representations are computed offline and reused. A personalized ranker LLM then combines the compact item representations with the user's history and feedback to make the recommendation. Item text is digested once, ahead of time, rather than read again for every request.

Both models are trained end to end, with two sources of signal: recommendation supervision, and distillation from a teacher. The runtime adds three accelerations to inference. Virtual token IDs give access to the embeddings. History computation is shared. And CUDA Graph replay is used for execution.

On quality, the benchmark results put the framework at a level comparable to the expert teacher model it learns from. On speed, a single H100 delivers approximately twenty-six thousand four hundred candidate scores per second. That is reported as about sixty times the throughput of the same teacher served with vLLM and automatic prefix caching.

The audience here is anyone who wants language-model recommendation without paying full-text prices on every candidate. The claim to weigh is the pairing: teacher-level quality on the benchmark, and a throughput multiple of about sixty on a single H100, from moving item reading offline and tightening the runtime around what remains.