ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker

ZooWork-ShopRanker. An Open, Preference-Aligned E-Commerce Reranker. By Siqiao Xue and two others, from the ZooWork Team. Published on the twenty-fifth of September. No code repository is linked to it.

ZooWork-ShopRanker is a family of open rerankers built for e-commerce search, released together with a benchmark called ShopRank-Bench. The motivation is a transfer problem. Open rerankers trained for general web retrieval carry over imperfectly to shopping, because there a ranking decision depends not only on topical relevance, but also on user preferences, product constraints, and how well one product fits compared with another. Those preference signals are difficult to supervise at scale. Real search traffic supplies authentic queries and candidates, but no clean pairwise labels saying which product is the better one.

The family comes in three sizes: zero point six billion, four billion, and eight billion. Each is aligned to judge-labeled shopping preference. ShopRank-Bench is a contamination-limited benchmark of about ten thousand preference pairs taken from private traffic. It comes in two text formats, and its pairs are tiered by how many judge families committed to each label, so a pair every family agreed on can be told apart from one where fewer did.

The labels come from a panel of reasoning large language models, drawn from different families and acting together as a preference oracle. Their judgments are position-debiased and sorted into agreement tiers, and the rerankers are trained on those labels. The aligned eight billion flagship then serves as a distillation teacher. The four billion model and the zero point six billion model are fit to its scores, and then sharpened on judged pairs.

On the benchmark, the eight billion and four billion versions significantly outperform the strongest open reranker baseline. Every model in the family significantly beats its own un-aligned base, and the zero point six billion model beats its size peer. The gains hold in both formats, and they extend to common MTEB benchmarks. Anyone building product search on open components gets rerankers at three sizes, the smaller two described as efficient, and a benchmark that measures shopping preference rather than topic alone.

The design choice at the center is who decides what a better product is. Real traffic supplies the questions but not the answers, so the answers come from a panel of models from different families, with their level of agreement recorded rather than assumed. The models and the dual-format ShopRank-Bench are both released to support further research.