Labels Override Definitions in Jev-Style Typed Decision Models

Labels Override Definitions in Jev-Style Typed Decision Models. By Seyedarmin Azizi and two others, from the University of Southern California. Published on the first of October. No code repository is linked to it.

A typed decision model answers a fixed question about an input by returning a probability for each of several options the caller defines. Each option has a short label and a written definition, and the definition is where a developer states the rule the model should apply. Jev introduced this interface for routing, moderation and triage, and open implementations followed. The same operation occurs whenever a language model is used as a classifier by scoring label strings. The question is whether the probability follows the definitions or the labels. Mostly, it follows the labels. The authors call that preference option-label bias, and they trace it to how the prompt is rendered.

The study looks at the open implementations, whose weights can be inspected and patched. It covers four open-weight typed decision models, three ways of reading an answer from a Qwen2.5 backbone, eleven classification tasks, and PolicyBench, a synthetic routing suite introduced here in which the rule appears only in the definitions.

For laya-td, deleting every definition leaves accuracy unchanged: zero point eight five six against zero point eight four nine. Yet those same definitions, used on their own, support an accuracy of about zero point eight. Renaming the options to A and B raises accuracy by roughly zero point one five. Put those three results together. The definitions carry enough information to do the task, the model barely uses them when labels are present, and taking away meaningful labels helps.

One system, von, is unaffected, and its code base differs from laya's in a single expression. Laya writes each option as its label, a colon, and then its definition. Von writes only the definition. Changing that one expression, in both directions and with no weight changed, makes all three laya checkpoints exactly invariant, and creates the effect in von. When a label contradicts its definition, von's accuracy falls from zero point eight five to zero point two three. The bias moves with the expression, in both directions, which is what makes the location of the cause convincing.

The evidence comes from open implementations specifically, chosen because their weights can be inspected and patched. Within that range, the claim is about location. Earlier work attributed this failure to the constrained decision head these models use in place of a text decoder. These results place it in the prompt rendering instead. That distinction matters for anyone deploying one of these models. A fault in the decision head would call for different weights. A fault in the rendering can be changed in one line, with the weights left alone.

Practitioners get a way to check. The authors give a two-call test that tells which of the two cases applies to a given model, and they measure what four mitigations are worth. The broader caution: when a model reads a label sitting next to its definition, the label may be doing the deciding.