Change the Product, Keep the Parameters: Associative Algebra Layers for Transformers

Change the Product, Keep the Parameters. Associative Algebra Layers for Transformers. By Ilya Koziev and one other, from D A I M L D and the Artificial Intelligence Research Institute. Published on the twenty-sixth of September. No code repository is linked to it.

Fast matrix multiplication algorithms have a fixed target. The product itself never changes: the search is for a cheaper way to compute it. The question posed here steps sideways from that tradition. If a Transformer's projections are learned anyway, does the product they use have to be ordinary matrix multiplication at all, or could a different and cheaper product serve?

The starting point is an associative-algebra construction that swaps ordinary matrix multiplication for a sparser interaction table over the very same weight blocks. The parameters stay. The rule that combines them changes. From that construction the authors build a family whose arithmetic grows quadratically in the matrix dimension when the physical block size is held fixed.

Three properties are established for it. First, finite-shape constraints are derived so that it can execute on a GPU. Second, the construction is provably optimal for its bilinear rank, a guarantee that follows from the bound of Alder and Strassen. Optimality here is a proof, not an empirical observation. Third, it can be realised as row-typed rectangular projections, which remain compatible with causal masking and with KV-cached decoding. That last property matters for where the layer can go: it fits inside an ordinary autoregressive model that generates one token at a time, rather than needing a special setting of its own.

The empirical test is deliberately narrow. Two decoder-only Transformer language models of roughly one hundred and ten million parameters are trained from the same recipe on the same twelve point three billion token budget. They differ only in the feed-forward layer: one uses dense matrix multiplication, the other the associative-algebra product. Across four prompt domains, the algebraic model generates end to end between six point two and seven point eight percent faster. It also scores lower on all three of the downstream metrics reported. Speed moved one way and quality the other, in a comparison built so that nothing else differed.

The authors frame the outcome plainly, as a check of feasibility and trainability at small scale, with further investigation left for later. A modest throughput gain arrives together with a quality cost, and both are stated. What the paper establishes is that the product inside a learned layer is itself a design choice, one that can be changed while the parameters stay put.