Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

Dating the Model. Hidden Dates in System Prompts Affect LLM Evaluation. By Mario Sanz-Guerrero and three others, from Johannes Gutenberg University Mainz, Universidad Iberoamericana, and one other institution. Published on the twenty-ninth of September. No code repository is linked to it.

Reproducibility matters in science, and earlier work has already shown that LLM outputs shift with hardware and batching. This work adds a factor that had been overlooked: the current date, quietly injected into system prompts. Users cannot control it, and it changes every day. The claim is that model performance moves with that date and nothing else.

The study covers nine recent LLMs and six datasets, spanning multiple-choice question answering, math reasoning, code generation and machine translation. Changing only the date shifts results by up to six percent on multiple-choice questions, up to fourteen percent on math reasoning, and up to seven percent on code generation. On machine translation the swing reaches two point eight four BLEU.

Model rankings move as well, which reaches into leaderboards. The date effect is larger than other sources of non-determinism the authors compare it with, such as batch size and numerical precision. Standard prompting techniques do not help. Few-shot prompting leaves the sensitivity in place, and chain-of-thought even amplifies it.

These figures are maxima across the nine models and six datasets tested, in four task families. The abstract does not say which direction the date pushes performance, only that it varies, and the recommendation it draws is about evaluation practice rather than any fix inside the models.

Here is a variable that sits in the prompt, changes daily, and is invisible to the person running the test. The authors call for careful evaluation protocols so that comparisons between LLMs stay reproducible and fair.