Timestral

Timestral. By Adrien Petralia and three hundred and fifty-five others, from Mistral AI. Published on the twenty-seventh of October.

Probabilistic forecasting asks a model to say not just what a time series will do next, but how uncertain that is. This report presents a family of time series foundation models aimed at probabilistic forecasting, and describes their design and their benchmark results.

Timestral comes in four sizes: five million parameters, twenty-eight million, one hundred and twenty-three million, and two hundred and nine million. The results rest on three ingredients.

The first is Decoupled Horizon Forecasting, or DHF. It separates the prediction tokens from the context, which is encoded causally. That way longer horizons can be trained without masking any context, and every observed value does double duty: it is a forecast target, and it is context for the forecasts that come after it. Nothing has to be hidden from the model in order to teach it to look further ahead. The second is a covariate framework that natively supports covariates known in advance about the future, inside a causal architecture. The third is a two-stage pipeline. Pretraining uses only a diverse, large-scale synthetic corpus. Post-training then mixes real and synthetic data, with an extended context.

Timestral sets a new state of the art on three benchmarks: GIFT-Eval, TIME, and fev-bench. On every metric, the two largest models, Timestral-209M and Timestral-123M, take the top two places. That is a strong form of the claim. It is not one model edging ahead on one leaderboard, but the two largest members of the family holding first and second across all three benchmarks, whatever metric is used to rank them.

Weights for all four models are available on Hugging Face. The three ingredients each address a different part of the job: how horizons are trained, how outside information known in advance enters a causal model, and what data the model learns from, synthetic first and then real.

Timestral. By Adrien Petralia and three hundred and fifty-five others, from Mistral AI. Published on the twenty-seventh of October. This is the long version: the whole paper rather than the abstract.

A time series foundation model forecasts series it has never seen, without retraining, so it must be trained on forecasts of many lengths. The causal models that lead the field have a neat trick for this, and the trick has a hidden cost. To teach a model to forecast three patches ahead, you hide the two patches before the target. But those hidden patches then vanish from the context of every later forecast in the same training sequence. Training longer horizons quietly removes context. The authors call this coupling the horizon distribution to context masking, and the Timestral family of four models, from five million to two hundred and nine million parameters, is built to break it.

Two ideas set the stage. First, like most modern forecasters, Timestral cuts a series into non-overlapping patches, here of thirty-two time steps each, and turns each patch into one token. The second is the difference between bidirectional and causal models. A bidirectional encoder, such as Chronos-2, must re-encode the context for each forecast starting point. A causal model lets each position see only the past, so one forward pass yields a forecast from every position at once. The catch is that each of those forecasts is only one patch ahead. Contiguous Patch Masking, used in TiRex, Toto-2.0 and TimesFM-3, extends the reach: a run of masked patches after a visible one lets the model forecast several patches ahead. The masked patches stay in the sequence as placeholder tokens, and that is where the context is lost.

Decoupled Horizon Forecasting gives each patch two tokens. One is a context token holding the observed values, processed causally. The other is a prediction token: a copy of a single learned embedding, placed at the same temporal position, where that patch is forecast. Both run through the same Transformer with shared weights, but the attention mask keeps them apart. Context tokens never attend to prediction tokens. A prediction token attends to all context before its forecast block, and to the earlier prediction tokens in its own block. So the context is computed once and shared, and the forecasts never disturb it. During training, each sequence is tiled into consecutive forecast blocks with lengths drawn from a distribution. A block of length k trains one forecast at every distance from one to k, so that distribution sets how often each distance is practised, with no side effect on context. Missing data can then be simulated separately, as gaps of variable length at arbitrary time steps, and those hidden values are still scored. Missing values are flagged in their context tokens, so a gap no longer looks like a forecast position. The price is roughly double the tokens in training. At inference one block follows the context, so up to thirty-two patches, one thousand and twenty-four time steps, come out of a single forward pass.

Covariates are handled by roles. Each variate is a target, a past-only covariate, or a known-future covariate, and roles enter only through attention masks and normalisation, never as embeddings. Known-future covariates see the whole window, including the horizon. To stop leakage, they attend only to each other: otherwise one could read a target at a later patch and carry it back to an earlier one. Past-only covariates also do not read targets, a choice that treats them as exogenous. A quantile head then outputs twenty-one quantiles per time step, building them outward from the median by adding positive increments, so quantiles cannot cross. Pretraining uses synthetic data only, with a context of eight thousand one hundred and ninety-two steps, followed by shorter post-training on real and synthetic data with a context of thirty-two thousand seven hundred and sixty-eight steps.

On GIFT-Eval, TIME and fev-bench, the largest model ranks first on all six reported metrics, from a single checkpoint, and the hundred and twenty-three million model ranks second, ahead of every outside model, including models with billions of parameters. All four sizes sit on the accuracy-against-size Pareto front. The striking part is size: the twenty-eight million model beats Chronos-2 on all six metrics, and also the largest Toto-2.0 model, about ninety times its size, and on fev-bench the five million model beats a Toto model with about four hundred and eighty times its parameters. The margins at the top are small, though. On GIFT-Eval, the lead over t0-gamma is under one percent in scaled error. On IND-Eval, an industrial benchmark the authors ran themselves, the gap over t0-gamma widens to about seven percent. Inference needs no GPU.

The paper has no limitations section; it presents itself as a report on design and benchmark results. There is no ablation of any of the three ingredients. The synthetic generators, which the authors call a significant part of their effort and frame as defining the model's prior, are described only in broad terms. Scores are single aggregate numbers with no error bars or repeated runs, and the IND-Eval comparisons come from the authors' own evaluation runs of competitors. The costs are stated: about double the tokens in training, latency growing roughly in proportion to context beyond two thousand and forty-eight steps, and covariates costing more than extra targets.

The idea to carry away is that a masked token was doing two jobs, and should do only one. By giving each patch a context token and a separate prediction token, Timestral lets every observation be both a target and a piece of history, and lets the horizon mix be chosen freely. The authors note that accuracy still improves from the second-largest to the largest model, which is their argument for training bigger ones.