TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models
Deqing Fu, Huangyuan Su, Rajat Sen, Taman Narayan, Sujay Sanghavi, Abhimanyu Das, Weihao Kong
Google Research · Google DeepMind · University of Southern California · Harvard University · University of Texas at Austin
Published September 29, 2026 · on the feed September 30, 2026 · ▲ 11 · arXiv 2609.37989
The abstract
Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.
The Brief
2:11A short podcast summary of the paper. Written and voiced by AI.
Transcript
TabFM-Auto. Self-Evolving Pipelines for Tabular Foundation Models. By Deqing Fu and six others, from Google Research, Google DeepMind, and three other institutions. Published on the twenty-ninth of September. No code repository is linked to it.
Two approaches to tabular data each leave something on the table. Tabular foundation models reach strong zero-shot accuracy by pretraining on synthetic tables, but they ignore the column names, the task descriptions, and the auxiliary files that carry what a dataset actually means. Self-evolving machine learning engineering agents, MLE agents for short, take the other route, and train a model from scratch on each dataset. Searching jointly over features, architectures and hyperparameters makes them noisy, and prone to overfitting.
TabFM-Auto splits the work between the two. The model stays fixed: it is TabFM, a tabular foundation model. What evolves is the data pipeline around it, and a language model agent does the evolving. Rather than build a new model for every dataset, the agent works on what the fixed model is fed and on what happens to its output.
The agent is guided by the dataset's metadata and by validation feedback. It iteratively refines four things: data cleaning, feature engineering, context selection, and post-processing, each round aimed at reducing TabFM's error.
Across all fifty-one datasets of the TabArena benchmark, five TabFM-Auto configurations, using different agents and language models, take the top five overall positions. The best of them raises TabFM from one thousand seven hundred and eighty-five Elo to two thousand and thirteen. The pipelines it discovers also transfer to other frozen tabular foundation models with no further search, adding between sixty-nine and one hundred and forty-three Elo. And on the eight tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.
A division of labour runs through the design. The foundation model supplies a strong, fixed predictor; the agent supplies the reading of column names and task descriptions that the model was ignoring. And what the agent builds is portable: a pipeline found for TabFM lifts other frozen models without another search.
Paper Quest · 7 chapters · about 40 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A model that reads numbers but not meanings. Start there.
You have a pretrained tabular model that predicts on a new table in one forward pass. It is accurate, but it only sees numbers and category indices: a column named systolic_bp looks the same to it as an anonymous X_17.
The road ahead
- 1 · The problem ~7 min A pretrained table model predicts well but cannot read what columns mean, and agents that can read them bring noisy, overfitting training.
- 2 · What came before ~3 min
- 3 · The key idea ~8 min Keep the pretrained table model frozen as the predictor, and let a language model agent rewrite only the data pipeline around it.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min A language model agent repeatedly edits a pipeline of cleaning, features, context selection and post-processing around frozen TabFM, keeping what cross-validation rewards.
- 6 · The experiment ~6 min You’ll guess: On the TabArena benchmark, how many of the top five overall positions did TabFM-Auto configurations take?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.