ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search

Jiyan He, Guang Liang, Hao Liu, Haoxiang Guan, Jinbo Sun, Junyi Guo, Wenjun Feng, Yantai Xie and 14 others

Zhongguancun Academy · Zhongguancun Institute of Artificial Intelligence

Published September 11, 2026 · on the feed September 15, 2026 · ▲ 257 · arXiv 2609.13356

AgentsLLM training & RLReasoning & math Full code arXiv ↗Hugging Face ↗Weights ↗Dataset ↗

The abstract

In this work, we present ZGCM-1, a fully open 7B dense foundation model trained from scratch with extreme data, system, and algorithmic efficiency. ZGCM-1 is founded on a core premise: compact models cannot passively memorize the open web, but can overcome parametric capacity limits by coupling deliberate internal thinking with active external tool use. To support this paradigm across a 256K context, we develop an end-to-end, high-efficiency open training recipe: Architecture & System Co-design: interleaved gated sliding-window and full attention, and a stable FP8 Muon optimizer; Progressive Curriculum & MDP Mid-Training: context scaling across 16K, 64K, and 256K, and the reformulation of interaction traces into Markov Decision Processes. Furthermore, we establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations, data curation, and rapid diagnostic evaluation. Extensive evaluations show that ZGCM-1-7B is competitive across 7B model family on general benchmarks. On several challenging mathematical reasoning and agentic search suites, it remains competitive with frontier models orders of magnitude larger, such as Qwen3-235B-A22B and GLM-5.1. We also show that our pre-training design offers a ~4.2x efficiency improvement in 16K pre-training time-to-loss. Across the full development lifecycle, we distill eight actionable empirical findings-spanning architectural scaling, SFT quality pruning, long-context generalization, and agentic co-training dynamics. To facilitate community research, we open-source model weights from the pre-training, mid-training, and post-training stages, intermediate checkpoints, training code, per-stage data and data recipes, and W&B logs.

The Brief

3:01

A short podcast summary of the paper. Written and voiced by AI.

Transcript

ZGCM-1. A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search. By Jiyan He and twenty-one others, from Zhongguancun Academy and the Zhongguancun Institute of Artificial Intelligence. Published on the eleventh of September. Its code repository is worked out: about three thousand five hundred source files, with a readme and a licence.

ZGCM-1 is a fully open dense foundation model with seven billion parameters, trained from scratch and aimed at math and agentic search. Fully open here means a long list of released artifacts. There are model weights from the pre- training stage, the mid-training stage and the post-training stage. There are intermediate checkpoints, the training code, the data for each stage together with its data recipe, and the W&B logs from training.

The model supports a context of two hundred fifty-six thousand tokens. It is built on one premise: a compact model cannot passively memorize the open web, but it can overcome the limits of its parametric capacity by coupling deliberate internal thinking with active external tool use. Rather than storing everything, it reasons and it looks things up.

On evaluation, ZGCM-1-7B is competitive within the family of seven billion parameter models on general benchmarks. On several challenging mathematical reasoning and agentic search suites, it remains competitive with frontier models orders of magnitude larger, with Qwen3-235B-A22B and GLM-5.1 named as examples.

The training recipe is built for efficiency in data, in systems and in algorithms, end to end. On architecture and systems, which were co-designed, it interleaves gated sliding-window attention with full attention, and it uses a stable FP8 Muon optimizer. Mid-training follows a progressive curriculum. Context is scaled in steps: sixteen thousand tokens, then sixty-four thousand, then two hundred fifty-six thousand. And interaction traces are reformulated as Markov Decision Processes.

The development itself ran on what the authors call an AI-native research and development workflow, in which agent swarms autonomously manage cluster operations, data curation, and rapid diagnostic evaluation. The pre-training design is reported to give about a four point two times efficiency improvement in pre-training time-to-loss at a sixteen thousand token context. Across the full development lifecycle, the authors distill eight actionable empirical findings, spanning architectural scaling, SFT quality pruning, long-context generalization, and the dynamics of agentic co-training.

The stated purpose of the release is community research. Because weights come from every stage, along with intermediate checkpoints, per-stage data and logs, the release serves someone studying how a compact model is trained for long context and tool use, not only someone who wants the finished model.

ZGCM-1 rests on a bet that a small model should think and reach for tools rather than try to hold the web in its weights. The release makes that bet inspectable from the first pre-training stage to the last post-training checkpoint.

Paper Quest · 7 chapters · about 40 min

Play this paper

Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.

Chapter 1 🧩 · The problem

Small model, big ambitions. The wall here is memory.

You have the compute for a small model, about seven billion parameters, and you want it to compete on hard maths and on searching the web for answers.

The road ahead

  1. 1 · The problem ~6 min With compute for only a small model of about seven billion parameters, you must find an edge on hard maths and web questions that size alone cannot give.
  2. 2 · What came before ~3 min
  3. 3 · The key idea ~9 min A small model stops trying to memorise the web; it thinks things through at length and looks up what it lacks with tools.
  4. 4 · Choose your own research ~3 min
  5. 5 · Build the method ~8 min Pre-train a 7B model from scratch, mid-train it on longer contexts and step-by-step agent decisions, then post-train and release every stage.
  6. 6 · The experiment ~6 min You’ll guess: How large an efficiency improvement in 16K pre-training time-to-loss did ZGCM-1's pre-training design give?
  7. 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Join free to play the whole quest ›

Free, with Google or an email address. Your stars and progress are kept.

KipQuest: the day’s AI research, as podcasts and quests. All papers Android app Privacy Contact Episodes are written and voiced by AI from each paper. Summaries can contain mistakes: the paper is the source.