Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
Team Kandinsky, Julia Agafonova, Bulat Akhmatov, Mikhail Aksyutin, Grigorii Alekseenko, Anastasia Aliaskina, Olga Androsova, Vladimir Arkhipkin and 80 others
Kandinsky Lab
Published October 4, 2026 · on the feed October 6, 2026 · ▲ 161 · arXiv 2610.05608
The abstract
We present Kandinsky 6.0 Video, a family of foundation diffusion models for synchronized text-to-audio-video generation, comprising Kandinsky 6.0 Video Lite (3B parameters) and Kandinsky 6.0 Video Pro (29B parameters). Both models generate 5-second video clips with synchronized 44 kHz audio, including lip-sync, in text-to-audio-video (T2AV) and image-to-audio-video (I2AV) modes; a built-in super-resolution model raises the output resolution to Full-HD (1920times1080). Building on the video generation capabilities of Kandinsky 5.0, Kandinsky 6.0 Video employs a dual-stream CrossDiT architecture that connects a pretrained video stream and a newly trained audio stream through bidirectional cross-attention for temporal and semantic alignment. Our continuous pretraining strategy first trains the audio stream from scratch on large-scale audio corpora and then trains both streams jointly on paired audio-video data while preserving unimodal fidelity; pretraining is followed by supervised fine-tuning, reinforcement-learning-based post-training, and distillation. In side-by-side human evaluation, Kandinsky 6.0 Video Pro clearly outperforms its predecessor, Kandinsky 5.0 Video Pro, and remains competitive with leading audio-video generation models, particularly in speech quality. To accelerate open research and deployment in multimedia generation, we release the code, model checkpoints, and diffusers integration under the MIT license.
The Brief
2:30A short podcast summary of the paper. Written and voiced by AI.
Transcript
Kandinsky 6.0 Video. Foundation Models for Synchronized Video and Audio Generation. By Team Kandinsky and eighty-seven others, from Kandinsky Lab. Published on the fourth of October. Its code repository is worked out: about one hundred and thirty source files, with a readme and a licence.
Kandinsky 6.0 Video sets itself a joint task: generate a video clip and its soundtrack together, from text or from an image, so that the two stay synchronized all the way down to lip-sync. Its predecessor, Kandinsky 5.0, already generated video. Adding sound means a second modality has to be learned and then tied to the first, in time and in meaning, without either one losing quality along the way. The abstract names exactly those pressures: temporal and semantic alignment on one side, preserving unimodal fidelity on the other.
The release is a family of foundation diffusion models in two sizes. Kandinsky 6.0 Video Lite has three billion parameters. Kandinsky 6.0 Video Pro has twenty-nine billion. Both produce five-second clips with synchronized audio at forty-four kilohertz, lip-sync included, in a text-to-audio-video mode and an image-to-audio-video mode. A built-in super-resolution model then lifts the output to Full-HD. Underneath is a dual-stream CrossDiT architecture: a pretrained video stream and a newly trained audio stream, side by side, connected through bidirectional cross-attention for temporal and semantic alignment.
Training is staged so that the new half learns before it has to cooperate. Continuous pretraining first trains the audio stream from scratch on large-scale audio corpora. Only then are both streams trained jointly, on paired audio-video data, with the stated aim of preserving each modality's own fidelity. Pretraining is followed by three further steps: supervised fine-tuning, post-training based on reinforcement learning, and finally distillation.
The evaluation is a side-by-side human comparison. In it, Kandinsky 6.0 Video Pro clearly outperforms its predecessor, Kandinsky 5.0 Video Pro. Against leading audio-video generation models it remains competitive, particularly in speech quality.
Code, model checkpoints and a diffusers integration are released under the MIT license, with the stated aim of accelerating open research and deployment in multimedia generation. Underneath the model sizes and resolutions sits a build order. A video model that already worked is kept, a sound model is grown beside it from nothing, and cross-attention running in both directions joins the two into one generator.
Paper Quest · 7 chapters · about 41 min
Play this paper
Learn it by doing: make the calls the authors made, guess the result before you see it, and defend it in a viva graded on your own words. Chapter 1 is right here.
Chapter 1 🧩 · The problem
A video model that is silent. The task is to give it a voice without breaking what it sees.
You have an open model that already makes good short video clips from text. Now you want it to make the soundtrack too: speech, background sound, lip-sync, all in step with the picture.
The road ahead
- 1 · The problem ~7 min An open video model makes good silent clips, and the task is to add matching sound without losing quality in either picture or sound.
- 2 · What came before ~3 min
- 3 · The key idea ~9 min Keep the trained video model as one stream, add a new audio stream trained alone first, link them with two-way cross-attention, then train both together.
- 4 · Choose your own research ~3 min
- 5 · Build the method ~8 min Training runs in stages: audio alone, then joint pretraining, then fine-tuning, then reinforcement learning, then distillation for fast sampling.
- 6 · The experiment ~6 min You’ll guess: In side-by-side human evaluation, how did Kandinsky 6.0 Video Pro compare with leading audio-video generation models?
- 7 · The viva ~5 min Three questions from the examiners, answered in your own words. Win it for the paper’s trophy.
Free, with Google or an email address. Your stars and progress are kept.