SynthFaces: Balanced Large Scale Human Dataset

SynthFaces. Balanced Large Scale Human Dataset. By Sergey Karpukhin and three others, from FusionBrain Lab, HSE University, and one other institution. Published on the twenty-ninth of October.

SynthFaces is a highly balanced, identity-centric dataset of human faces, and every image and identity in it is synthetic. It exists because of a problem with the real thing. Real-world human datasets carry inherent biases and privacy risks: they represent identities unevenly and often contain sensitive personal information. Generative models trained on that skewed data inherit it, and perform unfairly, especially at identity-preserving generation.

What the dataset offers is balance by construction. Its identities are diverse in a way the abstract describes as label diversity not present in real human-centric datasets, and the authors further contextualize the collected images. Because the released images and identities are synthetic, the dataset reduces direct reliance on real personal photographs. The abstract does not give a count of images or identities; the title describes it as large scale. What it stresses instead is the property that matters for fairness: identities that are evenly represented, rather than skewed toward whoever happened to be photographed most.

It comes out of a pipeline for controllable generation of face datasets. The starting point is the Arc2Face diffusion model, which the authors expand with a flow matching generative model that operates in the manifold of identity embeddings. The claim is that this pairing achieves two goals at once: generative fidelity, and the diversity of identities that real data lacks. Privacy is not left to assumption. Identity separation is checked empirically, with a nearest-neighbor non-leakage metric.

It is aimed at anyone training or tuning models that should be fair across identities, for a range of downstream tasks. The authors demonstrate one such use themselves, finetuning an existing adapter-based method for personalized generation on SynthFaces. That choice of demonstration fits the motivation, since identity-preserving generation is the task where models trained on skewed real data were said to perform unfairly.

Here the fairness argument and the privacy argument point the same way. A generator steered through identity space can fill in the identities that real collections underrepresent, and because none of those identities is a real person, the balance comes without drawing on anyone's personal images.