AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

AV-FABLE. Audio-Visual Manipulation Forensics in Long-Form Videos. By Yuesheng Huang and ten others, from Tsinghua University and Peking University. Published on the twenty-fourth of October. No code repository is linked to it.

A localised AI manipulation in a long video often touches only a brief event, with authentic content all around it. Calling the whole video real or fake does not help a forensic analyst much. What is needed is when the edit happened. Yet existing audio-visual deepfake benchmarks mostly use short talking-head clips, and the work that does address long-form temporal forensics is still predominantly visual.

AV-FABLE is built to measure sparse manipulations in open-scene, long-form audio-visual video. The task is temporal localisation, and the key choice is that audio and visual each get their own timeline. That dual-timeline design allows modality-specific localisation, and it exposes errors that a single joint timeline would hide. If a sound edit and a picture edit are scored on one shared track, a detector can look right while confusing which stream was touched. Separate tracks make that confusion visible. The scenes are open rather than staged talking heads, and the manipulations are sparse, so most of every video is genuine and the edited moments have to be found within it.

The benchmark contains sixteen thousand six hundred and fifty twin pairs, each a pristine video and its manipulated counterpart. Together they come to nine hundred and ninety-eight hours. The manipulations were created with twenty-seven editing configurations. Pairing every manipulated video with its untouched original means each edit has an exact reference to be measured against.

The first finding concerns audio-visual large language models used zero-shot. Prompted directly, they detect manipulations at near-chance levels and localise them poorly. The second finding cuts the other way. When supervised readouts are trained on the frozen perception features of the same models, strong forensic signals come back. The authors read this as evidence that manipulation cues are encoded in these representations, but are not effectively exposed through the language outputs.

That observation motivates NFSR, Native Forensics via Separated Readouts. It reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localisation in-domain, and across all three out-of-distribution settings, one of which transfers to held-out videos from an external long-video forensic benchmark.

The benchmark asks a sharper question than real or fake, and the answer it returns is a split one. The models already perceive the manipulation; the language interface is where the evidence gets lost. AV-FABLE, the code and the model weights are to be released publicly.