Deepfake benchmark audit reveals detectors fail in the wild

Arxiv CS.SD (Sound/Audio ML)

The paper audits deepfake benchmarks using frozen self-supervised representations, finding that detectors scoring well on benchmarks routinely fail in real-world settings. A concerning feedback loop emerges as benchmarks drive increasingly specialized detectors.

DeepfakesBenchmarks

See 6 more sources

What Do Deepfake Benchmarks Measure? An Audit Using Frozen Self-Supervised Representations2 days agoSamuel Pagon, Yixuan Shen, Vishal Asnani, Feng Liu

Trustworthy Image Authentication using Forensic Knowledge Graphs4 days agoTai D. Nguyen, Matthew C. Stamm

ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection4 days agoChenhao Dang, Dantong Zhu, Jun Yang, Conghui He, Weijia Li

REKEY: Metadata-Grounded Visual-Key Regeneration for Contamination-Resilient VQA Evaluation5 days agoTengjie Lin, Yutao Sun, Jingwei Ni, Shuhan Ge, Hao-Xuan Ma, Yanting Miao, Wangyue Lu, Mingshuai Chen, Tiancheng Zhao, Jianwei Yin

From Text Metrics to Model Internals: A Study of Whisper ASR Hallucination Detection5 days agoJan Jasi\'nski, Mateusz Bara\'nski, Julitta Bartolewska, Marcin Witkowski, Konrad Kowalczyk

Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)19 days agoFelix Akeret

Arxiv CS.SD (Sound/Audio ML)·Georgii AparinVadim PopovTasnima SadekovaAssel Yermekova

Jun 26, 4:00 AM