Paper proposes contextual calibration before multimodal fusion

Arxiv CS.SD (Sound/Audio ML)

The paper introduces a method to calibrate multimodal signals before fusion, determining when a modality might be distracting rather than helpful. Experiments show improved performance on audio-visual and language-sound tasks.

multimodal

Arxiv CS.SD (Sound/Audio ML)·Jiyuan LiuLiangwei Nathan ZhengWei Emma ZhangXinpei WangWeitong Chen

8 days ago