Apple researchers tame text-to-sounding video generation with modality conditioning

The paper addresses two challenges: weak text conditioning and misalignment between audio and video modalities. It proposes a framework integrating cross-modal attention and joint conditioning to improve synchronization.