Apple proposes calibrated sparse attention to speed up text-to-video generation

The method identifies that most token-to-token connections are redundant and uses a calibration step to learn which to attend to, speeding up generation in diffusion models while maintaining quality. The paper details how sparse attention is learned and applied in a transformer backbone.