Radial Attention

Radial Attention is a scalable sparse attention mechanism that enhances video diffusion models by achieving O(n log n) computational complexity. It utilizes physics-inspired sparsity to enforce spatially local and temporally decaying attention, which mirrors energy dissipation in physical systems. This approach allows for the efficient generation of longer videos while maintaining high quality, making it suitable for applications in video generation and processing.