VMonarch：基于结构化注意力的高效视频扩散Transformer

The quadratic complexity inherent in the attention mechanism poses a significant limitation on the context scalability of Video Diffusion Transformers (DiTs). An observation reveals that the highly sparse spatio-temporal attention patterns prevalent in Video DiTs can be effectively represented by the Monarch matrix. This matrix family constitutes a class of structured matrices characterized by flexible sparsity, enabling sub-quadratic attention computation through an alternating minimization algorithm. Building upon this insight, VMonarch is proposed, a novel architecture designed for video diffusion models. VMonarch leverages the properties of Monarch matrices to address the efficiency bottlenecks of existing Video DiTs' attention mechanisms. By decomposing the attention operation into a series of smaller, more efficient matrix multiplications, VMonarch substantially reduces computational costs while maintaining or even enhancing model performance. Specifically, the structured nature of Monarch matrices facilitates sparse attention weighting without compromising critical information, thereby efficiently handling long video sequences. VMonarch's design allows the model to better capture both local and global dependencies within video data, circumventing the immense computational overhead typically associated with traditional self-attention mechanisms when applied to high-resolution or long-duration videos. This approach not only improves training and inference efficiency but also lays the groundwork for developing larger-scale, higher-quality video generation models in the future. Experimental results demonstrate VMonarch's superior efficiency and performance across various video generation tasks.