Complex Transformer: A Framework for Modeling Complex-Valued Sequence
Muqiao Yang, Martin Q. Ma, Dongyu Li, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov
Abstract
While deep learning has received a surge of interest in a variety of fields in recent years, major deep learning models barely use complex numbers. However, speech, signal and audio data are naturally complex-valued after Fourier Transform, and studies have shown a potentially richer representation of complex nets. In this paper, we propose a Complex Transformer, which incorporates the transformer model as a backbone for sequence modeling; we also develop attention and encoder-decoder network operating for complex input. The model achieves state-of-the-art performance on the MusicNet dataset and an In-phase Quadrature (IQ) signal dataset.
Results
Related Papers
Automatic Classification and Segmentation of Tunnel Cracks Based on Deep Learning and Visual Explanations2025-07-18A Survey of Deep Learning for Geometry Problem Solving2025-07-16Uncertainty Quantification for Motor Imagery BCI -- Machine Learning vs. Deep Learning2025-07-10Chat-Ghosting: A Comparative Study of Methods for Auto-Completion in Dialog Systems2025-07-08Deep Learning Optimization of Two-State Pinching Antennas Systems2025-07-08AXLearn: Modular Large Model Training on Heterogeneous Infrastructure2025-07-07Determination Of Structural Cracks Using Deep Learning Frameworks2025-07-03Generalized Adaptive Transfer Network: Enhancing Transfer Learning in Reinforcement Learning Across Domains2025-07-02