Description
Fastformer is an type of Transformer which uses additive attention as a building block. Instead of modeling the pair-wise interactions between tokens, additive attention is used to model global contexts, and then each token representation is further transformed based on its interaction with global context representations.
Papers Using This Method
Multi-Granularity Vision Fastformer with Fusion Mechanism for Skin Lesion Segmentation2025-04-04An Analysis of Linear Complexity Attention Substitutes with BEST-RQ2024-09-04FUM: Fine-grained and Fast User Modeling for News Recommendation2022-04-10Fastformer: Additive Attention Can Be All You Need2021-08-20