Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Methods/Content-based Attention

Content-based Attention

GeneralIntroduced 200033 papers

Description

Content-based attention is an attention mechanism based on cosine similarity:

$f_{att}\left(\textbf{h}_{i}, \textbf{s}\_{j}\right) = \cos\left[\textbf{h}\_{i};\textbf{s}\_{j}\right]$

It was utilised in Neural Turing Machines as part of the Addressing Mechanism.

We produce a normalized attention weighting by taking a softmax over these attention alignment scores.

Papers Using This Method

Intelligent DoS and DDoS Detection: A Hybrid GRU-NTM Approach to Network Security2025-04-10 Memory-augmented conformer for improved end-to-end long-form ASR2023-09-22 FashionNTM: Multi-turn Fashion Image Retrieval via Cascaded Memory2023-08-20 TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition2023-05-09 Token Turing Machines2022-11-16 Similarity and Content-based Phonetic Self Attention for Speech Recognition2022-03-19 HiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer2022-02-15 AttentionHTR: Handwritten Text Recognition Based on Attention Encoder-Decoder Networks2022-01-23 Hard-Attention for Scalable Image Classification2021-02-20 Robust High-dimensional Memory-augmented Neural Networks2020-10-05 Explainable Inference on Sequential Data via Memory-Tracking2020-07-11 Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR2020-02-14 Sequence-to-sequence Singing Synthesis Using the Feed-forward Transformer2019-10-22 Memory-Augmented Recurrent Networks for Dialogue Coherence2019-10-16 A Neural Turing~Machine for Conditional Transition Graph Modeling2019-07-15 Understanding Memory Modules on Learning Simple Algorithms2019-07-01 A review on Neural Turing Machine2019-04-10 Few-Shot Generalization Across Dialogue Tasks2018-11-28 Context-Aware Neural Model for Temporal Information Extraction2018-07-01 A Taxonomy for Neural Memory Networks2018-05-01