Description
Primer is a Transformer-based architecture that improves upon the Transformer architecture with two improvements found through neural architecture search: squared RELU activations in the feedforward block, and depthwise convolutions added to the attention multi-head projections: resulting in a new module called Multi-DConv-Head-Attention.
Papers Using This Method
SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment2025-05-20A review of DNA restriction-free overlapping sequence cloning techniques for synthetic biology2025-05-06Primer C-VAE: An interpretable deep learning primer design method to detect emerging virus variants2025-03-03Characteristic Performance Study on Solving Oscillator ODEs via Soft-constrained Physics-informed Neural Network with Small Data2024-08-19The curious case of A31P, a topology-switching mutant of the Repressor of Primer protein : A molecular dynamics study of its folding and misfolding2024-04-01The Effects of Political Martyrdom on Election Results: The Assassination of Abe2023-05-29Brainformers: Trading Simplicity for Efficiency2023-05-29Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks2023-01-23N-Grammer: Augmenting Transformers with latent n-grams2022-07-13Piecewise Linear Neural Networks and Deep Learning2022-06-18Enriching and Characterizing T-Cell Repertoires from 3' Barcoded Single-Cell Whole Transcriptome Amplification Products2022-03-21Searching for Efficient Transformers for Language Modeling2021-12-01N-grammer: Augmenting Transformers with latent n-grams2021-11-16Primer: Searching for Efficient Transformers for Language Modeling2021-09-17