Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

ETC

Extended Transformer Construction

Natural Language ProcessingIntroduced 200036 papers

Description

Extended Transformer Construction, or ETC, is an extension of the Transformer architecture with a new attention mechanism that extends the original in two main ways: (1) it allows scaling up the input length from 512 to several thousands; and (2) it can ingesting structured inputs instead of just linear sequences. The key ideas that enable ETC to achieve these are a new global-local attention mechanism, coupled with relative position encodings. ETC also allows lifting weights from existing BERT models, saving computational resources while training.

Papers Using This Method

Two-Player Zero-Sum Games with Bandit Feedback2025-06-17 Koopman-Based Event-Triggered Control from Data2025-04-19 Performance-Barrier Event-Triggered PDE Control of Traffic Flow2025-01-01 Automated Toll Management System Using RFID and Image Processing2024-12-02 Hierarchical Event-Triggered Systems: Safe Learning of Quasi-Optimal Deadline Policies2024-09-15 Performance-Barrier Event-Triggered Control of a Class of Reaction-Diffusion PDEs2024-07-11 Contextual Dynamic Pricing: Algorithms, Optimality, and Local Differential Privacy Constraints2024-06-04 Event-Triggered Robust Cooperative Output Regulation for a Class of Linear Multi-Agent Systems with an Unknown Exosystem2024-03-01 Replication-proof Bandit Mechanism Design with Bayesian Agents2023-12-28 Learning-based Scheduling for Information Accuracy and Freshness in Wireless Networks2023-10-24 Listen to Minority: Encrypted Traffic Classification for Class Imbalance with Contrastive Pre-Training2023-08-31 High-dimensional Contextual Bandit Problem without Sparsity2023-06-19 Permutation Decision Trees2023-06-05 An Improved Heart Disease Prediction Using Stacked Ensemble Method2023-04-12 Asynchronous Event-Triggered Control for Non-Linear Systems2022-11-25 LittleBird: Efficient Faster & Longer Transformer for Question Answering2022-10-21 Short Text Pre-training with Extended Token Classification for E-commerce Query Understanding2022-10-08 Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency2022-05-26 Thompson Sampling for Bandit Learning in Matching Markets2022-04-26 ETCetera: beyond Event-Triggered Control2022-03-03