Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

STD

Spatial-Channel Token Distillation

GeneralIntroduced 200032 papers

Description

The Spatial-Channel Token Distillation method is proposed to improve the spatial and channel mixing from a novel knowledge distillation (KD) perspective. To be specific, we design a special KD mechanism for MLP-like Vision Models called Spatial-channel Token Distillation (STD), which improves the information mixing in both the spatial and channel dimensions of MLP blocks. Instead of modifying the mixing operations themselves, STD adds spatial and channel tokens to image patches. After forward propagation, the tokens are concatenated for distillation with the teachers’ responses as targets. Each token works as an aggregator of its dimension. The objective of them is to encourage each mixing operation to extract maximal task-related information from their specific dimension.

Papers Using This Method

Geo-Registration of Terrestrial LiDAR Point Clouds with Satellite Images without GNSS2025-07-08 Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs2025-05-25 RE-TRIP : Reflectivity Instance Augmented Triangle Descriptor for 3D Place Recognition2025-05-22 A New Paradigm in IBR Modeling for Power Flow and Short Circuit Analysis2025-04-14 DualQuat-LOAM: LiDAR Odometry and Mapping parameterized on Dual Quaternions2025-04-07 GAA-TSO: Geometry-Aware Assisted Depth Completion for Transparent and Specular Objects2025-03-21 Multimodal AI on Wound Images and Clinical Notes for Home Patient Referral2025-01-22 MHGNet: Multi-Heterogeneous Graph Neural Network for Traffic Prediction2025-01-07 Leveraging Prompt Learning and Pause Encoding for Alzheimer's Disease Detection2024-12-09 BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection2024-11-21 Spotlight Text Detector: Spotlight on Candidate Regions Like a Camera2024-09-25 Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition2024-08-19 IFTD: Image Feature Triangle Descriptor for Loop Detection in Driving Scenes2024-06-12 Excess Delay from GDP: Measurement and Causal Analysis2024-05-18 Adapting an Artificial Intelligence Sexually Transmitted Diseases Symptom Checker Tool for Mpox Detection: The HeHealth Experience2024-04-23 On Extending the Automatic Test Markup Language (ATML) for Machine Learning2024-04-04 Disturbance Ratio for Optimal Multi-Event Classification in Power Distribution Networks2024-02-18 Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN2024-02-03 Bootstrap Your Own Variance2023-12-06 Uncertainty quantification for deep learning-based schemes for solving high-dimensional backward stochastic differential equations2023-10-05