Image Classification on ImageNet-1K (with DeiT-T)

Metric: GFLOPs (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	GFLOPs▼	Extra Data	Paper	Date↕	Code
1	Base (DeiT-T)	1.2	No	Training data-efficient image transformers & dis...	2020-12-23	Code
2	MCTF ($r=8$)	1	No	Multi-criteria Token Fusion with One-step-ahead ...	2024-03-15	Code
3	SPViT (1.0G)	1	No	SPViT: Enabling Faster Vision Transformers via S...	2021-12-27	Code
4	LTMP (80%)	1	No	Learned Thresholds Token Merging and Pruning for...	2023-07-20	Code
5	SPViT	1	No	Pruning Self-attentions into Convolutional Layer...	2021-11-23	Code
6	SPViT (0.9G)	0.9	No	SPViT: Enabling Faster Vision Transformers via S...	2021-12-27	Code
7	ToMe ($r=8$)	0.9	No	Token Merging: Your ViT But Faster	2022-10-17	Code
8	S$^2$ViTE	0.9	No	Chasing Sparsity in Vision Transformers: An End-...	2021-06-08	Code
9	dTPS	0.8	No	Joint Token Pruning and Squeezing Towards More A...	2023-04-21	Code
10	BAT	0.8	No	Beyond Attentive Tokens: Incorporating Token Imp...	2022-11-21	Code
11	eTPS	0.8	No	Joint Token Pruning and Squeezing Towards More A...	2023-04-21	Code
12	PPT	0.8	No	PPT: Token Pruning and Pooling for Efficient Vis...	2023-10-03	Code
13	EvoViT	0.8	No	Evo-ViT: Slow-Fast Token Evolution for Dynamic V...	2021-08-03	Code
14	LTMP (60%)	0.8	No	Learned Thresholds Token Merging and Pruning for...	2023-07-20	Code
15	ToMe ($r=12$)	0.8	No	Token Merging: Your ViT But Faster	2022-10-17	Code
16	MCTF ($r=16$)	0.7	No	Multi-criteria Token Fusion with One-step-ahead ...	2024-03-15	Code
17	PS-ViT	0.7	No	Patch Slimming for Efficient Vision Transformers	2021-06-05	-
18	LTMP (45%)	0.7	No	Learned Thresholds Token Merging and Pruning for...	2023-07-20	Code
19	DPS-ViT	0.6	No	Patch Slimming for Efficient Vision Transformers	2021-06-05	-
20	MCTF ($r=20$)	0.6	No	Multi-criteria Token Fusion with One-step-ahead ...	2024-03-15	Code
21	ToMe ($r=16$)	0.6	No	Token Merging: Your ViT But Faster	2022-10-17	Code
22	HVT-Ti-1	0.6	No	Scalable Vision Transformers with Hierarchical P...	2021-03-19	Code

#1Base (DeiT-T)SOTA
1.2
GFLOPs· 2020-12-23
Training data-efficient image transformers & distillation through attention Code
#2MCTF ($r=8$)
1
GFLOPs· 2024-03-15
Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers Code
#3SPViT (1.0G)
1
GFLOPs· 2021-12-27
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Code
#4LTMP (80%)
1
GFLOPs· 2023-07-20
Learned Thresholds Token Merging and Pruning for Vision Transformers Code
#5SPViT
1
GFLOPs· 2021-11-23
Pruning Self-attentions into Convolutional Layers in Single Path Code
#6SPViT (0.9G)
0.9
GFLOPs· 2021-12-27
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Code
#7ToMe ($r=8$)
0.9
GFLOPs· 2022-10-17
Token Merging: Your ViT But Faster Code
#8S$^2$ViTE
0.9
GFLOPs· 2021-06-08
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Code
#9dTPS
0.8
GFLOPs· 2023-04-21
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers Code
#10BAT
0.8
GFLOPs· 2022-11-21
Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers Code
#11eTPS
0.8
GFLOPs· 2023-04-21
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers Code
#12PPT
0.8
GFLOPs· 2023-10-03
PPT: Token Pruning and Pooling for Efficient Vision Transformers Code
#13EvoViT
0.8
GFLOPs· 2021-08-03
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer Code
#14LTMP (60%)
0.8
GFLOPs· 2023-07-20
Learned Thresholds Token Merging and Pruning for Vision Transformers Code
#15ToMe ($r=12$)
0.8
GFLOPs· 2022-10-17
Token Merging: Your ViT But Faster Code
#16MCTF ($r=16$)
0.7
GFLOPs· 2024-03-15
Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers Code
#17PS-ViT
0.7
GFLOPs· 2021-06-05
Patch Slimming for Efficient Vision Transformers
#18LTMP (45%)
0.7
GFLOPs· 2023-07-20
Learned Thresholds Token Merging and Pruning for Vision Transformers Code
#19DPS-ViT
0.6
GFLOPs· 2021-06-05
Patch Slimming for Efficient Vision Transformers
#20MCTF ($r=20$)
0.6
GFLOPs· 2024-03-15
Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers Code
#21ToMe ($r=16$)
0.6
GFLOPs· 2022-10-17
Token Merging: Your ViT But Faster Code
#22HVT-Ti-1
0.6
GFLOPs· 2021-03-19
Scalable Vision Transformers with Hierarchical Pooling Code