KD++(T: regnety-16GF S:ViT-B)

Reported on 1 benchmark across 1 task · 1 paper · 1 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing1 result

Knowledge DistillationonImageNet
Top-1 accuracy %· 2023-05-26
83.6
best: 86.43 (ScaleKD (T:BEiT-L S:ViT-B/14))
SOTA
Improving Knowledge Distillation via Regularizing Feature Norm and Direction arXiv:2305.17007