KD++(T: ResNet-34 S:ResNet-18)

Reported on 1 benchmark across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing1 result

Knowledge DistillationonImageNet
Top-1 accuracy %· 2023-05-26
72.07
best: 86.43 (ScaleKD (T:BEiT-L S:ViT-B/14))
Improving Knowledge Distillation via Regularizing Feature Norm and Direction arXiv:2305.17007