VkD (T:RegNety 160 S:DeiT-Ti)

Reported on 1 benchmark across 1 task · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing1 result

Knowledge DistillationonImageNet
Top-1 accuracy %· 2024-03-10
79.2
best: 86.43 (ScaleKD (T:BEiT-L S:ViT-B/14))
$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections arXiv:2403.06213