SparseGPT (175B, 4:8 Sparsity)

Reported on 5 benchmarks across 3 tasks · 1 paper · 1 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Question AnsweringonStoryCloze
Accuracy· 2023-01-02
77.02
best: 96.3 (BLOOMZ)
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Common Sense ReasoningonARC (Challenge)
Accuracy· 2023-01-02
39.85
best: 96.4 (GPT-4 (few-shot, k=25))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Common Sense ReasoningonARC (Easy)
Accuracy· 2023-01-02
68.35
best: 95.2 (ST-MoE-32B 269B (fine-tuned))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774

Medical2 results

Language ModellingonWikiText-2
Test perplexity· uses extra data· 2023-01-02
8.45
best: 8.21 (SparseGPT (175B, 50% Sparsity))
SOTA
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Language ModellingonLAMBADA
Accuracy· 2023-01-02
78.77
best: 89.7 (PaLM-540B (Few-Shot))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774