SparseGPT (175B, 2:4 Sparsity)

Reported on 4 benchmarks across 3 tasks · 1 paper · 1 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical2 results

Language ModellingonWikiText-2
Test perplexity· uses extra data· 2023-01-02
8.73
best: 8.21 (SparseGPT (175B, 50% Sparsity))
SOTA
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Language ModellingonLAMBADA
Accuracy· 2023-01-02
79.47
best: 89.7 (PaLM-540B (Few-Shot))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774

Natural Language Processing2 results

Question AnsweringonStoryCloze
Accuracy· 2023-01-02
76.19
best: 96.3 (BLOOMZ)
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Common Sense ReasoningonARC (Challenge)
Accuracy· 2023-01-02
38.99
best: 96.4 (GPT-4 (few-shot, k=25))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774