OPT-175B (50% Sparsity)

Reported on 5 benchmarks across 3 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing3 results

Question AnsweringonPIQA
Accuracy· 2023-01-02
54.73
best: 90.1 (Unicorn 11B (fine-tuned))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Question AnsweringonStoryCloze
Accuracy· 2023-01-02
47.1
best: 96.3 (BLOOMZ)
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Common Sense ReasoningonARC (Challenge)
Accuracy· 2023-01-02
25.6
best: 96.4 (GPT-4 (few-shot, k=25))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774

Medical2 results

Language ModellingonLAMBADA
Accuracy· 2023-01-02
0.02
best: 89.7 (PaLM-540B (Few-Shot))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774
Language ModellingonWikiText-2
Test perplexity· uses extra data· 2023-01-02
234.77
best: 8.21 (SparseGPT (175B, 50% Sparsity))
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot arXiv:2301.00774