Natural Language Inference on CommitmentBank

Metric: Accuracy (higher is better)

LeaderboardDataset

Loading chart...

Results

Sort:

#	Model↕	Accuracy▼	Extra Data	Paper	Date↕	Code
1	PaLM 540B (finetuned)	100	No	PaLM: Scaling Language Modeling with Pathways	2022-04-05	Code
2	Vega v2 6B (KD-based prompt transfer)	99.2	No	Toward Efficient Language Model Pretraining and ...	2022-12-04	-
3	ST-MoE-L 4.1B (fine-tuned)	98.2	No	ST-MoE: Designing Stable and Transferable Sparse...	2022-02-17	Code
4	ST-MoE-32B 269B (fine-tuned)	98	No	ST-MoE: Designing Stable and Transferable Sparse...	2022-02-17	Code
5	Turing NLR v5 XXL 5.4B (fine-tuned)	97.6	No	Toward Efficient Language Model Pretraining and ...	2022-12-04	-
6	DeBERTa-1.5B	97.2	No	DeBERTa: Decoding-enhanced BERT with Disentangle...	2020-06-05	Code
7	T5-XXL 11B (fine-tuned)	96.8	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
8	T5-Large 770M (fine-tuned)	94.4	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
9	T5-Base 220M (fine-tuned)	94	No	Exploring the Limits of Transfer Learning with a...	2019-10-23	Code
10	PaLM 2-L (one-shot)	87.5	No	PaLM 2 Technical Report	2023-05-17	Code
11	PaLM 2-S (one-shot)	82.1	No	PaLM 2 Technical Report	2023-05-17	Code
12	PaLM 2-M (one-shot)	80.4	No	PaLM 2 Technical Report	2023-05-17	Code
13	GPT-3 175B (Few-Shot)	75.6	No	Language Models are Few-Shot Learners	2020-05-28	Code
14	N-Grammer 343M	67.9	No	N-Grammer: Augmenting Transformers with latent n...	2022-07-13	Code
15	AlexaTM 20B	67.9	No	AlexaTM 20B: Few-Shot Learning Using a Large-Sca...	2022-08-02	Code
16	Bloomberg GPT (one-shot)	53.57	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
17	GPT-NeoX (one-shot)	48.21	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
18	BLOOM 176B (one-shot)	48.21	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code
19	OPT 66B (one-shot)	44.64	No	BloombergGPT: A Large Language Model for Finance	2023-03-30	Code

#1PaLM 540B (finetuned)SOTA
100
Accuracy· 2022-04-05
PaLM: Scaling Language Modeling with Pathways Code
#2Vega v2 6B (KD-based prompt transfer)
99.2
Accuracy· 2022-12-04
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE
#3ST-MoE-L 4.1B (fine-tuned)SOTA
98.2
Accuracy· 2022-02-17
ST-MoE: Designing Stable and Transferable Sparse Expert Models Code
#4ST-MoE-32B 269B (fine-tuned)
98
Accuracy· 2022-02-17
ST-MoE: Designing Stable and Transferable Sparse Expert Models Code
#5Turing NLR v5 XXL 5.4B (fine-tuned)
97.6
Accuracy· 2022-12-04
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE
#6DeBERTa-1.5BSOTA
97.2
Accuracy· 2020-06-05
DeBERTa: Decoding-enhanced BERT with Disentangled Attention Code
#7T5-XXL 11B (fine-tuned)SOTA
96.8
Accuracy· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#8T5-Large 770M (fine-tuned)
94.4
Accuracy· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#9T5-Base 220M (fine-tuned)
94
Accuracy· 2019-10-23
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Code
#10PaLM 2-L (one-shot)
87.5
Accuracy· 2023-05-17
PaLM 2 Technical Report Code
#11PaLM 2-S (one-shot)
82.1
Accuracy· 2023-05-17
PaLM 2 Technical Report Code
#12PaLM 2-M (one-shot)
80.4
Accuracy· 2023-05-17
PaLM 2 Technical Report Code
#13GPT-3 175B (Few-Shot)
75.6
Accuracy· 2020-05-28
Language Models are Few-Shot Learners Code
#14N-Grammer 343M
67.9
Accuracy· 2022-07-13
N-Grammer: Augmenting Transformers with latent n-grams Code
#15AlexaTM 20B
67.9
Accuracy· 2022-08-02
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model Code
#16Bloomberg GPT (one-shot)
53.57
Accuracy· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#17GPT-NeoX (one-shot)
48.21
Accuracy· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#18BLOOM 176B (one-shot)
48.21
Accuracy· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code
#19OPT 66B (one-shot)
44.64
Accuracy· 2023-03-30
BloombergGPT: A Large Language Model for Finance Code