TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Natural Language Processing/Natural Language Inference

Natural Language Inference

65 benchmarks1961 papers

Natural language inference (NLI) is the task of determining whether a "hypothesis" is true (entailment), false (contradiction), or undetermined (neutral) given a "premise".

Example:

| Premise | Label | Hypothesis | | --- | ---| --- | | A man inspects the uniform of a figure in some East Asian country. | contradiction | The man is sleeping. | | An older and younger man smiling. | neutral | Two men are smiling and laughing at the cats playing on the floor. | | A soccer game with multiple males playing. | entailment | Some men are playing a sport. |

Approaches used for NLI include earlier symbolic and statistical approaches to more recent deep learning approaches. Benchmark datasets used for NLI include SNLI, MultiNLI, SciTail, among others. You can get hands-on practice on the SNLI task by following this d2l.ai chapter.

Further readings:

  • Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches

Benchmarks

Natural Language Inference on SNLI

% Test Accuracy% Train AccuracyDev AccuracyParameters% Dev AccuracyAccuracy

Natural Language Inference on MultiNLI

MatchedMismatchedAccuracyDev MatchedDev Mismatched

Natural Language Inference on ANLI test

A2A3A1

Natural Language Inference on WNLI

Accuracy

Natural Language Inference on LiDiRus

MCC

Natural Language Inference on RCB

Average F1Accuracy

Natural Language Inference on TERRa

Accuracy

Natural Language Inference on CommitmentBank

AccuracyF1

Natural Language Inference on FarsTail

% Test Accuracy

Natural Language Inference on MultiNLI Dev

MatchedMismatched

Natural Language Inference on RTE

Accuracy

Natural Language Inference on SNLI-VE val

Accuracy

Natural Language Inference on SNLI-VE test

Accuracy

Natural Language Inference on SciTail

AccuracyDev Accuracy% Dev Accuracy% Test Accuracy

Natural Language Inference on MedNLI

AccuracyParams (M)

Natural Language Inference on XNLI French

Accuracy

Natural Language Inference on XNLI

Accuracy

Natural Language Inference on QNLI

Accuracy

Natural Language Inference on V-SNLI

Accuracy

Natural Language Inference on WeiboPolls

ROUGE-1ROUGE-LBLEU-1BLEU-3

Natural Language Inference on XNLI Chinese

Accuracy

Natural Language Inference on XNLI Chinese Dev

Accuracy

Natural Language Inference on e-SNLI

AccuracyBLEU

Natural Language Inference on CICERO

ROUGE

Natural Language Inference on JamPatoisNLI

Accuracy

Natural Language Inference on e-SNLI-VE

Accuracy

Natural Language Inference on AX

Accuracy

Natural Language Inference on BioNLI

Macro F1

Natural Language Inference on HANS

1:1 Accuracy

Natural Language Inference on KUAKE-QQR

Accuracy

Natural Language Inference on KUAKE-QTR

Accuracy

Natural Language Inference on MED

1:1 Accuracy

Natural Language Inference on MNLI + SNLI + ANLI + FEVER

% Dev Accuracy% Test Accuracy

Natural Language Inference on MRPC

Acc

Natural Language Inference on Probability words NLI

1:1 Accuracy

Natural Language Inference on Quora Question Pairs

Accuracy

Natural Language Inference on SICK

1:1 Accuracy

Natural Language Inference on TabFact

Accuracy

Natural Language Inference on XWINO

Accuracy

Natural Language Inference on XNLI Zero-Shot English-to-French

Accuracy

Natural Language Inference on XNLI Zero-Shot English-to-German

Accuracy

Natural Language Inference on XNLI Zero-Shot English-to-Spanish

Accuracy