Tasks SotA Datasets Papers Methods Submit About

Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable Benchmarks All SotA Datasets Papers Methods

Community

Submit Results About

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/DistilBERT

DistilBERT

Reported on 35 benchmarks across 6 tasks · 7 papers · 6 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing18 results

Text ClassificationonCivil Comments
GMB BNSP· 2023-01-26
0.9644
SOTA
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonSearchsnippets
Accuracy· 2022-11-30
89.69
SOTA
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
Relation ExtractiononDataset: Relationship extraction for knowledge graph creation from biomedical literature (Gene-Disease relationships)
F1· 2022-01-05
91
SOTA
Comparison of biomedical relationship extraction methods and models for knowledge graph creation arXiv:2201.01647
Relation ExtractiononDataset: Relationship extraction for knowledge graph creation from biomedical literature (Gene-Disease relationships) n
F1· 2022-01-05
89
SOTA
Comparison of biomedical relationship extraction methods and models for knowledge graph creation arXiv:2201.01647
Text ClassificationonUK Key Stage Readability
F1· 2024-11-26
74.4
best: 99.6 (ELECTRA + ANN)
What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics arXiv:2411.17593
Text ClassificationonCivil Comments
AUROC· 2023-01-26
0.9804
best: 0.9818 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
GMB BPSN· 2023-01-26
0.874
best: 0.901 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
GMB Subgroup· 2023-01-26
0.8762
best: 0.8807 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
Macro F1· 2023-01-26
0.3879
best: 0.4749 (RoBERTa BCE)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
Micro F1· 2023-01-26
0.5115
best: 0.5958 (Unfreeze Glove ResNet 44)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
Precision· 2023-01-26
0.3572
best: 0.4835 (Unfreeze Glove ResNet 44)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonCivil Comments
Recall· 2023-01-26
0.9001
best: 0.9254 (XLNet)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
Text ClassificationonR8
Accuracy· 2022-11-30
97.981
best: 98.451 (DeBERTa)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
Text ClassificationonMR
Accuracy· 2022-11-30
85.31
best: 93.3 (VLAWE)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
Text ClassificationonTwitter
Accuracy· 2022-11-30
99.96
best: 99.97 (ERNIE 2.0)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
Question AnsweringonFairytaleQA
F1· 2022-03-26
0.082
best: 0.536 (BART fine-tuned on FairytaleQA)
Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension arXiv:2203.13947
Question AnsweringonFairytaleQA
Rouge-L· 2022-03-26
0.097
best: 0.533 (BART fine-tuned on FairytaleQA)
Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension arXiv:2203.13947
Question AnsweringonSQuAD1.1 dev
EM· 2019-10-02
77.7
best: 90.06 (T5-11B)
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter arXiv:1910.01108

Methodology17 results

ClassificationonCivil Comments
GMB BNSP· 2023-01-26
0.9644
SOTA
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonSearchsnippets
Accuracy· 2022-11-30
89.69
SOTA
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
ClassificationonUK Key Stage Readability
F1· 2024-11-26
74.4
best: 99.6 (ELECTRA + ANN)
What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics arXiv:2411.17593
Data MiningonIMDb Movie Reviews
Accuracy· 2023-08-07
93.4
best: 95.6 (ELECTRA)
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining arXiv:2308.03235
Data MiningonIMDb Movie Reviews
F1· 2023-08-07
93.5
best: 95.6 (ELECTRA)
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining arXiv:2308.03235
Interpretable Machine LearningonIMDb Movie Reviews
Accuracy· 2023-08-07
93.4
best: 95.6 (ELECTRA)
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining arXiv:2308.03235
Interpretable Machine LearningonIMDb Movie Reviews
F1· 2023-08-07
93.5
best: 95.6 (ELECTRA)
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining arXiv:2308.03235
ClassificationonCivil Comments
AUROC· 2023-01-26
0.9804
best: 0.9818 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
GMB BPSN· 2023-01-26
0.874
best: 0.901 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
GMB Subgroup· 2023-01-26
0.8762
best: 0.8807 (RoBERTa Focal Loss)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
Macro F1· 2023-01-26
0.3879
best: 0.4749 (RoBERTa BCE)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
Micro F1· 2023-01-26
0.5115
best: 0.5958 (Unfreeze Glove ResNet 44)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
Precision· 2023-01-26
0.3572
best: 0.4835 (Unfreeze Glove ResNet 44)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonCivil Comments
Recall· 2023-01-26
0.9001
best: 0.9254 (XLNet)
A benchmark for toxic comment classification on Civil Comments dataset arXiv:2301.11125
ClassificationonR8
Accuracy· 2022-11-30
97.981
best: 98.451 (DeBERTa)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
ClassificationonMR
Accuracy· 2022-11-30
85.31
best: 93.3 (VLAWE)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878
ClassificationonTwitter
Accuracy· 2022-11-30
99.96
best: 99.97 (ERNIE 2.0)
Transformers are Short Text Classifiers: A Study of Inductive Short Text Classifiers on Benchmarks and Real-world Datasets arXiv:2211.16878