TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

SotA/Medical/Language Modelling

Language Modelling

111 benchmarks17610 papers

A language model is a model of natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, and information retrieval.

Large language models (LLMs), currently their most advanced form, are predominantly based on transformers trained on larger datasets (frequently using words scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as word n-gram language model.

Source: Wikipedia

Benchmarks

Language Modelling on WikiText-103

Test perplexityValidation perplexityNumber of params

Language Modelling on Penn Treebank (Word Level)

Test perplexityValidation perplexityParams

Language Modelling on enwik8

Bit per Character (BPC)Number of params

Language Modelling on WikiText-2

Test perplexityValidation perplexityNumber of params

Language Modelling on LAMBADA

AccuracyPerplexity

Language Modelling on The Pile

Bits per byteTest perplexity

Language Modelling on One Billion Word

PPLValidation perplexityNumber of params

Language Modelling on Text8

Bit per Character (BPC)Number of params

Language Modelling on Penn Treebank (Character Level)

Bit per Character (BPC)Number of params

Language Modelling on Hutter Prize

Bit per Character (BPC)Number of params

Language Modelling on OpenWebText

eval_perplexityeval_lossparameters

Language Modelling on SALMon

Sentiment ConsistencySpeaker ConsistencyGender ConsistencyRoom ConsistencyBackground (Domain) ConsistencyBackground (Random) ConsistencySentiment AlignmentBackground Alignment

Language Modelling on SCROLLS

Avg.QsprNrtvCNLIGovRepSumScrQMSumQALT EM-T/H

Language Modelling on C4

PerplexityTPUv3 HoursSteps

Language Modelling on LRA

AvgImageListOpsPathfinderTextRetrievalPathfinder-X

Language Modelling on Annotated corpus for semantic similarity of clinical trial outcomes (expanded corpus)

F1PrecisionRecall

Language Modelling on Annotated corpus for semantic similarity of clinical trial outcomes (original corpus)

F1PrecisionRecall

Language Modelling on SICK

MSEPearson CorrelationSpearman Correlation

Language Modelling on BIG-bench-lite

Accuracy

Language Modelling on BIOSSES

Pearson Correlation

Language Modelling on MultiNews test

Perplexity

Language Modelling on MultiNews val

Perplexity

Language Modelling on Wiki-40B

Perplexity

Language Modelling on CLUE (AFQMC)

Accuracy

Language Modelling on CLUE (C3)

Accuracy

Language Modelling on CLUE (CMNLI)

Accuracy

Language Modelling on CLUE (CMRC2018)

Accuracy

Language Modelling on CLUE (DRCD)

Accuracy

Language Modelling on CLUE (OCNLI_50K)

Accuracy

Language Modelling on CLUE (WSC1.1)

Accuracy

Language Modelling on FewCLUE (BUSTM)

Accuracy

Language Modelling on FewCLUE (CHID-FC)

Accuracy

Language Modelling on FewCLUE (CLUEWSC-FC)

Accuracy

Language Modelling on FewCLUE (EPRSTMT)

Accuracy

Language Modelling on FewCLUE (OCNLI-FC)

Accuracy

Language Modelling on VietMed

PPL

Language Modelling on Bookcorpus2

BPB

Language Modelling on Gutenberg PG-19

BPB

Language Modelling on OpenSubtitles

BPB

Language Modelling on PubMed Central

BPB

Language Modelling on StackExchange

BPB

Language Modelling on USPTO Backgrounds

BPB

Language Modelling on Ubuntu IRC

BPB

Language Modelling on 100 sleep nights of 8 caregivers

10%

Language Modelling on 2000 HUB5 English

10-stage average accuracy

Language Modelling on Arxiv HEP-TH citation graph

BPB

Language Modelling on Books3

BPB

Language Modelling on CHIP-STS

Macro F1

Language Modelling on ClinicalSTS

Pearson Correlation

Language Modelling on Curation Corpus

BPB

Language Modelling on DAVIS-DTA

CI

Language Modelling on DM Mathematics

BPB

Language Modelling on FreeLaw

BPB

Language Modelling on GitHub

BPB

Language Modelling on HackerNews

BPB

Language Modelling on MedSTS

Pearson Correlation

Language Modelling on NIH ExPorter

BPB

Language Modelling on OpenWebtext2

BPB

Language Modelling on PTB Diagnostic ECG Database

PPL

Language Modelling on PhilPapers

BPB

Language Modelling on Pile CC

BPB

Language Modelling on PubMed Cognitive Control Abstracts

BPB

Language Modelling on Text8 dev

Bit per Character (BPC)

Language Modelling on enwik8 dev

Bit per Character (BPC)

Language Modelling on enwiki8

Bit per Character (BPC)

Language Modelling on language-modeling-recommendation

1:1 Accuracy

Language Modelling on A1

0..5sec