TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/BERT-Large-CAS

BERT-Large-CAS

Reported on 6 benchmarks across 1 task · 1 paper · 4 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Medical6 results

  • Language ModellingonPenn Treebank (Word Level)
    Test perplexity· uses extra data· 2019-04-20
    31.3
    best: 20.5 (GPT-3 (Zero-Shot))
    SOTA
    Language Models with TransformersarXiv:1904.09408
  • Language ModellingonPenn Treebank (Word Level)
    Validation perplexity· uses extra data· 2019-04-20
    36.1
    SOTA
    Language Models with TransformersarXiv:1904.09408
  • Language ModellingonWikiText-2
    Test perplexity· uses extra data· 2019-04-20
    34.1
    best: 8.21 (SparseGPT (175B, 50% Sparsity))
    SOTA
    Language Models with TransformersarXiv:1904.09408
  • Language ModellingonWikiText-2
    Validation perplexity· uses extra data· 2019-04-20
    37.7
    best: 15.69 (GPT-2 (fine-tuned))
    SOTA
    Language Models with TransformersarXiv:1904.09408
  • Language ModellingonWikiText-103
    Test perplexity· 2019-04-20
    20.4
    best: 2.4 (RETRO (7.5B))
    Language Models with TransformersarXiv:1904.09408
  • Language ModellingonWikiText-103
    Validation perplexity· 2019-04-20
    19.6
    best: 13.11 (Ensemble of All)
    Language Models with TransformersarXiv:1904.09408