TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/CodeBERT (MLM)

CodeBERT (MLM)

Reported on 21 benchmarks across 3 tasks · 1 paper · 6 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Adversarial7 results

  • Text GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.48
    best: 20.39 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.79
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.55
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.51
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.59
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.95
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Natural Language Processing7 results

  • Code GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.48
    best: 20.39 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.79
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.55
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.51
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.59
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.95
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Computer Code7 results

  • Code Documentation GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.48
    best: 20.39 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.79
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.55
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.51
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.59
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.95
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155