TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/CodeBERT (MLM+RTD)

CodeBERT (MLM+RTD)

Reported on 21 benchmarks across 3 tasks · 1 paper · 12 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Adversarial7 results

  • Text GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.99
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21.32
    best: 26.23 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    14.56
    best: 21.87 (CodeTrans-MT-Large)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    8.46
    best: 15.26 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.41
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.66
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    9.54
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Natural Language Processing7 results

  • Code GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.99
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21.32
    best: 26.23 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    14.56
    best: 21.87 (CodeTrans-MT-Large)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    8.46
    best: 15.26 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.41
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.66
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    9.54
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Computer Code7 results

  • Code Documentation GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.99
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    21.32
    best: 26.23 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    14.56
    best: 21.87 (CodeTrans-MT-Large)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    8.46
    best: 15.26 (CodeTrans-MT-Base)
    SOTA
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.41
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.66
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    9.54
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155