TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/pre-train w/ code only

pre-train w/ code only

Reported on 21 benchmarks across 3 tasks · 1 paper

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Adversarial7 results

  • Text GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.15
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.12
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.39
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.3
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    20.71
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.07
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Text GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.36
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Natural Language Processing7 results

  • Code GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.15
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.12
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.39
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.3
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    20.71
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.07
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.36
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155

Computer Code7 results

  • Code Documentation GenerationonCodeSearchNet
    Smoothed BLEU-4· 2020-02-19
    15.15
    best: 15.99 (CodeBERT (MLM+RTD))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Python
    Smoothed BLEU-4· 2020-02-19
    15.12
    best: 20.39 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Go
    Smoothed BLEU-4· 2020-02-19
    26.39
    best: 26.79 (CodeBERT (MLM))
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - JavaScript
    Smoothed BLEU-4· 2020-02-19
    8.3
    best: 25.61 (Transformer)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Php
    Smoothed BLEU-4· 2020-02-19
    20.71
    best: 26.23 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Java
    Smoothed BLEU-4· 2020-02-19
    13.07
    best: 21.87 (CodeTrans-MT-Large)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155
  • Code Documentation GenerationonCodeSearchNet - Ruby
    Smoothed BLEU-4· 2020-02-19
    7.36
    best: 15.26 (CodeTrans-MT-Base)
    CodeBERT: A Pre-Trained Model for Programming and Natural LanguagesarXiv:2002.08155