TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/CodeGPT-adapted

CodeGPT-adapted

Reported on 9 benchmarks across 2 tasks · 1 paper · 9 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Computer Code9 results

  • Text-to-Code GenerationonCodeXGLUE - CONCODE
    BLEU· 2021-02-09
    32.79
    best: 41.48 (CodeT5)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Text-to-Code GenerationonCodeXGLUE - CONCODE
    CodeBLEU· 2021-02-09
    27.74
    best: 44.1 (CodeT5)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Text-to-Code GenerationonCodeXGLUE - CONCODE
    EM· 2021-02-09
    20.1
    best: 22.7 (CodeT5)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - Github Java Corpus
    Accuracy (token-level)· 2021-02-09
    77.13
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - Github Java Corpus
    EM (line-level)· 2021-02-09
    26.43
    best: 37.9 (CodeT5+ 770M)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - Github Java Corpus
    Edit Sim (line-level)· 2021-02-09
    63.03
    best: 72.25 (CodeT5+ 770M)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - PY150
    Accuracy (token-level)· 2021-02-09
    75.11
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - PY150
    EM (line-level)· 2021-02-09
    39.65
    best: 44.86 (CodeT5+ 770M)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664
  • Code CompletiononCodeXGLUE - PY150
    Edit Sim (line-level)· 2021-02-09
    69.84
    best: 74.22 (CodeT5+ 770M)
    SOTA
    CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and GenerationarXiv:2102.04664