TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Models/SPECTER

SPECTER

Reported on 13 benchmarks across 8 tasks · 2 papers · 7 SOTA

Note: results are matched by exact model name. Different papers may use the same name for different model variants.

Natural Language Processing7 results

  • Text ClassificationonSciDocs (MeSH)
    F1 (micro)· 2020-04-15
    86.4
    best: 88.7 (SciNCL)
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • Text ClassificationonSciDocs (MAG)
    F1 (micro)· 2020-04-15
    82
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • Document ClassificationonSciDocs (MeSH)
    F1 (micro)· 2020-04-15
    86.4
    best: 88.7 (SciNCL)
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • Document ClassificationonSciDocs (MAG)
    F1 (micro)· 2020-04-15
    82
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • Semantic Textual SimilarityonMTEB
    Spearman Correlation· 2022-10-13
    61.02
    best: 84.54 (AnglE-UAE)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316
  • Text ClusteringonMTEB
    V-Measure· 2022-10-13
    34.06
    best: 43.71 (ST5-XXL)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316
  • Text ClassificationonMTEB
    Accuracy· 2022-10-13
    52.37
    best: 73.42 (ST5-XXL)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316

Methodology5 results

  • Representation LearningonSciDocs
    Avg.· 2020-04-15
    80
    best: 81.8 (SciNCL)
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • ClassificationonSciDocs (MeSH)
    F1 (micro)· 2020-04-15
    86.4
    best: 88.7 (SciNCL)
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • ClassificationonSciDocs (MAG)
    F1 (micro)· 2020-04-15
    82
    SOTA
    SPECTER: Document-level Representation Learning using Citation-informed TransformersarXiv:2004.07180
  • RetrievalonMTEB
    nDCG@10· 2022-10-13
    15.88
    best: 50.25 (SGPT-5.8B-msmarco)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316
  • ClassificationonMTEB
    Accuracy· 2022-10-13
    52.37
    best: 73.42 (ST5-XXL)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316

Knowledge Base1 result

  • Text SummarizationonMTEB
    Spearman Correlation· 2022-10-13
    27.66
    best: 31.57 (MPNet-multilingual)
    MTEB: Massive Text Embedding BenchmarkarXiv:2210.07316