TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Methods/XLM

XLM

Natural Language ProcessingIntroduced 200057 papers
Source Paper

Description

XLM is a Transformer based architecture that is pre-trained using one of three language modelling objectives:

  1. Causal Language Modeling - models the probability of a word given the previous words in a sentence.
  2. Masked Language Modeling - the masked language modeling objective of BERT.
  3. Translation Language Modeling - a (new) translation language modeling objective for improving cross-lingual pre-training.

The authors find that both the CLM and MLM approaches provide strong cross-lingual features that can be used for pretraining models.

Papers Using This Method

BioBridge: Unified Bio-Embedding with Bridging Modality in Code-Switched EMR2024-12-16XLM for Autonomous Driving Systems: A Comprehensive Review2024-09-16Ax-to-Grind Urdu: Benchmark Dataset for Urdu Fake News Detection2024-03-20Mapping Transformer Leveraged Embeddings for Cross-Lingual Document Representation2024-01-12An Empirical study of Unsupervised Neural Machine Translation: analyzing NMT output, model's behavior and sentences' contribution2023-12-19MedAI Dialog Corpus (MEDIC): Zero-Shot Classification of Doctor and AI Responses in Health Consultations2023-10-19Benchmarking Procedural Language Understanding for Low-Resource Languages: A Case Study on Turkish2023-09-13PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity2023-05-13CLaC at SemEval-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling approaches for NER2023-05-05Exploring Methods for Building Dialects-Mandarin Code-Mixing Corpora: A Case Study in Taiwanese Hokkien2023-01-21ALIGN-MLM: Word Embedding Alignment is Crucial for Multilingual Pre-training2022-11-15BERT-Sort: A Zero-shot MLM Semantic Encoder on Ordinal Features for AutoML2022-06-01GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models2022-05-24Persian Natural Language Inference: A Meta-learning approach2022-05-18HiNER: A Large Hindi Named Entity Recognition Dataset2022-04-28Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models2022-04-11Are You Robert or RoBERTa? Deceiving Online Authorship Attribution Models Using Neural Text Generators2022-03-18"A Passage to India": Pre-trained Word Embeddings for Indian Languages2021-12-27Prix-LM: Pretraining for Multilingual Knowledge Base Construction2021-10-16Cross-Language Learning for Entity Matching2021-10-07