Named Entity Recognition (NER) on CoNLL 2003 (English)

Metric: F1 (higher is better)

LeaderboardDataset

Loading chart...

Results

Hide extra data

Sort:

#	Model↕	F1▼	Extra Data	Paper	Date↕	Code
1	ACE + document-context	94.6	No	Automated Concatenation of Embeddings for Struct...	2020-10-10	Code
2	LUKE 483M	94.3	No	LUKE: Deep Contextualized Entity Representations...	2020-10-02	Code
3	Co-regularized LUKE	94.22	No	Learning from Noisy Labels for Entity-Centric In...	2021-04-17	Code
4	LUKE + SubRegWeigh (K-means)	94.2	No	SubRegWeigh: Effective and Efficient Annotation ...	2024-09-10	Code
5	ASP+T5-3B	94.1	No	Autoregressive Structured Prediction with Langua...	2022-10-26	Code
6	FLERT XLM-R	94.09	Yes	FLERT: Document-Level Features for Named Entity ...	2020-11-13	Code
7	PL-Marker	94	No	Packed Levitated Marker for Entity and Relation ...	2021-09-13	Code
8	CL-KL	93.85	No	Improving Named Entity Recognition by External C...	2021-05-08	Code
9	XLNet-GCN	93.82	No	Named entity recognition architecture combining ...	2021-12-15	Code
10	RoBERTa + SubRegWeigh (K-means)	93.81	No	SubRegWeigh: Effective and Efficient Annotation ...	2024-09-10	Code
11	ASP+flan-T5-large	93.8	No	Autoregressive Structured Prediction with Langua...	2022-10-26	Code
12	InferNER	93.76	No	-	-	-
13	Cross-sentence context (First)	93.74	No	Exploring Cross-sentence Contexts for Named Enti...	2020-06-02	Code
14	XLM-RoBERTa-large union	93.69	No	Transformer-based Named Entity Recognition with ...	2024-06-25	-
15	Baseline + BS	93.65	No	Boundary Smoothing for Named Entity Recognition	2022-04-26	Code
16	ACE	93.64	No	Automated Concatenation of Embeddings for Struct...	2020-10-10	Code
17	BERT-CRF	93.6	No	Focusing on Potential Named Entities During Acti...	2021-11-06	Code
18	CNN Large + fine-tune	93.5	Yes	Cloze-driven Pretraining of Self-attention Netwo...	2019-03-19	-
19	Biaffine-NER	93.5	No	Named Entity Recognition as Dependency Parsing	2020-05-14	Code
20	GCDT + BERT-L	93.47	Yes	GCDT: A Global Context Enhanced Deep Transition ...	2019-06-06	Code
21	I-DARTS + Flair	93.47	Yes	-	-	Code
22	CrossWeigh + Pooled Flair	93.43	No	CrossWeigh: Training Named Entity Tagger from Im...	2019-09-03	Code
23	LSTM-CRF+ELMo+BERT+Flair	93.38	No	Neural Architectures for Nested NER through Line...	2019-08-19	Code
24	Hierarchical + BERT	93.37	No	Hierarchical Contextualized Representation for N...	2019-11-06	Code
25	BERT-CRF (Replicated in AdaSeq)	93.35	No	Improving Named Entity Recognition by External C...	2021-05-08	Code
26	BERT-MRC+DSC	93.33	Yes	Dice Loss for Data-imbalanced NLP Tasks	2019-11-07	Code
27	XLNet	93.28	No	Named entity recognition architecture combining ...	2021-12-15	Code
28	BARTNER	93.24	No	A Unified Generative Framework for Various NER S...	2021-06-02	Code
29	GoLLIE	93.1	Yes	GoLLIE: Annotation Guidelines improve Zero-Shot ...	2023-10-05	Code
30	Flair embeddings	93.09	Yes	-	-	Code
31	PromptNER [RoBERTa-large]	93.08	Yes	PromptNER: Prompt Locating and Typing for Named ...	2023-05-26	Code
32	W2NER	93.07	No	Unified Named Entity Recognition as Word-Word Re...	2021-12-19	Code
33	BERT-MRC	93.04	Yes	A Unified MRC Framework for Named Entity Recogni...	2019-10-25	Code
34	Locate and Label	92.94	No	Locate and Label: A Two-stage Identifier for Nes...	2021-05-14	Code
35	PIQN	92.87	No	Parallel Instance Query Network for Named Entity...	2022-03-20	Code
36	DiffusionNER	92.78	No	DiffusionNER: Boundary Diffusion for Named Entit...	2023-05-22	Code
37	HSCRF + softdict	92.75	Yes	-	-	Code
38	TENER	92.62	No	TENER: Adapting Transformer Encoder for Named En...	2019-11-10	Code
39	CVT + Multi-Task	92.61	Yes	Semi-Supervised Sequence Modeling with Cross-Vie...	2018-09-22	Code
40	CVT + Multi-Task + Large	92.61	No	Semi-Supervised Sequence Modeling with Cross-Vie...	2018-09-22	Code
41	Stack LSTM	92.43	No	Joint Learning of Named Entity Recognition and E...	2019-07-18	-
42	PromptNER [BERT-large]	92.41	No	PromptNER: Prompt Locating and Typing for Named ...	2023-05-26	Code
43	DGLSTM-CRF + ELMo (L=2) 3.0pt1-4.51.5	92.4	No	Dependency-Guided LSTM-CRF for Named Entity Reco...	2019-09-23	Code
44	GRN	92.34	No	GRN: Gated Relation Network to Enhance Convoluti...	2019-07-12	Code
45	Neural-CRF+AE	92.29	No	Evaluating the Utility of Hand-crafted Features ...	2018-08-28	Code
46	MGNER	92.28	No	Multi-Grained Named Entity Recognition	2019-06-20	Code
47	BiLSTM-CRF+ELMo	92.22	Yes	Deep contextualized word representations	2018-02-15	Code
48	SpanRel	92.2	No	Generalizing Natural Language Analysis through S...	2019-11-10	Code
49	LD-Net	92.03	No	Efficient Contextualized Representation: Languag...	2018-04-20	Code
50	GCDT	91.96	No	GCDT: A Global Context Enhanced Deep Transition ...	2019-06-06	Code
51	Hierarchical	91.96	No	Hierarchical Contextualized Representation for N...	2019-11-06	Code
52	CRF + AutoEncoder	91.87	No	Evaluating the Utility of Hand-crafted Features ...	2018-08-28	Code
53	PRISM	91.8	No	-	-	Code
54	GraphIE (GCN+BiLSTM)	91.74	No	-	-	-
55	Bi-LSTM-CRF + Lexical Features	91.73	No	Robust Lexical Features for Improved Neural Netw...	2018-06-09	Code
56	IntNet + BiLSTM-CRF	91.64	No	Learning Better Internal Structure of Words for ...	2018-10-29	-
57	Yang et al. ([2017a])	91.62	No	Neural Reranking for Named Entity Recognition	2017-07-17	Code
58	Bi-LSTM-CNN	91.62	No	Named Entity Recognition with Bidirectional LSTM...	2015-11-26	Code
59	S-LSTM	91.57	No	Sentence-State LSTM for Text Representation	2018-05-07	Code
60	LSTM with dynamic skip	91.56	No	Long Short-Term Memory with Dynamic Skip Connect...	2018-11-09	Code
61	Adversarial Bi-LSTM	91.56	No	Robust Multilingual Part-of-Speech Tagging via A...	2017-11-14	Code
62	HSCRF	91.38	No	Hybrid semi-Markov CRF for Neural Sequence Label...	2018-05-10	Code
63	IXA pipes	91.36	No	Robust Multilingual Named Entity Recognition wit...	2017-01-31	Code
64	NCRF++	91.35	No	NCRF++: An Open-source Neural Sequence Labeling ...	2018-06-14	Code
65	Yang et al.	91.26	No	Transfer Learning for Sequence Tagging with Hier...	2017-03-18	Code
66	LM-LSTM-CRF	91.24	No	Empower Sequence Labeling with Task-Aware Neural...	2017-09-13	Code
67	Bi-LSTM-CNN-CRF	91.22	No	-	-	Code
68	BLSTM-CNN-CRF	91.21	No	End-to-end Sequence Labeling via Bi-directional ...	2016-03-04	Code
69	Bi-LSTM + Logic rules	91.18	No	Harnessing Deep Neural Networks with Logic Rules	2016-03-21	Code
70	LSTM-CRF	90.94	No	Neural Architectures for Named Entity Recognition	2016-03-04	Code
71	GCN	88.63	No	Named entity recognition architecture combining ...	2021-12-15	Code
72	SWEM-CRF	86.28	No	Baseline Needs More Love: On Simple Word-Embeddi...	2018-05-24	Code
73	VSL-GG-Hier	84.7	No	Variational Sequential Labelers for Semi-Supervi...	2019-06-23	Code

#1ACE + document-contextSOTA
94.6
F1· 2020-10-10
Automated Concatenation of Embeddings for Structured Prediction Code
#2LUKE 483MSOTA
94.3
F1· 2020-10-02
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Code
#3Co-regularized LUKE
94.22
F1· 2021-04-17
Learning from Noisy Labels for Entity-Centric Information Extraction Code
#4LUKE + SubRegWeigh (K-means)
94.2
F1· 2024-09-10
SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization Code
#5ASP+T5-3B
94.1
F1· 2022-10-26
Autoregressive Structured Prediction with Language Models Code
#6FLERT XLM-R
94.09
F1· Extra Data· 2020-11-13
FLERT: Document-Level Features for Named Entity Recognition Code
#7PL-Marker
94
F1· 2021-09-13
Packed Levitated Marker for Entity and Relation Extraction Code
#8CL-KL
93.85
F1· 2021-05-08
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning Code
#9XLNet-GCN
93.82
F1· 2021-12-15
Named entity recognition architecture combining contextual and global features Code
#10RoBERTa + SubRegWeigh (K-means)
93.81
F1· 2024-09-10
SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization Code
#11ASP+flan-T5-large
93.8
F1· 2022-10-26
Autoregressive Structured Prediction with Language Models Code
#12InferNER
93.76
F1
No paper
#13Cross-sentence context (First)SOTA
93.74
F1· 2020-06-02
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT Code
#14XLM-RoBERTa-large union
93.69
F1· 2024-06-25
Transformer-based Named Entity Recognition with Combined Data Representation
#15Baseline + BS
93.65
F1· 2022-04-26
Boundary Smoothing for Named Entity Recognition Code
#16ACE
93.64
F1· 2020-10-10
Automated Concatenation of Embeddings for Structured Prediction Code
#17BERT-CRF
93.6
F1· 2021-11-06
Focusing on Potential Named Entities During Active Label Acquisition Code
#18CNN Large + fine-tuneSOTA
93.5
F1· Extra Data· 2019-03-19
Cloze-driven Pretraining of Self-attention Networks
#19Biaffine-NER
93.5
F1· 2020-05-14
Named Entity Recognition as Dependency Parsing Code
#20GCDT + BERT-L
93.47
F1· Extra Data· 2019-06-06
GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling Code
#21I-DARTS + Flair
93.47
F1· Extra Data
No paperCode
#22CrossWeigh + Pooled Flair
93.43
F1· 2019-09-03
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations Code
#23LSTM-CRF+ELMo+BERT+Flair
93.38
F1· 2019-08-19
Neural Architectures for Nested NER through Linearization Code
#24Hierarchical + BERT
93.37
F1· 2019-11-06
Hierarchical Contextualized Representation for Named Entity Recognition Code
#25BERT-CRF (Replicated in AdaSeq)
93.35
F1· 2021-05-08
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning Code
#26BERT-MRC+DSC
93.33
F1· Extra Data· 2019-11-07
Dice Loss for Data-imbalanced NLP Tasks Code
#27XLNet
93.28
F1· 2021-12-15
Named entity recognition architecture combining contextual and global features Code
#28BARTNER
93.24
F1· 2021-06-02
A Unified Generative Framework for Various NER Subtasks Code
#29GoLLIE
93.1
F1· Extra Data· 2023-10-05
GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction Code
#30Flair embeddings
93.09
F1· Extra Data
No paperCode
#31PromptNER [RoBERTa-large]
93.08
F1· Extra Data· 2023-05-26
PromptNER: Prompt Locating and Typing for Named Entity Recognition Code
#32W2NER
93.07
F1· 2021-12-19
Unified Named Entity Recognition as Word-Word Relation Classification Code
#33BERT-MRC
93.04
F1· Extra Data· 2019-10-25
A Unified MRC Framework for Named Entity Recognition Code
#34Locate and Label
92.94
F1· 2021-05-14
Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition Code
#35PIQN
92.87
F1· 2022-03-20
Parallel Instance Query Network for Named Entity Recognition Code
#36DiffusionNER
92.78
F1· 2023-05-22
DiffusionNER: Boundary Diffusion for Named Entity Recognition Code
#37HSCRF + softdict
92.75
F1· Extra Data
No paperCode
#38TENER
92.62
F1· 2019-11-10
TENER: Adapting Transformer Encoder for Named Entity Recognition Code
#39CVT + Multi-TaskSOTA
92.61
F1· Extra Data· 2018-09-22
Semi-Supervised Sequence Modeling with Cross-View Training Code
#40CVT + Multi-Task + Large
92.61
F1· 2018-09-22
Semi-Supervised Sequence Modeling with Cross-View Training Code
#41Stack LSTM
92.43
F1· 2019-07-18
Joint Learning of Named Entity Recognition and Entity Linking
#42PromptNER [BERT-large]
92.41
F1· 2023-05-26
PromptNER: Prompt Locating and Typing for Named Entity Recognition Code
#43DGLSTM-CRF + ELMo (L=2) 3.0pt1-4.51.5
92.4
F1· 2019-09-23
Dependency-Guided LSTM-CRF for Named Entity Recognition Code
#44GRN
92.34
F1· 2019-07-12
GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition Code
#45Neural-CRF+AESOTA
92.29
F1· 2018-08-28
Evaluating the Utility of Hand-crafted Features in Sequence Labelling Code
#46MGNER
92.28
F1· 2019-06-20
Multi-Grained Named Entity Recognition Code
#47BiLSTM-CRF+ELMoSOTA
92.22
F1· Extra Data· 2018-02-15
Deep contextualized word representations Code
#48SpanRel
92.2
F1· 2019-11-10
Generalizing Natural Language Analysis through Span-relation Representations Code
#49LD-Net
92.03
F1· 2018-04-20
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling Code
#50GCDT
91.96
F1· 2019-06-06
GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling Code
#51Hierarchical
91.96
F1· 2019-11-06
Hierarchical Contextualized Representation for Named Entity Recognition Code
#52CRF + AutoEncoder
91.87
F1· 2018-08-28
Evaluating the Utility of Hand-crafted Features in Sequence Labelling Code
#53PRISM
91.8
F1
No paperCode
#54GraphIE (GCN+BiLSTM)
91.74
F1
No paper
#55Bi-LSTM-CRF + Lexical Features
91.73
F1· 2018-06-09
Robust Lexical Features for Improved Neural Network Named-Entity Recognition Code
#56IntNet + BiLSTM-CRF
91.64
F1· 2018-10-29
Learning Better Internal Structure of Words for Sequence Labeling
#57Yang et al. ([2017a])
91.62
F1· 2017-07-17
Neural Reranking for Named Entity Recognition Code
#58Bi-LSTM-CNNSOTA
91.62
F1· 2015-11-26
Named Entity Recognition with Bidirectional LSTM-CNNs Code
#59S-LSTM
91.57
F1· 2018-05-07
Sentence-State LSTM for Text Representation Code
#60LSTM with dynamic skip
91.56
F1· 2018-11-09
Long Short-Term Memory with Dynamic Skip Connections Code
#61Adversarial Bi-LSTM
91.56
F1· 2017-11-14
Robust Multilingual Part-of-Speech Tagging via Adversarial Training Code
#62HSCRF
91.38
F1· 2018-05-10
Hybrid semi-Markov CRF for Neural Sequence Labeling Code
#63IXA pipes
91.36
F1· 2017-01-31
Robust Multilingual Named Entity Recognition with Shallow Semi-Supervised Features Code
#64NCRF++
91.35
F1· 2018-06-14
NCRF++: An Open-source Neural Sequence Labeling Toolkit Code
#65Yang et al.
91.26
F1· 2017-03-18
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks Code
#66LM-LSTM-CRF
91.24
F1· 2017-09-13
Empower Sequence Labeling with Task-Aware Neural Language Model Code
#67Bi-LSTM-CNN-CRF
91.22
F1
No paperCode
#68BLSTM-CNN-CRF
91.21
F1· 2016-03-04
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF Code
#69Bi-LSTM + Logic rules
91.18
F1· 2016-03-21
Harnessing Deep Neural Networks with Logic Rules Code
#70LSTM-CRF
90.94
F1· 2016-03-04
Neural Architectures for Named Entity Recognition Code
#71GCN
88.63
F1· 2021-12-15
Named entity recognition architecture combining contextual and global features Code
#72SWEM-CRF
86.28
F1· 2018-05-24
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms Code
#73VSL-GG-Hier
84.7
F1· 2019-06-23
Variational Sequential Labelers for Semi-Supervised Learning Code