Papers With Code 2 | ML Benchmarks, SotA Results & Code

Dataset Description

Authors: Ludovic Moncla, Katherine McDonough and Denis Vigier in the framework of the GEODE project.
Data source: ARTFL Encyclopédie Project, University of Chicago
Github repository: https://github.com/GEODE-project/ner-spancat-edda
Language: French
License: cc-by-nc-4.0
Zenodo repository: https://zenodo.org/records/10530177

Dataset Summary

This dataset contains semantic annotations (at the token and span levels) for named entities (such as Spatial, Person, and MISC), nominal entities, as well as nested named entities, spatial relations, and other relevant information within French encyclopedic entries.

The span tagset is as follows:

NC-Spatial: a common noun that identifies a spatial entity (nominal spatial entity) including natural features, e.g. ville, la rivière, royaume.
NP-Spatial: a proper noun identifying the name of a place (spatial named entities), e.g. France, Paris, la Chine.
ENE-Spatial: nested spatial entity , e.g. ville de France, royaume de Naples, la mer Baltique.
Relation: spatial relation, e.g. dans, sur, à 10 lieues de.
Latlong: geographic coordinates, e.g. Long. 19. 49. lat. 43. 55. 44.
NC-Person: a common noun that identifies a person (nominal spatial entity), e.g. roi, l'empereur, les auteurs.
NP-Person: a proper noun identifying the name of a person (person named entities), e.g. Louis XIV, Pline, les Romains.
ENE-Person: nested people entity, e.g. le czar Pierre, roi de Macédoine
NP-Misc: a proper noun identifying entities not classified as spatial or person, e.g. l'Eglise, 1702, Pélasgique.
ENE-Misc: nested named entity not classified as spatial or person, e.g. l'ordre de S. Jacques, la déclaration du 21 Mars 1671.
Head: entry name
Domain-Mark: words indicating the knowledge domain (usually after the head and between parenthesis), e.g. Géographie, Geog., en Anatomie.

Supported Tasks

token-classification or span-classification: The dataset can be used to train a model for token-classification or span-classification. It is more specifically designed for spatial role labelling. A spacy custom spancat model is available at: https://huggingface.co/GEODE/fr_spacy_custom_spancat_edda.

Dataset Structure

The dataset is provided as JSONL files[^1] where each row follows the following structure:

{
  "text": "ILLESCAS, (Géog.) petite ville d'Espagne <...> ",
  "meta": {"volume": 8, "head": "ILLESCAS", "author": "unsigned", "domain_article": "Géographie", "domain_paragraph": "Géographie", "article": 2637, "paragraph": 1},
  "tokens": [{"text": "ILLESCAS", "start": 0, "end": 8, "id": 0, "ws": false}, {"text": ",", "start": 8, "end": 9, "id": 1, "ws": true}, {"text": "(", "start": 10, "end": 11, "id": 2, "ws": false}, {"text": "Géog", "start": 11, "end": 15, "id": 3, "ws": false}, {"text": ".", "start": 15, "end": 16, "id": 4, "ws": false}, {"text": ")", "start": 16, "end": 17, "id": 5, "ws": true}, {"text": "petite", "start": 18, "end": 24, "id": 6, "ws": true}, {"text": "ville", "start": 25, "end": 30, "id": 7, "ws": true}, {"text": "d'", "start": 31, "end": 33, "id": 8, "ws": false}, {"text": "Espagne", "start": 33, "end": 40, "id": 9, "ws": false}, {"text": ",", "start": 40, "end": 41, "id": 10, "ws": true} <...>],
  "spans": [{"text": "ILLESCAS", "start": 0, "end": 8, "token_start": 0, "token_end": 0, "label": "Head"}, {"text": "Géog.", "start": 11, "end": 16, "token_start": 3, "token_end": 4, "label": "Domain-mark"}, {"text": "petite ville", "start": 18, "end": 30, "token_start": 6, "token_end": 7, "label": "NC-Spatial"}, {"text": "petite ville d'Espagne", "start": 18, "end": 40, "token_start": 6, "token_end": 9, "label": "ENE-Spatial"}, {"text": "petite ville d'Espagne, dans la nouvelle Castille", "start": 18, "end": 67, "token_start": 6, "token_end": 14, "label": "ENE-Spatial"}, {"text": "Espagne", "start": 33, "end": 40, "token_start": 9, "token_end": 9, "label": "NP-Spatial"}, <...>]
}

Each data contains four main fields:

text: plain text of a paragraph.
meta: metadata from the ARTFL Encyclopédie about the paragraph such volume, article, paragraph id, headword, etc.
tokens: list of tokens, with their text, id, start and end position at the character level.
spans: list of spans (i.e., annotations), with their text, label, start and end position at the character level.

[^1]:spaCy binary files are also available on the Github and Zenodo repositories.

Data Splits

The dataset consists of 2200 paragraphs randomly selected out of 2001 Encyclopédie's entries. All paragraphs were written in French and are distributed as follows among the Encyclopédie knowledge domains:

| Knowledge domain | Paragraphs | |---|:---:| | Géographie | 1096 | | Histoire | 259 | | Droit Jurisprudence | 113 | | Physique | 92 | | Métiers | 92 | | Médecine | 88 | | Philosophie | 69 | | Histoire naturelle | 65 | | Belles-lettres | 65 | | Militaire | 62 | | Commerce | 48 | | Beaux-arts | 44 | | Agriculture | 36 | | Chasse | 31 | | Religion | 23 | | Musique | 17 |

The spans/entities were labeled by the project team along with using pre-labelling with early models to speed up the labelling process. A train/val/test split was used. Validation and test sets are composed of 200 paragraphs each: 100 classified as 'Géographie' and 100 from another knowledge domain. The datasets have the following breakdown of tokens and spans/entities.

| | Train | Validation | Test| |---|:---:|:---:|:---:| |Paragraphs| 1,800 | 200 | 200| | Tokens | 132,398 | 14,959 | 13,881 | | NC-Spatial | 3,252 | 358 | 355 | | NP-Spatial | 4,707 | 464 | 519 | | ENE-Spatial | 3,043 | 326 | 334 | | Relation | 2,093 | 219 | 226 | | Latlong | 553 | 66 | 72 | | NC-Person | 1,378 | 132 | 133 | | NP-Person | 1,599 | 170 | 150 | | ENE-Person | 492 | 49 | 57 | | NP-Misc | 948 | 108 | 96 | | ENE-Misc | 255 | 31 | 22 | | Head | 1,261 | 142 | 153 | | Domain-Mark | 1,069 | 122 | 133 |

Additional Information

Dataset Curators

List of people involved in annotating the dataset:

Ludovic Moncla (@ludovicmoncla), INSA Lyon, CNRS, LIRIS UMR 5205
Katherine McDonough (@kmcdono2, Lancaster University & The Alan Turing Institute

Cite this work

Moncla, L., Vigier, D., & McDonough, K. (2024). GeoEDdA: A Gold Standard Dataset for Geo-semantic Annotation of Diderot & d’Alembert’s Encyclopédie. In proceedings of the 2nd International Workshop on Geographic Information Extraction from Texts (GeoExT'24), ECIR Conference, Glasgow, UK.

Acknowledgement

The authors are grateful to the ASLAN project (ANR-10-LABX-0081) of the Université de Lyon, for its financial support within the French program "Investments for the Future" operated by the National Research Agency (ANR). Data courtesy the ARTFL Encyclopédie Project, University of Chicago.

Dataset Description

Authors: Ludovic Moncla, Katherine McDonough and Denis Vigier in the framework of the GEODE project.
Data source: ARTFL Encyclopédie Project, University of Chicago
Github repository: https://github.com/GEODE-project/ner-spancat-edda
Language: French
License: cc-by-nc-4.0
Zenodo repository: https://zenodo.org/records/10530177

Dataset Summary

The span tagset is as follows:

NC-Spatial: a common noun that identifies a spatial entity (nominal spatial entity) including natural features, e.g. ville, la rivière, royaume.
NP-Spatial: a proper noun identifying the name of a place (spatial named entities), e.g. France, Paris, la Chine.
ENE-Spatial: nested spatial entity , e.g. ville de France, royaume de Naples, la mer Baltique.
Relation: spatial relation, e.g. dans, sur, à 10 lieues de.
Latlong: geographic coordinates, e.g. Long. 19. 49. lat. 43. 55. 44.
NC-Person: a common noun that identifies a person (nominal spatial entity), e.g. roi, l'empereur, les auteurs.
NP-Person: a proper noun identifying the name of a person (person named entities), e.g. Louis XIV, Pline, les Romains.
ENE-Person: nested people entity, e.g. le czar Pierre, roi de Macédoine
NP-Misc: a proper noun identifying entities not classified as spatial or person, e.g. l'Eglise, 1702, Pélasgique.
ENE-Misc: nested named entity not classified as spatial or person, e.g. l'ordre de S. Jacques, la déclaration du 21 Mars 1671.
Head: entry name
Domain-Mark: words indicating the knowledge domain (usually after the head and between parenthesis), e.g. Géographie, Geog., en Anatomie.

Supported Tasks

token-classification or span-classification: The dataset can be used to train a model for token-classification or span-classification. It is more specifically designed for spatial role labelling. A spacy custom spancat model is available at: https://huggingface.co/GEODE/fr_spacy_custom_spancat_edda.

Dataset Structure

The dataset is provided as JSONL files[^1] where each row follows the following structure:

{
  "text": "ILLESCAS, (Géog.) petite ville d'Espagne <...> ",
  "meta": {"volume": 8, "head": "ILLESCAS", "author": "unsigned", "domain_article": "Géographie", "domain_paragraph": "Géographie", "article": 2637, "paragraph": 1},
  "tokens": [{"text": "ILLESCAS", "start": 0, "end": 8, "id": 0, "ws": false}, {"text": ",", "start": 8, "end": 9, "id": 1, "ws": true}, {"text": "(", "start": 10, "end": 11, "id": 2, "ws": false}, {"text": "Géog", "start": 11, "end": 15, "id": 3, "ws": false}, {"text": ".", "start": 15, "end": 16, "id": 4, "ws": false}, {"text": ")", "start": 16, "end": 17, "id": 5, "ws": true}, {"text": "petite", "start": 18, "end": 24, "id": 6, "ws": true}, {"text": "ville", "start": 25, "end": 30, "id": 7, "ws": true}, {"text": "d'", "start": 31, "end": 33, "id": 8, "ws": false}, {"text": "Espagne", "start": 33, "end": 40, "id": 9, "ws": false}, {"text": ",", "start": 40, "end": 41, "id": 10, "ws": true} <...>],
  "spans": [{"text": "ILLESCAS", "start": 0, "end": 8, "token_start": 0, "token_end": 0, "label": "Head"}, {"text": "Géog.", "start": 11, "end": 16, "token_start": 3, "token_end": 4, "label": "Domain-mark"}, {"text": "petite ville", "start": 18, "end": 30, "token_start": 6, "token_end": 7, "label": "NC-Spatial"}, {"text": "petite ville d'Espagne", "start": 18, "end": 40, "token_start": 6, "token_end": 9, "label": "ENE-Spatial"}, {"text": "petite ville d'Espagne, dans la nouvelle Castille", "start": 18, "end": 67, "token_start": 6, "token_end": 14, "label": "ENE-Spatial"}, {"text": "Espagne", "start": 33, "end": 40, "token_start": 9, "token_end": 9, "label": "NP-Spatial"}, <...>]
}

Each data contains four main fields:

text: plain text of a paragraph.
meta: metadata from the ARTFL Encyclopédie about the paragraph such volume, article, paragraph id, headword, etc.
tokens: list of tokens, with their text, id, start and end position at the character level.
spans: list of spans (i.e., annotations), with their text, label, start and end position at the character level.

[^1]:spaCy binary files are also available on the Github and Zenodo repositories.

Data Splits

Additional Information

Dataset Curators

List of people involved in annotating the dataset:

Ludovic Moncla (@ludovicmoncla), INSA Lyon, CNRS, LIRIS UMR 5205
Katherine McDonough (@kmcdono2, Lancaster University & The Alan Turing Institute

Cite this work

Moncla, L., Vigier, D., & McDonough, K. (2024). GeoEDdA: A Gold Standard Dataset for Geo-semantic Annotation of Diderot & d’Alembert’s Encyclopédie. In proceedings of the 2nd International Workshop on Geographic Information Extraction from Texts (GeoExT'24), ECIR Conference, Glasgow, UK.