LIT: Label-Informed Transformers on Token-Based Classification - Laboratoire d'Informatique Fondamentale et Appliquée de Tours
Conference Papers Year : 2024

LIT: Label-Informed Transformers on Token-Based Classification

Abstract

Transformer-based language models have led to the investigation of various embedding and modeling techniques for several downstream natural language processing tasks. Nevertheless, the comprehensive exploration of semantic information about the label from encoder and decoder components in these tasks is yet to be fully realized. In this paper, we propose LIT, an end-to-end pipeline architecture that integrates the transformer’s encoder-decoder mechanism with an additional label semantic to token classification tasks (i.e., historical named entity recognition (NER) and automatic term extraction (ATE)). Our findings demonstrate that LIT outperforms the benchmark in F1 with a maximal rise of 9.5% points in the historical NER task and 11.2% points in the ATE task for the gold standard excluding named entities.
Fichier principal
Vignette du fichier
TPDL___LIT__Label_Informed_Transformers_on_Token__and_Sequence_based_Classification (1).pdf (1.19 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04710036 , version 1 (26-09-2024)

Identifiers

Cite

Wenjun Sun, Hanh Thi Hong Tran, Carlos-Emiliano González-Gallardo, Mickaël Coustaty, Antoine Doucet. LIT: Label-Informed Transformers on Token-Based Classification. The 28th International Conference on Theory and Practice of Digital Libraries, Sep 2024, LJUBLJANA, Slovenia. pp.144-158, ⟨10.1007/978-3-031-72437-4_9⟩. ⟨hal-04710036⟩
48 View
33 Download

Altmetric

Share

More