Retrieval Augmented Generation for Historical Newspapers
Résumé
Nowadays, the accessibility and long-term preservation of historical records are significantly impacted by the sharp increase in the digitization of these archives. This shift creates new opportunities for researchers and students in multiple disciplines to broaden their knowledge or conduct multidisciplinary research. However, given the vast amount of data that needs to be analyzed, using this knowledge is not easy. Different natural language processing tasks such as named entity recognition, entity linking, and article separation have been developed to make this accessibility easier for the public by extracting information and structuring data. However, historical newspaper article aggregation is still unexplored. In this work, we demonstrate the potential of the retrieval-augmented generation framework that integrates large language models (LLMs), a semantic retrieval module, and knowledge bases to create a system capable of aggregating historical newspaper articles. In addition, we propose a set of metrics that permit evaluating these generative systems without requiring any ground truth. The results of our proposed RAG pipeline are promising at this early stage of the system. They show that semantic retrieval with the help of reranking and additional information (NER) reduces the impact of OCR errors and query misspellings.
Mots clés
CCS Concepts Information systems → Digital libraries and archives Retrieval models and ranking • Applied computing → Arts and humanities Digital libraries and archives • Computing methodologies → Natural language generation Natural language processing Digital Humanities
Retrieval-Augmented Generation
Large Language Models
Historical Newspapers
CCS Concepts
Information systems → Digital libraries and archives
Retrieval models and ranking
• Applied computing → Arts and humanities
Digital libraries and archives
• Computing methodologies → Natural language generation
Natural language processing Digital Humanities
Fichier principal
JCDL2024_12October_4pages_ref_RAG4historicalnewspapers.pdf (865.43 Ko)
Télécharger le fichier
Origine | Fichiers produits par l'(les) auteur(s) |
---|---|
Licence |