Lucie Training Dataset: Multilingual Text for LLM Pretraining

A curated collection of text data in English, French, German, Spanish, and Italian, culled from sources including web data, video subtitles, academic papers, digital books, newspapers, and magazines. The dataset was used to pretrain the Lucie-7B foundation LLM and also contains samples of diverse programming languages. It was authored by OpenLLM-France and last updated on the Hugging Face platform in May 2025.

Use Cases

• Pretrain multilingual language models based on text from web data, books, and academic papers.
• Fine-tune models for specific languages based on curated samples in English, French, German, Spanish, and Italian.
• Train or evaluate models on OCR-processed text based on the inclusion of scanned documents.
• Develop code generation or understanding models based on the inclusion of diverse programming language samples.

Strengths

• Multilingual content covering five major European languages.
• Diverse text sources including web data, subtitles, academic papers, books, and magazines.
• Includes text processed by Optical Character Recognition (OCR), which may capture scanned documents.
• Contains samples of diverse programming languages alongside natural language text.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and total dataset size are unknown, which may limit suitability assessment.
• Data may reflect geographic, temporal, or source bias inherent to the original collection methods.

Provenance

Source

OpenLLM-France via Hugging Face.

Collection Method

Curated collection from web data, video subtitles, academic papers, digital books, newspapers, and magazines, with some OCR processing.

Freshness

Last updated 2025-05-27 12:27:16; freshness should be verified.