Sage Pretrain Corpus: Korean and English Text for LLM Training

A license-audited Korean and English text corpus for the Sage Korean LLM project. The dataset includes over 46 million documents from sources like FineWeb-2 Korean, each with license and source metadata. It was created by seongchaeae and last updated in May 2026.

Use Cases

• Pretraining Korean language models based on the large-scale Korean text content.
• Training multilingual models using the combined Korean and English text data.
• Studying license compliance for AI training data based on the provided source and license fields.
• Filtering or subsetting training data by source or license type for specific project requirements.

Strengths

• Contains over 46 million text documents from the FineWeb-2 Korean source alone.
• Each record includes license and source metadata, facilitating compliance checks.
• Explicitly designed for LLM pretraining, indicating a focus on data quality for that task.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count for the full corpus is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Aggregated from multiple sources including FineWeb-2 Korean (HuggingFaceFW/fineweb-2 kor_Hang).

Collection Method

Curated and license-audited for the Sage Korean LLM project.

Freshness

Last updated 2026-05-21 13:42:39; freshness should be verified.