Til Corpus Exp078: Kazakh-Dominant Pretraining Data

7.21 billion tokens of Kazakh-dominant text, curriculum-ordered so quality rises toward the end. The corpus, created by TilQazyna, is split into a bulk tier of 4.83 billion tokens and a premium anneal tier of 2.38 billion tokens.

Use Cases

• Pretrain language models based on the large-scale Kazakh-dominant text corpus.
• Implement curriculum learning strategies based on the described quality-ordered structure.
• Benchmark model performance on different data quality tiers (q4 vs q5) mentioned in the description.

Strengths

• Contains 7.21 billion tokens, providing substantial volume for model training.
• Uses a curriculum ordering where data quality likely improves, as described.
• Includes a premium 'anneal' tier of 2.38 billion tokens for final training stages.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Freshness should be verified as the last metadata update was 2026-06-17.

Provenance

Source

TilQazyna on Hugging Face.

Collection Method

Likely web-crawled or compiled from various Kazakh text sources, based on its nature as a pretraining corpus.

Freshness

Last updated 2026-06-17 09:24:11.

Geography

Kazakh-dominant, suggesting a focus on Kazakhstan or Kazakh-speaking regions.