Nanogpt-tr-v5: Tokenized Turkish Text Corpus for 200M Parameter LM Training

V5 (200M Turkish LM) training data tokenized into three quality tiers totaling approximately 14.94 billion tokens. The dataset includes a 32K vocabulary BPE tokenizer and a validation split. It was created by author 'musabc' and last updated on Hugging Face in May 2026.

Use Cases

• Training a 200M parameter Turkish language model based on the described tokenized data.
• Evaluating model performance on a held-out validation set based on the v5_val.bin file.
• Applying the provided BPE tokenizer to new Turkish text for preprocessing.
• Comparing language model quality across different data quality tiers (Web, Medium, Premium).

Strengths

• Contains approximately 14.94 billion tokens across three distinct quality tiers.
• Includes a dedicated validation set of about 150 million tokens for model evaluation.
• Provides a 32K vocabulary BPE tokenizer trained on the highest-quality data tier.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Aggregated from multiple Turkish text sources including OSCAR, mC4, forums, BellaTurca, Cosmos, CulturaX, Havadis, Cosmopedia, Wiki, Wikisource, theses, academic texts, and FinePDFs.

Collection Method

Data was tokenized for language model training.

Freshness

Last updated 2026-05-16 06:39:36; freshness should be verified.

Geography

Turkey (Turkish language content)