Dolma3 Dolmino Mix 10B: High-Quality Text Corpus for Language Model Training

A 10-billion-token mixture of high-quality text data, curated by AllenAI and released on January 5, 2026. This dataset was specifically used for the stage 2 annealing training of the Olmo 3 language model. It is a smaller subset of a larger 100-billion-token mixture, intended for research and educational use.

Use Cases

• Training language models based on the described high-quality text mixture.
• Conducting annealing experiments for model training based on the dataset's stated purpose.
• Benchmarking smaller, curated data mixtures against larger corpora based on the described 10B vs. 100B scale.
• Researching data quality and mixture strategies for language model pre-training.

Strengths

• Dataset size is explicitly defined as 10 billion tokens.
• Described as a mixture of high-quality data, curated for a specific training stage.
• Created by AllenAI, a recognized research organization.
• Has a clear, stated purpose for language model annealing training.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and specific file formats are unknown, which may limit suitability assessment.
• The description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

AllenAI

Collection Method

Curated mixture of high-quality data, likely aggregated from multiple sources.

Time Range

null

Freshness

Last updated 2026-01-05 16:25:38.

Geography

null