Dolma3 Mix 6T: 6-Trillion-Token Mixture for Olmo 3 7B Reproduction

Allen Institute for AI released this 6-trillion-token text mixture in January 2026 specifically for the reproduction of the Olmo 3 7B model. It contains a massive collection of web and scientific content, including data processed via olmOCR. The dataset is a specialized version of the primary Dolma 3 mix, modified for specific model alignment.

Use Cases

• Reproducing the Olmo 3 7B model weights using the specific 1025-7B data mixture
• Analyzing the distribution of scientific versus web content using the text field
• Investigating the effects of redacted data points on model loss by identifying '[REMOVED]' markers

Strengths

• 6-trillion-token scale for high-capacity language model training
• Directly mapped to the Olmo 3 7B training configuration
• Includes high-fidelity olmOCR-processed scientific data

Limitations

• Redacted content marked as '[REMOVED]' in the text field limits exact bit-for-bit reproduction
• Narrowly scoped for reproduction rather than general-purpose model development

Provenance

Source

Allen Institute for AI (allenai)

Collection Method

Web scraping and OCR processing of scientific publications via olmOCR

Freshness

Last updated January 15, 2026.

Geography

United States