Sutra 100M: 70,435 Synthetic Pedagogical Entries for LLM Pretraining

Sutra 100M contains 70,435 synthetic educational entries totaling approximately 100 million tokens, released by codelion in March 2026. The dataset utilizes the Sutra framework to produce structured pedagogical content specifically designed for language model pretraining across multiple domains.

Use Cases

• Pretraining language models using the 100 million tokens of structured pedagogical text
• Evaluating the learning efficiency of models on synthetic educational content
• Fine-tuning text generation models to produce content following specific pedagogical structures

Strengths

• 70,435 educational entries
• Approximately 100 million tokens
• Apache 2.0 license
• Structured pedagogical formatting

Limitations

• Synthetic generation may result in repetitive linguistic patterns or lack the stylistic variety of human-authored textbooks
• Potential for hallucinations inherent in synthetic data generation frameworks

Provenance

Source

codelion via the Sutra framework

Collection Method

synthetic

Freshness

Last updated March 2026.