PPTP: Procedural Pretraining Pile for Formal and Mathematical Reasoning

Reasoning-core's Procedural Pretraining Pile (PPTP) is a text dataset designed for formal and symbolic reasoning tasks. The data is procedurally generated on CPU and can be scaled to trillion tokens, with adjustable difficulty. The dataset was last updated on Hugging Face on April 28, -2026.

Use Cases

• Pre-training language models on formal reasoning tasks based on the described planning and conjecture entailment categories.
• Supervised fine-tuning for logic-based natural language inference (logic_nli) as mentioned in the description.
• Training models for mathematical computation, such as solving equation systems and arithmetic problems.
• Mid-training models on procedurally generated proof reconstruction tasks.
• Benchmarking model performance on scalable, difficulty-adjusted symbolic reasoning problems.

Strengths

• Data is procedurally generated, allowing for scalable production up to trillion tokens.
• Difficulty of the generated tasks is adjustable via a single control parameter.
• Explicitly designed for multiple training stages: pre-training, mid-training, and supervised fine-tuning.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count, file formats, and dataset size are unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

reasoning-core on Hugging Face

Collection Method

Procedurally generated on CPU

Time Range

null

Freshness

Last updated 2026-04-28 08:45:40; freshness should be verified.

Geography

null