FineMath Educational Content from CommonCrawl

FineMath is a dataset of mathematical educational content filtered from the CommonCrawl web corpus. It contains 34 billion tokens in one version and 54 billion tokens in another, curated by HuggingFaceTB using a classifier trained with Llama-3.1-70B-Instruct annotations. The dataset was last updated in February 2025.

Use Cases

• Fine-tune language models on mathematical explanations and step-by-step problem-solving text.
• Train classifiers to identify high-quality educational content within web-scale text corpora.
• Benchmark model performance on mathematical comprehension using filtered educational passages.
• Analyze the structure and linguistic patterns of instructional mathematics content.

Strengths

• Contains 34 billion tokens of filtered mathematical content.
• Includes an expanded 54 billion token version with additional InfiMM-WebMath data.
• Content is filtered for educational quality using a classifier trained by a large language model.

Limitations

• The specific row count, column structure, and sample data are unknown.
• The dataset's geographic and temporal coverage from CommonCrawl is unspecified.
• Potential class imbalance or bias towards web-available mathematical content styles.

Provenance

Source

CommonCrawl web corpus.

Collection Method

Filtered using a mathematical content classifier trained with annotations from Llama-3.1-70B-Instruct.

Time Range

null

Freshness

Last updated 2025-02-06.

Geography

null