MathNet-Retrieve: 15,000 Olympiad Problems for Mathematical Retrieval Benchmarking

MathNet-Retrieve is a benchmark for math-aware information retrieval, created by ShadenA and last updated in June 2026. It contains 15,000 queries, each with a mathematically equivalent reformulation target provided at three difficulty tiers. The benchmark is designed to test retrieval systems on problems where the surface form is disguised while the underlying mathematical structure is preserved.

Use Cases

• Benchmarking math-aware retrieval systems based on the described three-tier difficulty structure.
• Training models to identify semantically equivalent mathematical problems based on the described 'hard negatives' corpus.
• Studying the difficulty of retrieving reformulated problems as surface forms are increasingly disguised.

Strengths

• Contains 15,000 queries for evaluation.
• Provides one gold-equivalent target per query for three distinct difficulty tiers (easy, medium, hard).
• Designed with a corpus of hard negatives to increase retrieval challenge.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

ShadenA on Hugging Face, associated with the MathNet project.

Collection Method

Likely compiled from Olympiad problems and their reformulations, as described in the associated paper.

Freshness

Last updated 2026-06-16 05:19:06; freshness should be verified.