U-MATH: 1,100 University-Level Math Problems for LLM Evaluation

1,100 unpublished university-level math problems sourced from real teaching materials, designed to evaluate the mathematical reasoning of Large Language Models. The benchmark is balanced across six core topics and includes 20% multimodal problems with visual elements. It was created by Toloka and last updated on 2026-01-30.

Use Cases

• Benchmarking LLM performance on university-level mathematics based on the 1,100 problem corpus.
• Evaluating multimodal reasoning in LLMs based on problems involving graphs and diagrams.
• Analyzing model strengths and weaknesses across six core mathematical topics as described in the benchmark.
• Fine-tuning LLMs for advanced mathematical problem-solving using real teaching materials.

Strengths

• Contains 1,100 unpublished university-level problems, providing a substantial test set.
• Balanced across six core mathematical topics for structured evaluation.
• Includes 20% multimodal problems with visual elements like graphs and diagrams.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Last updated 2026-01-30 19:21:41; freshness should be verified.

Provenance

Source

Toloka via Hugging Face.

Collection Method

Sourced from real university teaching materials.

Freshness

Last updated 2026-01-30 19:21:41.