RealMath-Eval: Benchmark for Evaluating LLM Judges on Human Mathematical Reasoning

RealMath-Eval is a benchmark for evaluating large language model judges on authentic human mathematical reasoning. The dataset was created by RicharMd and is associated with the paper 'RealMath-Eval: The Evaluation Gap in Judging Human Mathematical Reasoning'. The dataset was last updated on Hugging Face on June 10, 2026.

Use Cases

• Evaluating LLM performance on judging human mathematical reasoning based on the benchmark's tasks.
• Benchmarking new models against established baselines for mathematical judgment.
• Analyzing the gap between model and human evaluation in mathematical problem-solving.

Strengths

• Dataset is explicitly designed as a benchmark for evaluating LLM judges, providing a clear purpose.
• Associated with a named arXiv preprint (arXiv:2606.10254) and a GitHub repository, suggesting academic rigor.
• Last update timestamp is provided (2026-06-10 02:09:12).

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and dataset size are unknown, which may limit suitability assessment.
• The full description is hosted externally, requiring a click-through for complete details.

Provenance

Source

RicharMd (author) via Hugging Face.

Collection Method

Likely contains processed benchmark data and derived artifacts from the associated research paper.

Freshness

Last updated 2026-06-10 02:09:12; freshness should be verified.