ArxivMathGradingBench: 35 Annotated Mathematics Papers with Errors

ArxivMathGradingBench is a benchmark of 35 arXiv mathematics papers, each containing an error later corrected by the authors. The dataset provides paper metadata and annotations for the location of the errors. It was introduced by LukeBailey181Pub to evaluate large language model proof verifiers on research-level mathematics.

Use Cases

• Benchmarking LLM proof verification systems based on annotated error locations.
• Studying common error patterns in mathematical research papers based on the provided annotations.
• Training models to identify logical inconsistencies in formal or pseudo-formal mathematics.

Strengths

• Contains 35 real research-level mathematics papers from arXiv.
• Each paper is annotated with the specific location of an error corrected by the original authors.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset provides metadata and annotations; the paper PDFs must be downloaded separately.

Provenance

Source

LukeBailey181Pub on Hugging Face.

Collection Method

Curated from arXiv mathematics papers where authors published a correction.

Freshness

Last updated 2026-06-11 20:30:53; freshness should be verified.