Hard Math Reasoning Questions with Large Numbers

GSM-Hard contains 1,319 challenging grade-school math problems created by modifying the GSM8K dataset. Reasoning-machines constructed this benchmark by replacing original numbers with larger, less common values to increase difficulty. It was published on Hugging Face in January 2023.

Use Cases

• Benchmarking model performance on solving math problems containing large numerical values.
• Evaluating the robustness of language models to numerical scaling and uncommon number usage in reasoning tasks.
• Training models to perform multi-step arithmetic operations on questions with intentionally difficult number choices.

Strengths

• Contains 1,319 text-based math problems designed for higher difficulty.
• Derived from the established GSM8K dataset, ensuring a foundation of structured reasoning problems.
• Problems are specifically engineered with larger, less common numbers to test generalization.

Limitations

• Relatively small size with only 1,319 examples, limiting its use for training data-hungry models.
• Limited to English language and a specific domain of grade-school math word problems.
• Potential label noise or ambiguity inherited from the source GSM8K dataset modifications.

Provenance

Source

Derived from the GSM8K dataset by reasoning-machines.

Collection Method

Constructed by algorithmically replacing numbers in GSM8K questions with larger, less common values.

Freshness

Last updated on Hugging Face in January 2023.