GSM8K-Platinum: Revised Grade School Math Word Problems

GSM8K-Platinum is a revised version of the full GSM8K test set, a dataset of grade school math word problems. It was created by running frontier models on each example and manually examining cases with model disagreement to provide a more accurate assessment of mathematical reasoning capabilities. The dataset was authored by madrylab and last updated on March 11, 2025.

Use Cases

• Benchmarking large language models on mathematical problem-solving based on the revised test set
• Training models for improved step-by-step reasoning on word problems
• Studying model failure modes and inconsistencies in mathematical reasoning

Strengths

• Dataset is a revised version of a known benchmark, suggesting curation for accuracy
• Manual examination of examples with model disagreement indicates a quality control process

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download
• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment

Provenance

Source

madrylab

Collection Method

Revised by running frontier models on the GSM8K test set and manually examining examples with disagreement.

Freshness

Last updated 2025-03-11 14:48:29; freshness should be verified