SMART: A Fine-Grained Benchmark for LLM Mathematical Reasoning

SMART is a benchmark for evaluating large language models on mathematical reasoning from a human cognitive process perspective. The benchmark decomposes problem-solving into four cognitive dimensions: Semantic Understanding, Mathematical Reasoning, Arithmetic Computation, and one other dimension. It was created by ewdfd and last updated on Hugging Face in April 2026.

Use Cases

• Benchmarking LLM performance on mathematical problem-solving based on the described cognitive dimensions.
• Analyzing model failure modes in semantic understanding of math word problems.
• Evaluating the arithmetic computation accuracy of language models.
• Studying the alignment of LLM reasoning steps with human cognitive processes inspired by Pólya’s theory.

Strengths

• Benchmark is designed from a fine-grained, human cognitive process perspective.
• Evaluation goes beyond final answers to assess multiple reasoning dimensions.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

ewdfd on Hugging Face

Collection Method

Likely created as an evaluation benchmark for LLMs.

Freshness

Last updated 2026-04-21 07:13:57; freshness should be verified.