Principia Bench: STEM Problem Statements for Mathematical Object Derivation

Principia Bench is a benchmark for evaluating language models on deriving mathematical objects from STEM-related problem statements. It contains problem statements and ground-truth answers filtered from four source benchmarks: RealMath, SuperGPQA, Physics, and ARB. The dataset is authored by Facebook.

Use Cases

• Benchmark language models on their ability to derive a ground-truth answer from a STEM-related problem statement.
• Evaluate model performance on problem statements filtered from the four source benchmarks (RealMath, SuperGPQA, Physics, ARB).
• Assess language models on tasks where the answer is expressed as a mathematical object.

Strengths

• Problem statements are sourced from four established benchmarks: RealMath, SuperGPQA, Physics, and ARB.
• Instances are filtered to require answers expressed as mathematical objects.
• Dataset is designed for a specific evaluation task in mathematical reasoning.

Limitations

• The dataset size, row count, and column structure are unknown.
• Sample data and file formats are unavailable for inspection.
• The specific filtering criteria for problem statements are not detailed in the provided input.

Provenance

Source

facebook (Meta AI) on Hugging Face.

Collection Method

Problem statements are drawn and filtered from four existing benchmarks: RealMath, SuperGPQA, Physics, and ARB.

Freshness

Last updated on 2025-12-18.