OlympiadBench: 1,000+ Bilingual Multimodal Math and Physics Problems

OlympiadBench contains between 1,000 and 10,000 bilingual scientific problems in mathematics and physics, designed for evaluating AGI reasoning. Created by Hothan and published at ACL 2024, the dataset includes both text-based and multimodal questions in English and Chinese.

Use Cases

• Benchmarking Visual Question Answering (VQA) models on complex physics diagrams
• Evaluating bilingual mathematical reasoning in English and Chinese
• Testing zero-shot performance of LLMs on competition-level scientific problems

Strengths

• Bilingual support for English and Chinese languages
• Multimodal content including both text and images
• Olympiad-level difficulty for testing advanced reasoning

Limitations

• Small sample size between 1,000 and 10,000 records
• Domain restricted to mathematics and physics

Provenance

Source

ACL 2024 (arXiv:2402.14008)

Freshness

Last updated June 2025; includes fixes for image display issues.