MMMU Distribution Simulation: Synthetic Multimodal Multiple-Choice Questions

A synthetic dataset of 234 question-answer pairs designed to mirror the distribution of the MMMU-Pro benchmark. It contains 78 unique questions across 30 academic subjects, each presented in three different visual and textual formats. The dataset was created by YiYang109 and last updated on Hugging Face in May 2026.

Use Cases

• Benchmarking model robustness based on question presentation variants (4-option vs. 10-option MCQ).
• Training models on a diverse set of academic subjects mentioned in the description.
• Studying the impact of visual presentation on question difficulty using the different image formats.
• Developing synthetic data generation techniques for multimodal evaluation tasks.

Strengths

• Contains 234 total data points (78 questions × 3 variants).
• Covers a wide range of 30 academic subjects.
• Provides controlled variation with three distinct presentation formats per question.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known but the total data scale is small.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Hugging Face user YiYang109.

Collection Method

Synthetically generated to mirror the MMMU-Pro benchmark distribution.

Freshness

Last updated 2026-05-21 01:44:01; freshness should be verified.