JMMMU: A Japanese Multimodal Understanding Benchmark for Large Models

JMMMU is a Japanese multimodal benchmark designed to evaluate the performance of large multimodal models. The dataset was created by analyzing the existing MMMU benchmark for cultural dependencies and adapting culture-agnostic subjects with native Japanese speakers. The dataset was authored by JMMMU and last updated on the Hugging Face platform in May 2025.

Use Cases

• Benchmarking model performance on Japanese multimodal reasoning tasks based on the described evaluation purpose.
• Fine-tuning models for Japanese-specific cultural and educational content based on the benchmark's adaptation process.
• Analyzing cultural dependencies in multimodal AI systems based on the benchmark's stated design methodology.

Strengths

• Dataset is explicitly designed as a benchmark for evaluating large multimodal models in Japanese.
• Creation involved analysis of cultural dependencies and adaptation by native Japanese speakers.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and file size are unknown, which may limit suitability assessment.

Provenance

Source

JMMMU

Collection Method

Adapted from the MMMU benchmark with analysis and translation by native Japanese speakers.

Freshness

Last updated 2025-05-30 13:10:48; freshness should be verified.