Video-MME Logical: A Benchmark for Video-Based Multimodal Logical Reasoning

3,750 samples for evaluating multimodal logical reasoning in videos. The benchmark archive contains videos, images, and a JSON test set with fields for evaluation. It was created by marcuskwan and last updated on 2026-06-29.

Use Cases

• Benchmarking video-based logical reasoning models based on the provided question-answer pairs.
• Training multimodal models for visual question answering using the paired video, image, and text data.
• Analyzing model performance across different difficulty levels based on the 'difficulty' field.
• Developing evaluation frameworks for AI systems that require understanding of temporal and visual logic.

Strengths

• Contains 3,750 video samples, providing a substantial test set.
• Includes 900 images, offering complementary visual data.
• Structured for evaluation with specific fields like 'id', 'question', and 'answer'.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Last updated 2026-06-29 09:02:18; freshness should be verified.

Provenance

Source

huggingface

Freshness

Last updated 2026-06-29 09:02:18.