VideoThinkBench: Multimodal Reasoning via Video Generation for CVPR 2026

VideoThinkBench contains between 1,000 and 10,000 samples designed to evaluate video generation as a multimodal reasoning paradigm, released by the OpenMOSS-Team in late 2025. The dataset supports tasks across text, image, and video modalities and was featured at CVPR 2026. It includes a specific 500-sample 'minitest' subset for rapid benchmarking of visual reasoning capabilities.

Use Cases

• Benchmarking text-to-video models on logical reasoning tasks
• Evaluating visual question answering (VQA) through video synthesis
• Training image-to-video models to follow complex multimodal instructions

Strengths

• Peer-reviewed at CVPR 2026
• Includes a 500-sample minitest for rapid evaluation
• Supports multiple task categories including Image-to-Video and Visual Question Answering

Limitations

• Small sample size (1K-10K records) compared to general-purpose video datasets
• Evaluation requires video generation capabilities which are computationally expensive

Provenance

Source

OpenMOSS-Team, 'Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm' (CVPR 2026)

Freshness

Last updated March 2026 following its acceptance to CVPR 2026.