Spatial Mental Modeling Benchmark with Limited Views

MindCube is a benchmark for evaluating Vision Language Models' ability to form spatial mental models from limited visual information. It contains 21,154 questions across 3,268 images, created by MLL-Lab. The dataset was last updated in November 2025.

Use Cases

• Benchmarking VLM performance on cognitive mapping tasks using the dataset's 3,268 images and associated position questions.
• Evaluating perspective-taking capabilities by testing model predictions on orientation questions for objects in scenes.
• Assessing mental simulation for 'what-if' movement dynamics through the dataset's specific question types.
• Analyzing gaps in spatial reasoning by comparing model performance across the three defined capability categories.

Strengths

• Contains 21,154 questions for evaluation.
• Built from 3,268 distinct images.
• Structured to assess three specific spatial capabilities: cognitive mapping, perspective-taking, and mental simulation.

Limitations

• Unknown total number of unique objects or scenes underlying the 3,268 images.
• Potential for unknown biases in image selection and question design.
• Dataset size and scope may be limited compared to general vision-language benchmarks.

Provenance

Source

MLL-Lab

Collection Method

Constructed as a benchmark; specific data gathering methodology is not detailed in the provided description.

Freshness

Last updated in November 2025.