280 visual generation tasks requiring models to interpret input images and produce annotated outputs like circling specific objects. These tasks target fundamental visual reasoning capabilities typically mastered by three-year-old children, highlighting current gaps in Multimodal Large Language Models (MLLMs).
Use Cases
- Evaluate the spatial reasoning of MLLMs by measuring their ability to accurately circle objects within the generated output image
- Benchmark visual grounding capabilities by comparing model-generated annotations against the ground truth visual tasks
- Analyze the discrepancy between high-level language reasoning and low-level visual perception in multimodal architectures
Strengths
- 280 distinct visual generation tasks focused on basic spatial and object understanding
- Requires models to generate annotated output images, such as circling specific visual elements
- Designed specifically to benchmark Multimodal Large Language Models (MLLMs) against early childhood visual milestones