VisRes Bench: 10K-100K Visual Reasoning Tests for VLMs

VisRes Bench contains 10,000 to 100,000 image-text pairs designed to evaluate the visual reasoning of Vision-Language Models (VLMs) in naturalistic settings. Developed by researchers at TII (tiiuae) and updated in March 2026, it isolates visual logic by removing contextual language supervision.

Use Cases

• Benchmarking zero-shot visual question answering (VQA) performance using the image and text pairs
• Quantifying the degree of linguistic prior reliance in multimodal models by testing against reasoning-focused prompts
• Evaluating image-to-text reasoning logic in naturalistic visual environments

Strengths

• Scale of 10,000 to 100,000 records for statistically significant evaluation
• Apache 2.0 license allows for open research and commercial use
• Associated with Arxiv paper 2512.21194 for methodological transparency

Limitations

• Lack of detailed column documentation in the primary metadata
• Potential for high error rates in models that rely heavily on text-based context

Provenance

Source

TII (tiiuae), Arxiv:2512.21194

Freshness

Last updated March 2026.

Geography

United States