VI-Probe: A Visual Illusion Benchmark for Probing VLM Perception

VI-Probe is a benchmark dataset for probing whether vision-language models rely on visual perception or memorized priors. It includes classic visual illusion cases with original images, graded perturbations, matched controls, and variant prompts. The dataset was created by author xxsun and was last updated on June 10, 2026.

Use Cases

• Benchmarking visual perception vs. recall in VLMs based on responses to visual illusions.
• Testing model robustness using graded perturbations of illusion images.
• Evaluating model sensitivity to visual guides and reverse-prompt variants described in the dataset.

Strengths

• Dataset is designed for controlled probing with multiple variants including original images, perturbations, and controls.
• Focuses on a specific, well-defined research question regarding VLM perception.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

huggingface

Collection Method

Created to accompany the paper 'Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions'.

Freshness

Last updated 2026-06-10 20:53:26; freshness should be verified.