SpatialUncertain is a controlled 3D benchmark designed to evaluate Vision-Language Models (VLMs). It contains approximately 6,600 questions for occlusion and 3,700 for perspective ambiguity, testing whether models know when not to answer spatial reasoning questions. The dataset was created by Yuezhangjoslin and serves as a companion to the paper 'Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?', with a last recorded update in June 2026.
Use Cases
- Benchmarking VLM performance on spatial questions under occlusion based on the described occlusion_benchmark
- Evaluating model sensitivity to perspective ambiguity using the perspective_benchmark
- Training or fine-tuning VLMs to recognize and abstain from answering ambiguous spatial queries
- Analyzing failure modes of VLMs in 3D reasoning tasks as outlined in the companion paper
Strengths
- Contains approximately 6,600 questions specifically for occlusion scenarios
- Includes approximately 3,700 questions for perspective ambiguity scenarios
- Designed as a controlled benchmark for a specific research question on VLM uncertainty
Limitations
- Column-level documentation is absent; field semantics must be inferred after download
- Row count is unknown for the full dataset, which may limit suitability assessment
- Description metadata is limited; actual data quality requires manual inspection after download
Provenance
- Source
- Yuezhangjoslin on Hugging Face
- Collection Method
- Created as a companion to a research paper; likely synthetically generated or curated from 3D environments.
- Freshness
- Last updated 2026-06-01 00:09:18