UltraVR: Diagnostic Benchmark for Ultra-Resolution Visual Reasoning

UltraVR is a diagnostic benchmark for evidence-grounded visual reasoning over ultra-resolution images, spanning four high-value scenarios: CCTV surveillance, remote sensing, pathology, and industrial anomaly detection. Created by Gexin Huang and hosted on Harvard Dataverse, it was last updated in May 2026. The dataset includes structured ground-truth chains of thought with step-level questions and operation labels for process-level diagnosis.

Use Cases

• Benchmarking model performance on ultra-resolution visual reasoning based on the four diagnostic domains.
• Diagnosing failure modes in evidence acquisition and local perception based on the structured operation-level annotations.
• Developing new prompting or cropping strategies to improve VLM performance on high-resolution images.
• Analyzing the gap between visual fact retrieval and downstream inference based on the chain-of-thought annotations.

Strengths

• Includes structured ground-truth chains of thought with step-level questions, intermediate answers, and operation labels.
• Spans four complementary high-value domains: CCTV surveillance, remote sensing, pathology, and industrial anomaly detection.
• Benchmark results reveal a substantial performance gap, with the strongest model reaching only about 44% accuracy.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and file formats are unknown, which may limit suitability assessment.
• Data may reflect bias inherent to the specific diagnostic scenarios constructed for the benchmark.

Provenance

Source

Harvard Dataverse

Collection Method

Constructed as a diagnostic benchmark for evaluating vision-language models.

Time Range

null

Freshness

Last updated 2026-05-06 21:23:01; freshness should be verified.

Geography

null