Visual Reasoning Benchmark Results for Two Image Generation Models

songyiren's dataset contains full generation and evaluation results from running the Visual Reasoning Benchmark Suite v3 on two image generation models. The benchmark includes 1,705 items across 10 tasks such as figure completion, spatial generation, mazes, Sudoku, board games, matchsticks, orthographic views, and visual math proofs. Results were judged by the gemini-3.1-pro-preview model and the dataset was last updated on July 8, 2026.

Use Cases

• Benchmarking image generation models based on performance across 10 visual reasoning tasks.
• Analyzing model strengths and weaknesses based on scores for tasks like maze solving and visual math proofs.
• Comparing the performance of the two specified models, including gemini-3.1-flash-image-preview, on the benchmark suite.

Strengths

• Benchmark results are based on 1,705 evaluation items, providing a substantial test set.
• Evaluation covers 10 distinct visual reasoning tasks, allowing for multi-faceted model assessment.
• Results were judged by a specified model (gemini-3.1-pro-preview), providing a consistent evaluation methodology.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for specific analysis needs.
• Description metadata is limited; actual data quality and structure require manual inspection after download.

Provenance

Source

songyiren via Hugging Face

Collection Method

Results from running the Visual Reasoning Benchmark Suite v3 and judging outputs with gemini-3.1-pro-preview.

Freshness

Last updated 2026-07-08 11:36:40; freshness should be verified.