EmbSpatial-Bench: 3,640 QA Pairs for Egocentric Spatial Understanding

FlagEval provides a benchmark for evaluating embodied spatial understanding in large vision-language models. The dataset contains 3,640 question-answer pairs automatically derived from embodied scenes, covering six spatial relationships from an egocentric perspective. It was adapted from an original image-format dataset by Phineas476 and last updated on 2025-04-21.

Use Cases

• Benchmarking model performance on embodied spatial understanding tasks based on the described QA pairs.
• Training models to recognize six specific spatial relationships from an egocentric perspective.
• Evaluating the generalization of vision-language models on automatically generated scene-based questions.

Strengths

• Contains 3,640 QA pairs, providing a substantial benchmark size.
• Covers six distinct spatial relationships, offering structured evaluation categories.
• Converted from an original image format to a more accessible structure for ease of use.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Adapted from Phineas476/EmbSpatial-Bench.

Collection Method

Automatically derived from embodied scenes.

Freshness

Last updated 2025-04-21 04:19:43; freshness should be verified.