ImageEval2026 Task1 AynVQA: Culturally Grounded Arabic Visual Question Answering

Ayn-VQA is a multimodal evaluation dataset for culturally specific Arabic image understanding. It was created by QCRI as part of the ImageEval 2026 Shared Task at ArabicNLP 2026 and was last updated on June 4, -2026. The dataset tests a model's ability to answer questions about images and distinguish grounded descriptions from plausible hallucinations, with tasks offered in both English and Modern Standard Arabic.

Use Cases

• Benchmarking visual question answering models based on culturally specific Arabic imagery.
• Evaluating model robustness against hallucination based on the task of telling grounded descriptions apart from plausible ones.
• Training multimodal models for Arabic language understanding based on the parallel English and Modern Standard Arabic tracks.
• Researching cultural bias and grounding in AI systems based on the dataset's focus on culturally specific images.

Strengths

• Focuses on culturally specific Arabic image understanding, a niche area.
• Offers parallel evaluation tracks in English and Modern Standard Arabic.
• Includes tasks designed to test for hallucination, a key model failure mode.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

QCRI

Collection Method

Created for the ImageEval 2026 Shared Task at ArabicNLP 2026.

Time Range

Likely contemporary to the 2026 conference.

Freshness

Last updated 2026-06-04 08:59:10; freshness should be verified.

Geography

Focus on Arabic cultural contexts.