Multicultural Visual Question Answering Benchmark for VLMs

Vision-Language Models are Confused Tourists evaluates the cultural robustness of VLMs, a largely untested dimension crucial for supporting diverse societies. The dataset was created by author patrickamadeus and was last updated in December 2025. It contains image-text pairs designed to test model stability across diverse cultural inputs.

Use Cases

• Assessing VLM performance on visual question answering tasks with culturally diverse image content.
• Measuring label consistency or prediction shifts when models process inputs from different cultural contexts.
• Benchmarking model robustness against cultural variations present in the image and text data pairs.

Strengths

• Dataset is specifically designed to test a key, under-evaluated dimension of VLM performance: cultural robustness.
• Data is fully usable and available in Parquet format, as noted on the Hugging Face page.

Limitations

• Specific quantitative details like row count, column names, and sample size are not provided in the input.
• The README is noted as incomplete, which may limit initial understanding of the data structure.

Provenance

Source

Hugging Face dataset created by patrickamadeus.

Collection Method

null

Time Range

null

Freshness

Last updated on 2025-12-20.

Geography

Region 'us' is indicated in platform tags, but full spatial coverage is unknown.