m-WildVision: Multilingual Vision-Language Benchmark in 23 Languages

CohereLabs created a multilingual evaluation set by translating 500 challenging user queries from the original English-only WildVision test set. The dataset covers 23 languages and is intended for automatic LLM judging of vision-language models. It was last updated on March 6, 2026.

Use Cases

• Benchmarking multilingual vision-language model performance based on translated prompts.
• Evaluating model robustness across 23 languages based on the dataset's multilingual scope.
• Assessing model response quality on challenging user queries sourced from the WildVision-Arena platform.

Strengths

• Covers 23 languages, providing a broad multilingual scope.
• Based on 500 challenging user queries designed for robust evaluation.
• Created by translating prompts from a published benchmark (Lu et al., 2024).

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect translation or source bias inherent to the original English prompts.

Provenance

Source

CohereLabs

Collection Method

Created by translating prompts from the original English-only WildVision (vision_bench_0617) test set.

Time Range

2024

Freshness

Last updated 2026-03-06 08:37:28

Geography

Multilingual, covering 23 languages.