EXAMS-V: A Multimodal Multilingual Exam Benchmark for Vision-Language Models

Introduced in 2024, EXAMS-V is a benchmark dataset designed to evaluate vision-language models. It features multi-discipline, multimodal, and multilingual exam questions. The dataset was created by researchers from Mohamed bin Zayed University of Artificial Intelligence and Sofia University.

Use Cases

• Benchmarking vision-language model performance based on multimodal exam questions
• Evaluating model reasoning across multiple disciplines based on the described benchmark structure
• Assessing AI model capabilities in multilingual contexts based on the dataset's multilingual nature
• Training or fine-tuning multimodal models on a diverse set of exam-style problems

Strengths

• Designed as a challenging benchmark for evaluating vision-language models
• Covers multiple disciplines and languages according to the description

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Last updated 2024-05-07 12:19:13; freshness should be verified

Provenance

Source

Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi & Sofia University

Freshness

Last updated 2024-05-07 12:19:13