ALM-Bench: A Multimodal Benchmark for Culturally Diverse and Low-Resource Languages

A benchmark for evaluating Large Multimodal Models (LMMs) on cultural context, local sensitivities, and low-resource language support, integrating visual cues. The dataset was created by MBZUAI and was last updated on February 28, 2025. It is associated with a CVPR 2025 publication.

Use Cases

• Benchmarking model performance on cultural context understanding based on the described focus.
• Evaluating multimodal integration for low-resource languages based on the dataset's stated purpose.
• Testing model sensitivity to local visual and textual cues as mentioned in the description.
• Assessing the global fairness and inclusivity of vision-language models based on the benchmark's goals.

Strengths

• Created by a research institution (MBZUAI) for a CVPR 2025 publication.
• Explicitly designed to address cultural diversity and low-resource language support in AI.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and dataset scale are unknown, which may limit suitability assessment.

Provenance

Source

MBZUAI

Freshness

Last updated 2025-02-28 18:05:33; freshness should be verified.