MAKIEVAL: Multilingual Cultural Awareness Evaluation for LLMs

MAKIEVAL is a large-scale multilingual benchmark designed to study how Large Language Models represent cultural knowledge across languages, countries, and domains. The dataset consists of model-generated responses together with automatically collected annotations. It was created by Raoyuan and last updated on June 2, 2026.

Use Cases

• Benchmarking LLM cultural knowledge across different languages based on the multilingual framework.
• Evaluating model-generated responses for cultural bias or awareness.
• Studying the variation in cultural knowledge representation across different countries and domains.

Strengths

• Designed as a large-scale multilingual benchmark.
• Focuses on cultural knowledge across languages, countries, and domains.
• Includes model-generated responses with automatic annotations.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Hugging Face dataset by author Raoyuan, accompanying an academic paper.

Collection Method

Likely constructed using an automatic Wikidata-based framework as described in the paper title.

Freshness

Last updated 2026-06-02 16:58:32; freshness should be verified.

Geography

Multinational, covering multiple countries as implied by the cultural focus.