EXAMS: 24,000 Multilingual High School Questions Across 16 Languages

EXAMS provides over 24,000 high school examination questions across 16 languages and 24 school subjects. Developed by mhardalov, this benchmark supports multilingual and cross-lingual question answering research within the Natural and Social Sciences. It covers 8 distinct language families, making it a diverse resource for evaluating language model performance in academic contexts.

Use Cases

• Benchmarking multilingual LLMs using the 24,000+ multiple-choice questions
• Evaluating cross-lingual knowledge transfer across 8 different language families
• Analyzing model accuracy across 24 specific academic subjects like biology or history

Strengths

• 24,000+ high-quality exam questions
• Covers 16 languages across 8 language families
• Includes 24 distinct school subjects from Natural and Social Sciences

Limitations

• Restricted to multiple-choice format, which may not capture full reasoning capabilities
• Specific column headers and data schema are not explicitly defined in the provided summary
• Potential geographic bias towards the 16 included languages

Provenance

Source

mhardalov

Collection Method

Sourced from high school examinations

Freshness

Last updated in February 2024; reflects high school curricula available at the time of collection.

Geography

Multilingual (16 languages including Arabic, Bulgarian, Italian, and others)