SeaExam is a dataset for evaluating Large Language Models (LLMs) on a diverse set of Southeast Asian languages including English, Chinese, Indonesian, Thai, and Vietnamese. The dataset was created by SeaLLMs and last updated on May 31, 2024. Its goal is to enable fair and consistent comparisons across LLMs while mitigating data contamination risks.
Use Cases
- Benchmarking LLM performance based on the described multilingual evaluation tasks.
- Assessing model fairness and consistency across languages as mentioned in the description.
- Mitigating data contamination risks in LLM evaluation as a stated goal of the dataset.
Strengths
- Explicitly designed for fair and consistent comparison across different LLMs.
- Covers a diverse set of languages including English, Chinese, Indonesian, Thai, and Vietnamese.
- Aims to mitigate the risk of data contamination, a key concern in LLM evaluation.
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download.
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count and file formats are unknown, which may limit suitability assessment.
Provenance
- Source
- SeaLLMs
- Collection Method
- Likely constructed for evaluation purposes; method details require checking the full description.
- Time Range
- null
- Freshness
- Last updated 2024-05-31 09:27:49.
- Geography
- Southeast Asia (languages include English, Chinese, Indonesian, Thai, Vietnamese)