SIB-200: Multilingual Topic Classification Dataset Covering 205 Languages

SIB-200 is a topic classification dataset based on the Flores-200 multilingual benchmark. It is the largest publicly available dataset of its kind, covering 205 languages and dialects. The dataset was created by Davlan and last updated on February 19, 2024.

Use Cases

• Train multilingual topic classifiers based on Wikipedia sentences.
• Benchmark model performance across 205 languages for topic classification.
• Analyze linguistic or cultural bias in topic classification models using the broad language coverage.

Strengths

• Covers 205 languages and dialects, making it the largest public dataset of its kind.
• Provides train, validation, and test splits for all 205 languages.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for large-scale training.

Provenance

Source

Based on the Flores-200 multilingual benchmark.

Freshness

Last updated 2024-02-19 16:34:23.