Oasst1 Shan Translation: Machine-Translated Conversational AI Data

A machine-translated version of the OpenAssistant/oasst1 dataset into the Shan language, created using the facebook/nllb-200-3.3B model. The dataset's author is NorHsangPha, and it was last updated on July 11, 2024. The description notes that the data quality has not been verified by human reviewers.

Use Cases

• Fine-tune language models for Shan based on machine-translated conversational data.
• Benchmark machine translation quality for low-resource languages using a known conversational corpus.
• Study the propagation of errors in automatically translated training data for AI assistants.

Strengths

• Derived from the established OpenAssistant/oasst1 dataset, which provides a known conversational structure.
• Specifically targets the Shan language, a low-resource language for AI/ML applications.
• Uses a large, publicly documented translation model (facebook/nllb-200-3.3B) for creation.

Limitations

• Data quality is explicitly noted as unchecked by humans and may be low.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for large-scale training.

Provenance

Source

OpenAssistant/oasst1 dataset, translated via facebook/nllb-200-3.3B model.

Collection Method

Automatic machine translation.

Freshness

Last updated 2024-07-11 04:48:37