Translations: Multilingual Bitext Corpus from OPUS and MTData

A bitext corpus assembled from upstream releases of mtdata and OPUS projects. The dataset includes source and target sentences with ISO 639-3 language codes and origin sub-corpus identifiers. It was created by author natgillin and last updated on June 2, 2026.

Use Cases

• Train machine translation models based on the parallel sentence pairs.
• Evaluate translation quality across different language pairs based on the source and target language columns.
• Analyze the composition and bias of training data based on the origin sub-corpus field.
• Create multilingual NLP datasets by filtering the corpus using the source_lang and target_lang codes.

Strengths

• Includes explicit source and target language identifiers using the ISO 639-3 standard.
• Tracks the origin of each sentence pair from specific upstream sub-corpora like opus-ccmatrix and opus-nllb.

Limitations

• Row count, total size, and specific language pair coverage are unknown, which may limit suitability assessment.
• Column-level documentation beyond the provided schema is absent; field semantics must be inferred after download.

Provenance

Source

Assembled from upstream mtdata and OPUS releases.

Collection Method

Likely aggregated from multiple open parallel text sources.

Freshness

Last updated 2026-06-02 18:04:16; freshness should be verified.