Vietnamese News Corpus From Binhvq

Subset of the ROOTS corpus, consisting of Vietnamese news text. It represents 1.06% of the total ROOTS corpus and 77.45% of the Vietnamese language portion within it. The data has undergone processing steps including document deduplication and filtering of empty documents.

Use Cases

• Train a language model on Vietnamese text data for natural language processing tasks.
• Analyze text patterns and document structures within a Vietnamese news corpus.
• Apply text deduplication techniques to the processed corpus to study data quality.

Strengths

• Represents 77.45% of the Vietnamese language content within the larger ROOTS corpus.
• Has undergone specific processing steps including dedup_document and filter_remove_empty_docs.

Limitations

• Specific details such as row count, column structure, and file formats are unknown.
• The dataset's size and temporal coverage are not specified in the provided input.

Provenance

Source

https://github.com/binhvq/news-corpus

Collection Method

Part of the BigScience ROOTS corpus processing; filters applied to Vietnamese text.

Time Range

null

Freshness

null

Geography

South-eastern Asia, Vietnam