Wikinews Articles in Tamil and Indic Languages

Encompassing a filtered subset of Wikinews articles, part of the ROOTS corpus. It includes text in Tamil and other Indic languages, with specific percentages of content per language provided in the description. The exact row count and column structure are unknown.

Use Cases

• Analyze text content and language distribution across the provided language percentages (e.g., ar, pt, en, fr, indic-ta, zh, es, ca).
• Apply NLP techniques for language identification or text classification on the filtered Wikinews articles.
• Study the effects of filters like 'filter_wiki_user_titles' and 'filter_wiki_non_text_type' on multilingual news text corpora.

Strengths

• Part of the large-scale, multilingual ROOTS corpus curated by BigScience.
• Provides specific language composition percentages, such as 0.2119% for indic-ta and 0.3036% for Portuguese (pt).
• Documents specific processing steps applied, including 'dedup_document' and language-specific filters.

Limitations

• The dataset size, row count, and column schema are not specified in the input.
• The temporal coverage and geographic scope of the Wikinews articles are unknown.
• The sample data and file formats are unavailable for inspection.

Provenance

Source

Wikinews, processed by BigScience.

Collection Method

Filtered and deduplicated from the original Wikinews source, with language-specific processing steps applied.

Time Range

null

Freshness

null

Geography

null