Portuguese Wikinews Articles From ROOTS Corpus

A collection of a subset of the ROOTS corpus focused on Portuguese-language Wikinews articles. It represents 0.3036% of the total Portuguese (pt) content within the larger corpus. The data has been processed by BigScience, applying filters such as 'filter_wiki_user_titles' and 'filter_wiki_non_text_type'.

Use Cases

• Analyze text patterns and journalistic style in Portuguese-language news articles from the Wikinews source.
• Train language models on filtered Portuguese text data that has undergone deduplication and content-type filtering.
• Study the distribution and characteristics of multilingual content, as the dataset includes metadata on language percentages like '0.3036 % of pt'.

Strengths

• Part of the large-scale, multilingual ROOTS corpus curated by BigScience.
• Includes specific filtering steps like 'filter_wiki_non_text_type' to improve data quality.
• Provides language composition statistics, showing it constitutes 0.3036% of the Portuguese content in ROOTS.

Limitations

• The exact number of rows, columns, and sample data are unavailable, limiting preliminary analysis.
• The dataset's size and specific file formats are unknown, complicating integration planning.
• Data is from 2022 and may not reflect current events or language use.

Provenance

Source

Wikinews, processed as part of the BigScience ROOTS corpus.

Collection Method

Filtered and processed from the Wikinews source, including steps like 'dedup_document'.

Freshness

Last updated on 2022-12-12.