Portuguese News Articles from CommonCrawl 2016 to 2024

Over 11 million Portuguese-language news articles curated from CommonCrawl News, spanning from 2016 to June 2024. The dataset has been cleaned, deduplicated, and language-filtered by author eduagarcia.

Use Cases

• Train a Portuguese language model on the cleaned and deduplicated article text corpus.
• Analyze temporal trends in news topics and coverage across the 2016-2024 date range.
• Perform cross-lingual analysis by comparing this Portuguese subset with other language versions of the CC-News collection.

Strengths

• Contains over 11 million articles, providing a large-scale text corpus.
• Covers a broad temporal range from 2016 to June 2024.
• Data has undergone cleaning, deduplication, and language filtering processes.

Limitations

• Specific column structure, file formats, and exact row count are not provided in the input.
• Potential bias towards news sources and topics captured by the CommonCrawl web crawl.

Provenance

Source

CommonCrawl News via stanford-oval/ccnews.

Collection Method

Web crawl, followed by cleaning, deduplication, and language detection/filtering.

Time Range

2016 to June 2024.

Freshness

Data updated to June 2024; dataset metadata last updated 2025-04 28.

Geography

null