News Article URLs from C4 Dataset

C4 Newslike Url Only contains a curated list of URLs for news-like web pages extracted from the C4 dataset. The dataset was created by bs-modeling-metadata and last updated in September 2021. It focuses on English-language content from US-region sources.

Use Cases

• Filtering web-scale text corpora for high-quality news articles using URL metadata.
• Training models to distinguish news-like text from other web content based on source URLs.
• Building domain-specific datasets for media analysis by sampling from this URL list.
• Studying the distribution and characteristics of news sources within large web corpora.

Strengths

• Derived from the large-scale C4 dataset, providing a substantial source pool.
• Focuses on a specific, high-quality text modality (news articles).
• Includes geographic metadata (Region: US) for regional analysis.

Limitations

• Contains only URLs, not the actual article text, requiring separate fetching.
• Data is from 2021, so linked content may be stale or unavailable.
• Specific row count and sample size are unknown.

Provenance

Source

Extracted from the C4 (Colossal Clean Crawled Corpus) dataset.

Collection Method

Filtered from a web crawl to identify news-like pages, likely based on URL patterns or source metadata.

Time Range

Crawl date of source C4 data is not specified.

Freshness

Last updated in 2021; static snapshot.

Geography

Primarily US-region sources.