Common Crawl Text Corpus for Language Modeling

A text corpus sourced from Common Crawl, curated by sc2qa. The dataset is associated with the arXiv paper 2109.04689 and was last updated in March 2022.

Use Cases

• Train language models on web-scale text data from the Common Crawl corpus.
• Analyze text modality patterns within the 'Size Categories100 Kn1 M' tag specification.
• Use as a pre-training corpus for NLP tasks leveraging the 'text' modality tag.

Strengths

• Derived from Common Crawl, a large-scale web crawl.
• Associated with a specific research paper (arXiv:2109.04689).
• Tagged with specific metadata including 'text' modality and 'us' region.

Limitations

• The exact size, row count, and column structure are unknown.
• Data is from 2022 or earlier and may not reflect current web content.
• Lack of detailed documentation on preprocessing and filtering methods.

Provenance

Source

Common Crawl web corpus.

Collection Method

Curated and processed by sc2qa.

Freshness

Last updated on 2022-03-30.

Geography

Tagged with 'us' region, but specific coverage is unknown.