Two categories of sampled English and Chinese Common Crawl text from the OpenSeek Pretraining Dataset v1.0. The collection includes English data from Nemotron-CC and Chinese data processed via the Nemotron-CC pipeline to ensure cross-lingual consistency.
Use Cases
- Pretrain large language models using the English and Chinese web text samples
- Analyze the effects of the Nemotron-CC data pipeline on Chinese language web text
- Test cross-lingual tokenization strategies on the provided English and Chinese text segments
Strengths
- Includes English Common Crawl data sourced from the Nemotron-CC dataset
- Features Chinese Common Crawl data processed using the Nemotron-CC data pipeline
- Provides a representative sample of the OpenSeek Pretraining Dataset v1.0