Dclm Baseline 8B Docpack Ctx2048: Tokenized Training Data Shards

201 tokenized data shards for training language models, stored in .tokbin format. The data was uploaded by marissatech and last updated on June 12, 2026. Each shard contains sequences of token IDs from a vocabulary of 24,256 tokens.

Use Cases

• Train decoder-only language models based on the provided tokenized text sequences.
• Benchmark model performance using the standardized baseline data shards.
• Preprocess text data for large-scale language model training pipelines.
• Analyze token distribution and sequence structure within the training corpus.

Strengths

• Contains 201 data shards, providing a substantial volume of training material.
• Uses a defined vocabulary size of 24,256 tokens, offering a consistent tokenization scheme.
• Includes a dataset manifest (dataset_manifest.json) for managing the shard collection.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

marissatech on Hugging Face

Collection Method

Tokenized text data prepared as training shards for language models.

Freshness

Last updated 2026-06-12 23:19:13; freshness should be verified.