DeepSeek Language Model Logits Packed to 8192 Sequence Length

Logits from the DeepSeek-DCLM model are packed into sequences of 8192 tokens for efficient training. The dataset, created by arcee-ai, contains compressed teacher logits for knowledge distillation. It was last updated in December 2025.

Use Cases

• Train a student model to match the teacher's logit distribution across a 129,280 token vocabulary.
• Implement knowledge distillation using the 32 exact top-k and 128 compressed logit values per token.
• Benchmark distillation algorithms on sequences with a fixed 8192-token length.
• Analyze the effect of polynomial inversion and degree 8 compression on logit fidelity.

Strengths

• Fixed sequence length of 8192 tokens for consistent batch processing.
• Logits are pre-packed and compressed, reducing storage and I/O overhead.
• High-dimensional output space with a vocabulary size of 129,280 tokens.

Limitations

• Unknown dataset size, row count, and sample diversity.
• Potential information loss from logit compression (k=128, polynomial_degree=8).
• No information on the source text data used to generate the logits.

Provenance

Source

arcee-ai

Collection Method

Logits generated by the DeepSeek-DCLM model and processed with legacy logit compression.

Freshness

Last updated December 10, 2025.