DeepSeek Mixed-Mode Reasoning Logits Packed for Sequence Length 16384

A dataset of teacher model logits for knowledge distillation, featuring a sequence length of 16384 tokens. It was created by Arcee-ai and published on Hugging Face in December 2025. The logits are pre-packed and compressed from a vocabulary of 129,280 tokens.

Use Cases

• Distill a smaller student model using the teacher logits from the 129,280 token vocabulary.
• Fine-tune language models on mixed-mode reasoning tasks using the pre-packed sequences of length 16384.
• Analyze the effect of logit compression with exact_k=32 and polynomial degree 0 on model performance.
• Benchmark training efficiency for long-context models using the preprocessed, packed training data.

Strengths

• Sequence length of 16384 tokens for long-context training.
• Logits derived from a vocabulary of 129,280 tokens.
• Pre-packed data format for efficient training pipeline integration.
• Logit compression applied with exact_k=32.

Limitations

• Unknown total number of training examples (rows).
• Limited information on data composition and source tasks.
• Potential bias towards the specific reasoning tasks used by the teacher model.

Provenance

Source

arcee-ai/DeepSeek-MixedModeReasoning-Logits-Packed-16384 on Hugging Face.

Collection Method

Logits generated by a teacher model, likely from the DeepSeek family, and pre-packed for training.

Freshness

Last updated in December 2025.