Qwen3 Language Model Logits For Knowledge Distillation

A dataset containing logits from the Qwen3 235B large language model, packed for efficient storage. It was created by Arcee-ai and published on Hugging Face in December 2025. The data is pre-packed for training sequences with a fixed length of 8192 tokens.

Use Cases

• Distill a smaller student model by training it to mimic the teacher model's logits output.
• Analyze model confidence and prediction distributions across the 151,936-dimensional logit space.
• Benchmark compression algorithms for logit storage using the provided delta_encoding and exact_dtype parameters.

Strengths

• Logits are pre-packed for sequences of 8192 tokens, simplifying data loading for training.
• Uses delta encoding and a specified exact_k of 32 for potential storage efficiency.
• Logits are derived from the Qwen3 235B model, a large and capable teacher model.

Limitations

• The number of training examples (rows) is unknown, making it difficult to assess dataset scale.
• Specific source data and preprocessing steps for generating the logits are not documented.
• The license for use and redistribution is unspecified.

Provenance

Source

arcee-ai/Qwen3-235B-Logits-Packed-8192 repository on Hugging Face.

Collection Method

Logits extracted from the Qwen3 235B model; specific generation process not detailed.

Freshness

Last updated on December 10, 2025.