Trellis-Coded Quantization Codebooks for LLM KV Cache Compression

Trained TCQ codebooks enable KV cache compression at 2-3 bits per parameter, as presented in the paper 'Closing the Gap: Trellis-Coded Quantization for KV Cache at 2-3 Bits'. The dataset was created by authors buun and Claude (Anthropic) from Anthropic and was last updated in April 2026. It includes the quantization codebooks, training scripts, and the associated research paper.

Use Cases

• Apply the provided TCQ codebooks to compress KV cache activations in transformer models during inference.
• Reproduce or extend the paper's results using the included codebook training scripts.
• Benchmark the performance of 2-3 bit TCQ against other quantization methods for KV cache.
• Analyze the structure of the trained codebooks to understand learned quantization patterns for attention keys and values.

Strengths

• Includes the core research artifacts: trained quantization codebooks and training scripts.
• Accompanied by the full, peer-reviewed conference paper detailing the methodology.

Limitations

• The dataset size, row count, and specific codebook dimensions are not provided.
• The dataset's utility is confined to the specific TCQ method and may not be directly applicable to other compression techniques.

Provenance

Source

Anthropic (authors buun and Claude).

Collection Method

Created to accompany the academic paper 'Closing the Gap: Trellis-Coded Quantization for KV Cache at 2-3 Bits'.

Freshness

Last updated in April 2026.