RTX 4060 Ti 8GB Turboquant KV Cache Benchmarks for Qwen3.6-35B

Benchmark results compare turboquant KV cache types against standard llama.cpp inference on specific hardware. The tests were conducted on an NVIDIA RTX 4060 Ti 8GB GPU with 32GB DDR5-6000 RAM, running the Qwen3.6-35B-A3B-UD-Q4_K_M model. The dataset was created by author witcheer and last updated on May 12, 2026.

Use Cases

• Compare inference speed between turboquant and standard KV cache implementations based on the benchmark results.
• Evaluate the performance impact of different quantization techniques on a hybrid SSM+attention model architecture.
• Assess the viability of running a 35B-parameter quantized model on an RTX 4060 Ti with 8GB VRAM.

Strengths

• Hardware configuration is fully specified, including GPU, CPU, RAM, and OS details.
• Benchmark targets a specific, named model version (Qwen3.6-35B-A3B-UD-Q4_K_M).

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Practitioner-tested benchmarks.

Freshness

Last updated 2026-05-12 12:21:51; freshness should be verified.