A practitioner test of Mixture-of-Experts (MoE) expert offload performance on a Qwen3.6-35B-A3B model using a consumer-grade 32GB RAM system. The dataset compares full and partial offload regimes, reporting token generation speeds. The benchmark was created by author witcheer and last updated on 2026-05-07.
Use Cases
- Benchmarking MoE offload strategies based on the described full and partial offload regimes.
- Estimating inference performance for the Qwen3.6-35B-A3B model based on the reported token-per-second metrics.
- Planning hardware requirements for LLM inference based on the RAM ceiling findings for a 32GB system.
- Comparing CPU vs. GPU expert placement efficiency based on the 4.8x speedup result.
Strengths
- Provides concrete performance metrics, including 7.40 tok/sec for full offload and 35.36 tok/sec for partial offload.
- Compares two distinct offload strategies (full and partial) on a specific hardware and model configuration.
- Includes a specific context length detail (32K context) for the partial offload test.
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download.
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
Provenance
- Source
- huggingface
- Collection Method
- Practitioner test conducted by the author.
- Time Range
- Benchmark dated 2026-05.
- Freshness
- Last updated 2026-05-07 13:01:45; freshness should be verified.
- Geography
- null