KernelBenchX: Reproducible GPU Kernel Code Generation Benchmark

KernelBenchX provides 176 benchmark tasks for evaluating Triton GPU-kernel code generation by LLMs. It measures buildability, numerical correctness against a deterministic test suite, and end-to-end speedup versus a GPU-matched golden reference. The dataset was created by BonnieWang and is hosted on Hugging Face.

Use Cases

• Benchmarking LLM performance on GPU kernel generation based on the 176 task specifications
• Evaluating numerical correctness of generated code based on the deterministic test harness
• Measuring end-to-end speedup of generated kernels against a golden reference implementation

Strengths

• Includes 176 benchmark tasks with specifications and reference implementations
• Provides a deterministic test harness for evaluating numerical correctness
• Measures end-to-end speedup against a GPU-matched golden reference

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count for the primary data is unknown, which may limit suitability assessment
• Freshness should be verified as the last update was 2026-05-25

Provenance

Source

BonnieWang on Hugging Face

Collection Method

Created as a reproducible evaluation benchmark for research, likely from synthetic or curated tasks.

Freshness

Last updated 2026-05-25 08:32:39