Kernelbench Hard Traces: Frontier AI Agent Code Generation Sessions

149 agent traces document AI coding agents writing optimized CUDA and Triton kernels for tasks like FP8 GEMM and paged attention. The traces are stored as JSONL files from Claude-Code sessions and are graded against a roofline model on GPUs like the RTX PRO 6000 Blackwell. Created by Infatoshi, the dataset was last updated on June 20, 2026.

Use Cases

• Analyze agent reasoning patterns based on the Claude-Code session format traces
• Benchmark AI performance on GPU kernel optimization tasks based on the roofline-graded scores
• Study code generation for specific compute patterns like FP8 GEMM or MoE based on the described problem types

Strengths

• Contains 149 distinct agent traces for analysis.
• Traces are graded using a roofline model on specific GPUs like the H100 PCIe and B200.
• Includes a manifest.csv file mapping runs to model, harness, problem, GPU, and score.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Secrets are redacted, which may affect reproducibility for some use cases.

Provenance

Source

Infatoshi via Hugging Face.

Collection Method

Traces generated by frontier coding agents during benchmark sessions.

Freshness

Last updated 2026-06-20 01:44:07; freshness should be verified.