OctoCodingBench: Benchmark for Scaffold-Aware Instruction Following in Coding Agents

OctoCodingBench is a benchmark for evaluating scaffold-aware instruction following in repository-grounded agentic coding. It was created by MiniMaxAI and last updated on Hugging Face on January 13, 2026. The benchmark focuses on whether coding agents follow rules while solving tasks, a dimension not covered by existing task-completion benchmarks.

Use Cases

• Benchmarking agent adherence to coding rules based on the scaffold-aware instruction-following focus
• Evaluating repository-grounded coding agents based on the described benchmark scope
• Comparing agent performance against existing task-completion benchmarks like SWE-bench

Strengths

• Focuses on a critical dimension (rule-following) not covered by existing benchmarks like SWE-bench
• Created by a named author (MiniMaxAI)
• Last updated date is explicitly provided (2026-01 13)

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Description metadata is limited; actual data quality requires manual inspection after download

Provenance

Source

MiniMaxAI

Collection Method

Likely constructed for benchmarking purposes, but specific gathering method is not detailed.

Time Range

null

Freshness

Last updated 2026-01-13 13:02:26

Geography

null