RootSE: Benchmark for Earliest-Step Failure Diagnosis on Repository-Level Coding Agents

RootSE comprises 102 instances from four representative repository-level coding agents, spanning 35 repositories across three programming languages. The dataset includes over 5,000 execution steps and approximately 30 million characters, with each instance annotated for failure diagnosis. It was created by dengdan1999 and last updated on Hugging Face in July 2026.

Use Cases

• Benchmarking the failure diagnosis capabilities of AI coding agents based on execution trajectories.
• Training models to identify the earliest decisive error step in multi-step coding tasks.
• Analyzing agent performance across different programming languages (Python, Go, JavaScript).

Strengths

• Contains 102 annotated instances from four different coding agents, providing comparative data.
• Spans 35 repositories and three programming languages, with Python representing 71.6% of the content.
• Includes over 5,000 execution steps and approximately 30 million characters of data.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for large-scale training.
• Data may reflect bias inherent to the specific coding agents and repositories selected.

Provenance

Source

Hugging Face dataset by dengdan1999.

Collection Method

Likely collected from the execution trajectories of four repository-level coding agents.

Freshness

Last updated 2026-07-10 06:45:04; freshness should be verified.