Terminal-Bench 2 Verified: AI Agent Evaluation Dataset with Environment Fixes

A verified version of the Terminal-Bench 2.0 dataset, last updated on 2026-04 28. It addresses identified environment and instruction issues, including updated Dockerfiles and runtime support for the Claude Code Agent. The dataset was created by author harithoppil and used to evaluate models like GLM-5 and Step 3.5-Flash.

Use Cases

• Benchmarking AI agent performance based on the described instruction-following tasks
• Testing environment compatibility for code agents based on the updated Dockerfiles and runtime support
• Evaluating model robustness based on the verified fixes to the original dataset's issues

Strengths

• Includes verified fixes for environment and instruction issues identified in the original Terminal-Bench 2.0
• Provides updated Dockerfiles and instructions to support the Claude Code Agent runtime
• Has been used to evaluate specific models, including GLM-5 and Step 3.5-Flash

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download
• Column-level documentation is absent; field semantics must be inferred after download
• Row count, file formats, and license information are unknown, which may limit suitability assessment

Provenance

Source

huggingface

Collection Method

Derived from the Terminal-Bench 2.0 dataset after a comprehensive review and modification.

Time Range

null

Freshness

Last updated 2026-04-28 08:15:22; freshness should be verified

Geography

null