CLI-Gym: 1,655 Verified Problem Instances for Runtime Environment Diagnosis

1,655 CLI-Gym problem instances released on 2026-02-28 by LiberCoders. Each instance is designed to evaluate and train an agent's ability to diagnose and repair real runtime environments via CLI interaction, with executable verification through unit tests.

Use Cases

• Training reinforcement learning agents for CLI-based problem diagnosis based on the described environment-intensive tasks.
• Benchmarking AI agents' ability to repair system configuration issues based on the executable verification through unit tests.
• Developing automated tools for dependency and permission management based on the described real runtime environment scenarios.

Strengths

• Contains 1,655 distinct problem instances, providing a substantial testbed.
• Each instance includes executable verification through unit tests, suggesting a defined ground truth.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Freshness should be verified as the last update was 2026-02-28 08:59:48.

Provenance

Source

LiberCoders via Hugging Face

Collection Method

Official dataset release from the CLI-Gym project.

Freshness

Last updated 2026-02-28 08:59:48.