EvoClaw Data: Software Evolution Itineraries for AI Agent Evaluation

EvoClaw Data contains 98 graded milestones extracted from 7 real-world open-source repositories spanning 5 programming languages. The dataset includes metadata, task specifications, dependency graphs, and test classifications for running evaluation trials. It was created by author hyd2apse and last updated on March 24, 2026.

Use Cases

• Benchmarking AI agents on software evolution tasks based on the provided milestone dependency graphs.
• Evaluating code generation models using the task specifications and SRS documents.
• Analyzing test outcome transitions (e.g., fail_to_pass) for AI-generated code.
• Studying software development patterns across multiple programming languages based on the repository metadata.

Strengths

• Contains 98 graded milestones for structured evaluation.
• Derived from 7 real-world open-source repositories across 5 programming languages.
• Includes multiple structured components like dependency graphs and test classifications.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

huggingface

Collection Method

Extracted from real-world open-source repositories.

Freshness

Last updated 2026-03-24 23:12:06; freshness should be verified.