Opus47-20T Prompterv2 Test: Software Engineering Agent Benchmark

200 instances comprise this benchmark for evaluating software engineering agents, with 95% file match and 89% function match rates. The dataset was created by author zvzv1919 and last updated on 2026-04 17. It appears to be part of a collection for testing agents on tasks like code location and function matching.

Use Cases

• Benchmarking code location agents based on the 'locate' agent function mentioned in the config.
• Evaluating function matching performance based on the reported 89% function match rate.
• Analyzing agent interaction patterns based on the reported average of 11.6 turns per instance.
• Testing software engineering agent robustness based on the reported file and function mismatch counts.

Strengths

• Contains 200 benchmark instances for consistent evaluation.
• Reports specific performance metrics: 95% file match and 89% function match rates.
• Provides config details such as agent function ('locate') and a defined benchmark name.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Likely generated as a test or evaluation set for software engineering agents.

Time Range

null

Freshness

Last updated 2026-04-17 22:17:38; freshness should be verified.

Geography

null