Opus47-20T Basepatch: Software Engineering Agent Benchmark Results

56 total instances of agent performance on a software engineering benchmark. The dataset, created by user zvzv1919, was last updated on 2026-04-17. It likely contains metrics from evaluating an agent's ability to locate and match code functions and files.

Use Cases

• Benchmarking AI agent performance on code location tasks based on the 'locate' agent function.
• Analyzing the accuracy of function and file matching in codebases based on the reported match and mismatch metrics.
• Studying agent interaction complexity based on the reported average number of turns per instance.

Strengths

• Provides specific evaluation metrics: 56 total instances, 44 file matches (78%), and 26 function matches (46%).
• Includes a clear benchmark configuration with agent function ('locate') and model details, suggesting a structured evaluation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

Hugging Face user zvzv1919.

Collection Method

Likely generated from evaluating an AI agent on a software engineering benchmark collection named 'swe-lite-sympy-11400'.

Time Range

null

Freshness

Last updated 2026-04-17 22:31:59; freshness should be verified.

Geography

null