Agentic Redteam Benchmark: 2,288 Multi-Step Agent Trajectories for Verifier Evaluation

Aegis AI released a benchmark in 2026 containing 2,288 multi-step agent trajectories for evaluating AI-agent governance verifiers. It includes 513 hand-authored gold-standard trajectories and 1,775 provenance-flagged augmented examples. The dataset is designed to score whether a verifier catches drift inside an agent's trajectory, not whether a prompt is harmful.

Use Cases

• Benchmarking verifier models based on their ability to detect drift in multi-step agent trajectories.
• Training and evaluating red-teaming agents using the hand-authored and augmented trajectory data.
• Researching AI safety and governance metrics for agentic systems using the per-step scoring methodology.

Strengths

• Contains 2,288 multi-step agent trajectories, providing a substantial corpus for evaluation.
• Includes 513 hand-authored gold-standard examples, offering a reliable baseline.
• Features 1,775 provenance-flagged augmented examples, expanding the dataset's scope.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

huggingface, author jash-ai, associated with Aegis AI.

Collection Method

Likely contains hand-authored and procedurally augmented agent trajectories.

Freshness

Last updated 2026-06-21 16:12:57; freshness should be verified.