AgentDoG-Lite: Agent Safety Diagnostic Test Set for Trajectory-Level Judgment

AgentDoG-Lite provides a test set for evaluating AI agent safety based on multi-turn execution trajectories. The dataset, created by AI45Research and last updated in July 2026, is designed for a specific challenge assessing whether an agent performs unsafe actions or follows unsafe decision patterns. It focuses on a binary safety judgment task for complete agent interaction sequences.

Use Cases

• Benchmarking training-free harness methods for agent safety based on trajectory-level evaluation
• Evaluating fine-tuned models on agent safety judgment tasks based on multi-turn interactions
• Testing AI systems on detecting unsafe patterns within tool calls and environment feedback mentioned in the description

Strengths

• Dataset is designed for a specific, defined evaluation task: trajectory-level agent safety judgment
• Focuses on multi-turn agent trajectories, a more complex evaluation than single-turn input/output safety checks

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download
• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment

Provenance

Source

AI45Research via Hugging Face

Collection Method

Likely created as a test set for a specific AI safety challenge.

Freshness

Last updated 2026-07-03 01:30:54; freshness should be verified