Agent Safety Bench: A Benchmark for Evaluating Tool-Using LLM Agents

Agent Safety Bench (ASB) is a benchmark for evaluating the safety of tool-using LLM agents. Each example pairs a natural-language instruction with one or more sandboxed tool environments to measure whether an agent completes tasks without taking unsafe actions. The dataset was created by author 'aradhye' and was last updated on 2026-05-04.

Use Cases

• Benchmarking agent safety performance based on natural-language instructions paired with tool environments.
• Developing safety guardrails for LLM agents based on the benchmark's task structure.
• Comparing different agent architectures on their propensity for unsafe actions in sandboxed environments.

Strengths

• The benchmark is specifically designed for the emerging domain of tool-using LLM agent safety.
• It pairs natural-language instructions with sandboxed tool environments, providing a structured testbed.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Last updated 2026-05-04 03:20:13; freshness should be verified.

Provenance

Source

huggingface

Freshness

Last updated 2026-05-04.