SkillHarm: Benchmark of Lifecycle-Aware Skill-Based Attacks on AI Agents

SkillHarm is a benchmark of skill-based attacks across the agent skill-use lifecycle, paired with a systematic taxonomy of 12 skill-relevant risks. The dataset was created by osunlp and was last updated on June 2, 2026. Every attack is documented on the dataset page.

Use Cases

• Evaluating agent robustness based on the taxonomy of 12 skill-relevant risks.
• Testing supply-chain attack surfaces based on the concept of third-party skills.
• Benchmarking adversarial attacks based on the lifecycle-aware skill-based attack framework.

Strengths

• Includes a systematic taxonomy of 12 skill-relevant risks.
• Focuses on attacks across the entire skill-use lifecycle.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

osunlp

Collection Method

Automated construction, as indicated by the description.

Freshness

Last updated 2026-06-02 03:44:43; freshness should be verified.