VLMSafe-420: 420 Multimodal Counterfactual Pairs for VLM Safety Circuits

VLMSafe-420 consists of 420 multimodal counterfactual pairs across 38 safety categories, developed by ArthT and updated in March 2026. The data is designed for mechanistic interpretability research to identify and analyze safety circuits within Vision-Language Models.

Use Cases

• Mechanistic interpretability of safety circuits using harmful/benign counterfactual pairs
• VLM safety benchmarking across 38 safety categories
• Adversarial testing using 50 JailbreakBench-style prompts

Strengths

• 420 counterfactual pairs for controlled safety testing
• 38 distinct safety categories
• Includes 50 JailbreakBench-style prompts
• 226 text-based counterfactual entries

Limitations

• Small sample size of 420 records
• Niche focus on mechanistic interpretability may limit general safety benchmarking

Provenance

Source

ArthT via Hugging Face

Freshness

Last updated March 2026.