CyberSecEval 3 Visual Prompt Injection is a multimodal benchmark from Meta for evaluating cybersecurity risks in LLMs. It contains text and image inputs designed to test visual prompt injection vulnerabilities. The dataset is part of a larger security benchmark suite and was last updated in March 2025.
Use Cases
- Benchmark LLM resilience to visual prompt injection attacks using text and image inputs
- Evaluate multimodal model security by analyzing responses to adversarial visual prompts
- Research adversarial machine learning techniques in the cybersecurity domain
Strengths
- Part of Meta's flagship CyberSecEval 3 benchmark suite for cybersecurity
- Focuses on the specific and relevant security domain of visual prompt injection
- Multimodal design incorporating both text and image inputs for testing
Limitations
- No public information on dataset size, row count, or specific features
- Limited to English language inputs, restricting cross-lingual security evaluation
- Benchmark nature may limit applicability for general model training beyond evaluation
Provenance
- Source
- Meta (facebook)
- Collection Method
- null
- Time Range
- null
- Freshness
- Last updated 2025-03-13
- Geography
- null