MHJ: Multi-Turn Human Jailbreaks for LLM Defense Testing

ScaleAI compiled 2,912 successful jailbreak prompts across 537 multi-turn conversations for the paper 'LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks'. The dataset includes metadata such as design choice comments from red teamers and the resulting attack success rate (ASR). It was last updated on the platform in September 2024.

Use Cases

• Benchmarking LLM defense robustness based on multi-turn adversarial prompts
• Analyzing red teaming strategies based on included design choice comments
• Training or fine-tuning detection models for jailbreak attempts
• Studying attack success rate (ASR) patterns across different conversation structures

Strengths

• Contains 2,912 specific adversarial prompts
• Structures attacks across 537 multi-turn conversations
• Includes human red teamer metadata such as design choice comments

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is known but other scale details like file formats and size are unknown
• Data may reflect bias inherent to the specific red teaming methodology and source platform

Provenance

Source

ScaleAI

Collection Method

Compiled from successful jailbreaks submitted by human red teamers for a specific research paper.

Time Range

Dataset card dated August 27, 2024.

Freshness

Last updated 2024-09-19 01:10:35

Geography

Region metadata indicates 'us', but specific geographic coverage of the data is unknown.