JAWS-Bench: Jailbreak Attacks Across Three Workspace Regimes

JAWS-Bench (Jailbreaks Across WorkSpaces) is a benchmark spanning three escalating workspace regimes: empty (JAWS-0), single-file (JAWS-1), and multi-file (JAWS-M). It includes a hierarchical, executable-aware Judge Framework that tests compliance, attack success, syntactic correctness, and runtime executability. The dataset was authored by Shoumik Saha and last updated on May 16, 2026.

Use Cases

• Benchmarking LLM resistance to jailbreak prompts based on the three workspace regimes.
• Evaluating attack success and runtime executability of generated code based on the described Judge Framework.
• Training and testing AI safety classifiers based on compliance and syntactic correctness metrics.

Strengths

• Benchmark spans three distinct, escalating workspace regimes (JAWS-0, JAWS-1, JAWS-M).
• Judge Framework evaluates four specific criteria: compliance, attack success, syntactic correctness, and runtime executability.
• Dataset is 2.9 MB in size and available under a CC-BY-4.0 license.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Data may reflect temporal or source bias inherent to its collection method.

Provenance

Source

Shoumik Saha via figshare

Freshness

Last updated 2026-05-16 03:15:05; freshness should be verified.