SpeechJBB: Audio Benchmark for Safety Alignment in Multilingual Speech Models

SpeechJBB is an audio benchmark for evaluating safety alignment and comprehension in large audio language models. It tests model responses to harmful spoken requests across monolingual speech, code-switched speech, and code-switched speech with pseudo-word obfuscation. The dataset was created by virginiaceccatelli and was last updated on 2026-06-16.

Use Cases

• Evaluating model refusal of harmful requests based on multilingual speech prompts.
• Benchmarking safety alignment robustness based on code-switched speech inputs.
• Testing model comprehension under natural-sounding pseudo-word obfuscation mentioned in the description.

Strengths

• Benchmark is specifically designed for evaluating safety alignment in audio language models.
• Dataset tests prompts across three distinct conditions: monolingual, code-switched, and obfuscated speech.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

virginiaceccatelli on Hugging Face

Freshness

Last updated 2026-06-16 15:00:54; freshness should be verified.