sWuggy: A Spoken Lexical-Discrimination Audio Benchmark

sWuggy is a spoken lexical-discrimination benchmark for evaluating spoken language models. Each item is a pair of a real word and a phonotactically matched pseudo-word, synthesized as audio. The dataset is hosted by the author 'coml' and was last updated on 2026-05-29.

Use Cases

• Benchmarking spoken language models based on their ability to discriminate real words from pseudo-words.
• Evaluating model performance on phonotactic plausibility judgments using synthesized audio pairs.
• Testing the robustness of audio feature extractors on controlled lexical stimuli.

Strengths

• The benchmark is specifically designed for lexical discrimination with phonotactically matched pairs.
• Audio data is provided in WebDataset tar shards, a format suitable for efficient large-scale processing.
• Includes per-item metadata required to run the evaluation.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Row count is unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.

Provenance

Source

huggingface, author 'coml'

Collection Method

Likely synthesized audio pairs created for benchmark purposes.

Freshness

Last updated 2026-05-29 14:55:47; freshness should be verified.