Gemma 4 Public Bench Eval: ASR Performance on English Test Sets

Decoded transcripts and word-level error metrics from the Gemma 4 Unified models evaluated as automatic speech recognition systems. The dataset includes results for two model sizes, gemma4:e4b and gemma4:12b, on three standard English test sets. The evaluation was produced locally with ollama and includes a script for full reproducibility.

Use Cases

• Benchmark ASR model performance based on word error rate metrics
• Compare the speech recognition accuracy of different Gemma 4 model sizes
• Reproduce ASR evaluation results using the included evaluation script
• Analyze transcription errors on standard English test sets

Strengths

• Includes results for two distinct model sizes (8B and 12B parameters)
• Evaluation is fully reproducible with the included `eval_asr.py` tool
• Covers three standard English test sets for ASR benchmarking

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count is unknown, which may limit suitability assessment
• Data may reflect temporal or source bias inherent to the specific test sets used

Provenance

Source

huckiyang on Hugging Face

Collection Method

Produced locally with ollama, evaluating Gemma 4 models on standard test sets.

Freshness

Last updated 2026-06-10 17:35:55