SpellKarm: A Procedurally Generated Spelling Benchmark for LLMs

SpellKarm is a multiple-choice benchmark created by fromziro to test the spelling capabilities of large language models. It contains approximately 1,000 English words, ranging from common stop words to complex multisyllabic terms. Each question has about 5 answer choices, resulting in a random chance accuracy of 20%.

Use Cases

• Benchmarking LLM spelling accuracy based on the described set of 1,000 English words.
• Evaluating model performance on simple versus complex vocabulary based on the range from stop words to multisyllabic words.
• Analyzing multiple-choice question design for language models based on the described 5-answer-choice structure.

Strengths

• Contains approximately 1,000 different English words, providing a varied vocabulary for testing.
• Includes a range of word difficulties, from simple stop words to complex multisyllabic examples.
• Structured as a multiple-choice test with about 5 choices per question, enabling controlled evaluation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

huggingface

Collection Method

Procedurally generated benchmark.

Time Range

null

Freshness

Last updated 2026-06-07 08:57:11; freshness should be verified.

Geography

null