KazLawBench: Bilingual Legal LLM Benchmark for Kazakhstan

3,098 expert-style evaluation records form a bilingual Russian and Kazakh benchmark for legal large language models in Kazakhstan. It spans seven tasks across three cognitive levels: memorization, applying, and retrieval. The dataset was built on authoritative statutory sources and de-identified Supreme Court bulletins by author raiym76.

Use Cases

• Benchmarking model memorization of legal articles based on the article_recall task.
• Evaluating LLM application of law based on tasks at the 'applying' cognitive level.
• Testing legal information retrieval capabilities based on tasks at the 'retrieval' cognitive level.
• Assessing model performance across languages based on the bilingual Russian and Kazakh structure.

Strengths

• 3,098 expert-style evaluation records provide a substantial test set.
• Covers seven distinct tasks across three cognitive levels, suggesting structured evaluation.
• Built on authoritative sources like adilet.zan.kz and sud.gov.kz, indicating a foundation in real legal texts.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known but other scale details like file formats and size are unknown.
• Last updated 2026-05-18 06:23:49; freshness should be verified.

Provenance

Source

Authoritative statutory sources (adilet.zan.kz) and de-identified Supreme Court bulletins (sud.gov.kz).

Collection Method

Likely compiled and annotated by experts for benchmark construction.

Freshness

Last updated 2026-05-18.

Geography

Republic of Kazakhstan.