Criminal Law Question-Answer Pairs With Source Passages

Isaacus's Legal RAG QA benchmark contains 138 question-answer-relevant passage triplets. It was sourced from the LibreTexts Introduction to Criminal Law textbook and includes 190 supporting passages and external materials. The dataset was created to evaluate end-to-end performance of legal retrieval-augmented generation applications.

Use Cases

• Benchmark retrieval systems on their ability to find the correct relevant_passages for a given legal question.
• Evaluate answer generation quality in legal RAG pipelines using the provided question-answer pairs as ground truth.
• Train or fine-tune models for legal question answering using the 138 triplets as supervision data.
• Analyze the relationship between question complexity and the number of required supporting passages from the 190 available.

Strengths

• Provides 138 curated question-answer-ground truth passage triplets for structured evaluation.
• Includes 190 source passages and external materials from a specific legal textbook.
• Focuses on the specialized domain of US criminal law, sourced from an authoritative educational platform.

Limitations

• Relatively small scale with only 138 QA pairs, limiting statistical power for some analyses.
• Narrow domain focus on introductory criminal law may not generalize to other legal areas.
• Original description indicates it was replaced by a larger benchmark, suggesting it may be a preliminary version.

Provenance

Source

LibreTexts' Introduction to Criminal Law textbook.

Collection Method

Sourced and curated into question-answer-relevant passage triplets by the author.

Time Range

null

Freshness

Last updated 2026-03-22.

Geography

US (inferred from textbook content and 'Regionus' tag).