Sign in to view source links and access this dataset
Description
1,979 claimed legal quotations from Showalter's Law Dictionary were verified by three frontier AI models. The benchmark records model verdicts and reasoning for each quote's accuracy in its cited case. Created by Kingsfield-Lawfare and released in May 2026, it evaluates the performance of Claude, Gemini, and GPT-4o.
Use Cases
Benchmarking AI model performance on legal fact-checking based on the recorded model verdicts.
Analyzing patterns in AI-generated reasoning for legal citation verification tasks.
Training models for legal NLP tasks based on the verified quotation and case attribution structure.
Studying hallucination rates in large language models within the legal domain.
Strengths
Contains 1,979 specific legal quotation verification instances.
Evaluates three prominent frontier AI models (Claude, Gemini, GPT-4o).
Quotations are drawn from a specific legal reference source, Showalter's Law Dictionary.
Limitations
Column-level documentation is absent; field semantics must be inferred after download.
Row count is known, but the specific features and data structure are unknown.
Freshness should be verified; last updated 2026-06-01.
Provenance
Source
Kingsfield-Lawfare
Collection Method
Quotations drawn from Showalter's Law Dictionary; verification performed by AI models.
Freshness
Last updated 2026-06-01.
License is unknown; terms of use must be verified before application.