Queries and raw model outputs analyzed in the paper 'Hallucination Free? Assessing the Reliability of Leading AI Legal Research Tools'. The dataset was created by authors Magesh, Surani, Dahl, Suzgun, Manning and Ho for the Journal of Empirical Legal Studies (2024, forthcoming). A random sample of 50% of the dataset is reserved for benchmarking.
Use Cases
- Benchmarking AI model reliability in legal research based on queries and outputs.
- Analyzing hallucination patterns in AI-generated legal text based on raw model outputs.
- Developing evaluation frameworks for legal AI tools based on the reserved benchmarking sample.
Strengths
- Dataset is associated with a forthcoming peer-reviewed journal article.
- Includes a reserved random sample of 50% for benchmarking purposes.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
- Last updated 2024-11-14 18:53:33; freshness should be verified.
Provenance
- Source
- reglab
- Freshness
- 2024-11-14