LegalBench.BR: Benchmark for LLMs on Brazilian Law Tasks

LegalBench.BR is a benchmark dataset for evaluating large language models on tasks grounded in Brazilian Law and written in Brazilian Portuguese. The dataset was created by author celsowm and was last updated on 2026-05-11. It combines synthetic legal questions, court-decision excerpts, legal entailment tasks, and closed-book questions to test model capabilities.

Use Cases

• Benchmarking LLM question-answering performance based on synthetic legal questions.
• Evaluating LLM classification accuracy on tasks derived from Brazilian court-decision excerpts.
• Testing LLM inference and recall abilities using legal entailment tasks described in the dataset.

Strengths

• Covers multiple legal areas within the Brazilian legal system.
• Designed specifically for tasks in Brazilian Portuguese, addressing a language-specific need.
• Combines multiple task types including question answering, classification, inference, and recall.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

celsowm

Collection Method

Likely compiled from synthetic legal questions, court-decision excerpts, and legal entailment tasks.

Freshness

Last updated 2026-05-11 23:19:08; freshness should be verified.

Geography

Brazilian legal system.