OABench: Brazilian Bar Exam Questions for LLM Evaluation

OABench provides a rigorous testbed for evaluating Large Language Models on Brazilian legal knowledge. The dataset is based on the comprehensive and challenging Unified Bar Exam required for law practice in Brazil. It was created by author felipeoes and last updated on 2025-02-27.

Use Cases

• Benchmarking LLM performance on legal question-answering based on Brazilian Bar Exam content.
• Evaluating domain-specific reasoning in Portuguese-language legal contexts based on the exam's wide range of topics.
• Fine-tuning or testing legal AI assistants for the Brazilian market based on realistic exam questions.

Strengths

• Based on the official, comprehensive Unified Bar Exam of the Brazilian Bar Association (OAB).
• Designed as a rigorous and realistic benchmark for the legal domain.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Last updated 2025-02-27 19:58:21; freshness should be verified.

Provenance

Source

Unified Bar Exam of the Brazilian Bar Association (OAB).

Collection Method

Likely compiled from exam questions and materials.

Freshness

2025-02-27 19:58:21

Geography

Brazil