Bright-Pro: Expert-Annotated Benchmark for Multi-Aspect Reasoning Retrieval

Bright-Pro is an expert-annotated extension of the BRIGHT benchmark for reasoning-intensive retrieval. The dataset, created by yale-nlp, was last updated on April 30, 2026. It provides queries paired with multi-aspect reasoning decompositions and curated gold passages.

Use Cases

• Fine-grained evaluation of retrievers based on multi-aspect reasoning decomposition.
• Analysis of whether a retriever covers complementary reasoning aspects based on weighted importance scores.
• Benchmarking retrieval systems on reasoning-intensive queries rather than simple relevance.
• Studying the relationship between query decomposition and passage relevance.

Strengths

• Dataset is expert-annotated, suggesting high-quality labels.
• Provides multi-aspect reasoning decompositions and weighted importance scores for fine-grained analysis.
• Includes curated gold passages organized by aspect for detailed evaluation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

yale-nlp

Collection Method

Expert-annotated extension of an existing benchmark.

Time Range

null

Freshness

Last updated 2026-04-30 02:20:40; freshness should be verified.

Geography

null