Oolong Synth: A Synthetic Benchmark for Long Context Reasoning Evaluation

A synthetic dataset created by oolongbench for evaluating long-context reasoning and aggregation capabilities in AI models. The dataset is structured with input context windows and corresponding questions, requiring models to produce answers. It was last updated on June 20, 2026, with corrections to some temporal queries.

Use Cases

• Benchmarking model performance on long-context reasoning tasks based on the described context-window and question structure.
• Evaluating information aggregation capabilities across extended text passages based on the dataset's purpose.
• Testing model robustness on temporal queries within long contexts, as suggested by the update notes.

Strengths

• Designed specifically for evaluating long-context reasoning, a key capability for modern AI.
• Includes corrections for temporal queries, indicating maintenance and refinement.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• The dataset is synthetic, which may not reflect real-world data distributions.

Provenance

Source

oolongbench

Collection Method

Synthetic generation for evaluation, as described in the associated research paper.

Freshness

Last updated 2026-06-20 23:54:02