Graphwalks: Multi-hop Reasoning Long Context Benchmark

Graphwalks is a long-context benchmark containing between 1,000 and 10,000 records, released by OpenAI in early 2026. It evaluates multi-hop reasoning by requiring models to execute graph operations, such as breadth-first search, on provided directed edge lists.

Use Cases

• Evaluating LLM performance on multi-hop reasoning using provided edge lists
• Testing long-context window capabilities through complex graph traversal operations
• Benchmarking breadth-first search accuracy in text-based graph representations

Strengths

• Created by OpenAI specifically for long-context benchmarking
• Contains 1,000 to 10,000 reasoning tasks
• Standardized graph structure where all nodes have at least degree 1
• Distributed under the MIT license

Limitations

• Limited to synthetic graph traversal tasks rather than real-world network data
• Lack of detailed column-level documentation in the primary metadata

Provenance

Source

OpenAI

Collection Method

synthetic

Freshness

Last updated March 2026.