REALM-Bench: Job Shop Scheduling Benchmark for LLMs and Multi-Agent Systems

436 total instances across four tiers (J1–J4) of Job Shop Scheduling Problems (JSSP) in a unified JSON schema. The dataset, created by GloriaGeng, is hosted on Hugging Face and was last updated on 2026-05-21. It is derived from the REALM-Bench paper as a real-world planning benchmark.

Use Cases

• Benchmarking LLM reasoning on static scheduling problems based on the J1 tier instances.
• Evaluating multi-agent system performance on dynamic scheduling scenarios based on the J2 tier description.
• Training or fine-tuning models for combinatorial optimization based on the structured JSSP problem format.

Strengths

• 436 instances provide a substantial testbed for evaluation.
• Instances are organized into four distinct tiers (J1–J4) for progressive difficulty.
• Uses a single unified JSON schema, which suggests consistent formatting.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count per instance is unknown, which may limit suitability assessment.
• The description references dynamic elements in J2, but the specifics are not detailed here.

Provenance

Source

GloriaGeng on Hugging Face, from the REALM-Bench paper.

Collection Method

Likely compiled from established JSSP benchmark suites (e.g., DMU, TA, ABZ, SWV, YN).

Freshness

Last updated 2026-05-21 22:03:51.