MultiWorld: Multi-Agent Video Data for World Models

MultiWorld is a large-scale dataset for training video world models, containing two complementary data sources. It includes over 100 hours of real human gameplay from the cooperative game It Takes Two, featuring dual-agent synchronized actions with distinct first-person viewpoints. The dataset also contains multi-robot manipulation trajectories from the RoboFactory source, as described by author Haoyuwu.

Use Cases

• Training video world models based on the described multi-view, multi-agent video sequences.
• Developing multi-agent reinforcement learning policies based on synchronized action trajectories from the It Takes Two gameplay.
• Benchmarking robotic manipulation algorithms based on the included RoboFactory trajectories.
• Researching human-robot or robot-robot collaboration based on the dual-agent, multi-view structure.

Strengths

• Contains over 100 hours of real human gameplay data.
• Features dual-agent synchronized actions with distinct first-person viewpoints, as described.
• Combines two complementary data sources: gameplay and robotic manipulation.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count, file formats, and total size are unknown, which may limit suitability assessment.
• License information is unknown, which may restrict usage.

Provenance

Source

huggingface

Collection Method

Collected from real human gameplay of It Takes Two and multi-robot manipulation trajectories from RoboFactory.

Freshness

Last updated 2026-04-20 17:46:54; freshness should be verified.