Robo2VLM: 100K-1M VQA Pairs from Robot Manipulation Trajectories

Robo2VLM 1 provides between 100,000 and 1,000,000 visual question-answering records derived from real-world robot manipulation trajectories. Created by researcher keplerccc and updated in late 2025, the dataset uses multi-modal robot data to enhance scene understanding in vision-language models. It bridges the gap between internet-scale image-text corpora and specific robotic visuomotor policies.

Use Cases

• Training Vision-Language Models (VLMs) to reason about robotic manipulation scenes using image and text pairs
• Benchmarking visual question answering performance specifically for robotic task planning
• Developing visuomotor policies that integrate natural language scene understanding

Strengths

• Scale of 100,000 to 1,000,000 records
• Apache 2.0 permissive license
• Derived from real-world 'in-the-wild' robot manipulation trajectories

Limitations

• Lack of explicit column documentation in the primary metadata
• Potential for automated label noise if VQA pairs were synthetically generated from trajectory logs

Provenance

Source

Arxiv paper 2505.15517

Collection Method

Extracted from large-scale in-the-wild robot manipulation datasets

Freshness

Last updated October 2025.