IsaacLab SO101 PullCube: 100 Episodes of Robot Manipulation with Skill Labels

100 episodes of robot manipulation data, totaling 32,208 frames, collected in the Isaac Sim 5.1 simulation environment. The dataset was generated by the CoRL2026-CSI team using a Code-as-Policies replay pipeline for the task of pulling a cube to a target marker. Each frame includes RGB camera views and per-frame natural-language skill labels generated by the Gemini model.

Use Cases

• Train imitation learning models based on the recorded robot trajectories and camera frames.
• Develop or evaluate vision-language models for robotics based on the per-frame natural-language skill annotations.
• Benchmark robotic manipulation policies in simulation using the provided task and episode structure.

Strengths

• Includes 32,208 frames of synchronized RGB data from two camera angles.
• Provides 100 complete task episodes for consistent training and evaluation.
• Features per-frame natural-language skill labels, which likely adds semantic context to the robot actions.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment for large-scale training.
• Data may reflect simulation bias inherent to the IsaacLab environment and the specific task.

Provenance

Source

CoRL2026-CSI via Hugging Face

Collection Method

Collected via SCRAPE-IsaacLab, a Code-as-Policies replay pipeline running inside Isaac Sim 5.1 / IsaacLab 2.3.2.

Freshness

Last updated 2026-06-02 10:49:01; freshness should be verified.