VitaSet: 5,145 Vision-Tactile QA Pairs for Physical Property Reasoning

VitaSet is a multimodal dataset for physical property reasoning, combining RGB vision and tactile sensing. It contains 5,145 human-verified question-answer pairs across three tasks: hardness classification, material property description, and surface roughness classification. The dataset was created by Bupt-Joy and last updated on 2025-12-29.

Use Cases

• Train models for hardness classification based on combined vision and tactile data.
• Develop systems for material property description from multimodal sensor inputs.
• Classify surface roughness using vision-tactile sensor fusion.
• Benchmark multimodal reasoning models for physical interaction tasks.

Strengths

• 5,145 human-verified QA pairs provide a substantial base for model training.
• Covers three distinct reasoning tasks: hardness, material property, and roughness classification.
• Data is multimodal, combining RGB vision and tactile sensing from a Franka Emika Panda robot and GelSight Mini sensor.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known, but other scale details like file formats and size are unknown.
• Data may reflect the specific biases inherent to the collection hardware and methodology.

Provenance

Source

Bupt-Joy via Hugging Face.

Collection Method

Collected using a Franka Emika Panda robot equipped with a GelSight Mini tactile sensor.

Time Range

null

Freshness

Last updated 2025-12-29 20:47:05.

Geography

null