RUOXUAN123 created a dataset for evaluating Theory of Mind capabilities in vision-language models. It contains video and annotation data collected from the ThreeDWorld and VirtualHome simulators. The dataset was last updated on May 15, 2026.
Use Cases
- Benchmarking vision-language models' Theory of Mind reasoning based on video and annotation data
- Training models to infer mental states from simulated visual scenarios described in the dataset
- Developing evaluation metrics for cognitive AI tasks using data from the ThreeDWorld simulator
- Comparing model performance across different simulated environments like VirtualHome
Strengths
- Data is collected from two distinct simulation platforms, ThreeDWorld and VirtualHome, suggesting a multi-source approach
- Each sample from the ThreeDWorld simulator includes both a video and corresponding ground-truth annotation
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download
- Column-level documentation is absent; field semantics must be inferred after download
- Row count is unknown, which may limit suitability assessment
Provenance
- Source
- Data collected from ThreeDWorld and VirtualHome simulators.
- Collection Method
- Data in ThreeDWorld was collected through manual keyboard control.
- Freshness
- Last updated 2026-05-15 03:51:06; freshness should be verified