Guardian FailCoT: Real-Robot Failure Detection Benchmarks

Guardian FailCoT bundles three real-world failure-detection benchmarks used to evaluate the Guardian vision-language model. The collection includes the UR5-Fail, RoboFail, and RoboVQA datasets. It was created by paulpacaud and last updated in May 2026.

Use Cases

• Evaluating vision-language models for failure detection based on real-robot manipulation data.
• Benchmarking cross-environment failure reasoning in robotics based on the three bundled datasets.
• Training models to identify manipulation failures from multi-view and single-view robot camera feeds.

Strengths

• Includes three distinct real-robot benchmarks (UR5-Fail, RoboFail, RoboVQA) for comprehensive evaluation.
• Specifically designed for evaluating the Guardian model as described in the 2026 paper by Pacaud et al.
• Last updated on 2026-05-05, indicating recent maintenance.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and dataset size are unknown, which may limit suitability assessment.
• Data may reflect bias inherent to the specific robot platforms and environments used.

Provenance

Source

paul pacaud on Hugging Face, aggregating benchmarks from multiple research papers.

Collection Method

Likely collected from real-robot manipulation experiments, as described for UR5-Fail and sourced from prior publications for RoboFail and RoboVQA.

Freshness

Last updated 2026-05-05 12:19:57; freshness should be verified.