Paraphrase Robustness Benchmark for Vision-Language-Action Models

LIBERO-Para is a controlled diagnostic benchmark for evaluating the paraphrase robustness of Vision-Language-Action models in robotic manipulation. It was created by HAI-Lab and was last updated in April 2026. The benchmark independently varies action expressions and object references to analyze how linguistic variation affects model performance.

Use Cases

• Evaluating model sensitivity to variations in action expressions within manipulation instructions.
• Measuring robustness to changes in object references in vision-language-action tasks.
• Conducting fine-grained analysis of how different linguistic components affect VLA performance.
• Benchmarking paraphrase robustness using controlled variations of core instruction elements.

Strengths

• Designed as a controlled benchmark enabling fine-grained analysis of linguistic variation.
• Focuses on two core linguistic components: action expressions and object references.

Limitations

• The specific dataset size, number of tasks, and sample counts are unknown.
• The benchmark's scope is limited to paraphrase robustness for a specific model type (VLA).

Provenance

Source

HAI-Lab

Collection Method

Created as a controlled diagnostic benchmark for research purposes.

Freshness

Last updated in April 2026.