Chinese Android GUI Benchmark for Agent Evaluation

CAGUI is a benchmark dataset for evaluating GUI agent models on Chinese Android applications. It focuses on two capabilities: grounding GUI components to semantics and planning multi-step actions to complete user goals. The dataset was created by openbmb and last updated in June 2025.

Use Cases

• Evaluate model performance on the grounding task of linking GUI widgets to OCR text.
• Benchmark agent models on the multi-step action planning task for completing user goals.
• Train or test models on real-world Chinese Android GUI data for automation tasks.

Strengths

• Designed for two complementary evaluation tasks: grounding and agent capabilities.
• Focuses on real-world Chinese Android applications.
• Last updated in June 2025, indicating recent maintenance.

Limitations

• Specific dataset size, row count, and column details are unknown.
• Sample data and file formats are unavailable for inspection.
• The dataset is specialized for Chinese Android GUIs, limiting generalizability to other platforms or languages.

Provenance

Source

huggingface

Collection Method

Designed as a benchmark for evaluating GUI agent models; specific collection method unknown.

Freshness

Last updated on 2025-06-14.

Geography

China (focus on Chinese Android applications)