8,334 episodes of graphical user interface (GUI) interaction data, released by hflqf88888 in August 2025. The dataset includes fine-grained low-level instructions, image descriptions, action intentions, and context reviews for each step. It is associated with a GitHub repository and an arXiv paper detailing its creation and purpose.
Use Cases
- Train multimodal agents for GUI navigation based on low-level instructions.
- Benchmark computer vision models on interpreting GUI screenshots from image descriptions.
- Develop models that predict user action intentions from visual and textual context.
- Study sequential decision-making in digital environments using step-by-step context reviews.
Strengths
- Contains 8,334 episodes, a larger scale than a previous version.
- Includes rich semantic annotations such as image descriptions and action intentions for each step.
Limitations
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
Provenance
- Source
- hflqf88888 on Hugging Face, associated with the OpenGVLab GitHub repository.
- Collection Method
- Likely gathered through automated or semi-automated recording of GUI interactions.
- Time Range
- null
- Freshness
- Last updated 2025-08-04 02:22:43; freshness should be verified.
- Geography
- null