ShowUI-desktop-8K: 8,000 PC Screenshots with GPT-4o Augmented Annotations

ShowUI-desktop-8K consists of approximately 8,000 PC-based UI grounding records featuring screenshots and annotations originally sourced from OmniAct. Created by showlab and updated in March 2025, the dataset provides visual and textual data for desktop interface interaction research. It utilizes GPT-4o to augment original labels with detailed attributes regarding appearance and functionality.

Use Cases

• UI element localization using screenshot images and spatial relationship labels
• Functional intent prediction based on augmented functionality attributes
• Visual grounding research using appearance-based descriptions of UI components

Strengths

• 8,000 annotated records
• GPT-4o augmented labels for appearance and functionality
• Derived from the established OmniAct source

Limitations

• Potential synthetic label noise from GPT-4o augmentation
• Limited to PC-based desktop environments, excluding mobile UI patterns

Provenance

Source

OmniAct and showlab (arXiv:2411.17465)

Collection Method

GPT-4o augmentation of existing OmniAct screenshots and annotations

Freshness

Last updated March 2025.