Sign in to view source links and access this dataset
Description
Hcompany's DragOn dataset provides a benchmark for GUI agents, created by Hcompany and last updated on June 4, 2026. It contains over 100,000 training images and 1,000,000 training tasks across four domains: text highlighting, spreadsheet cell selection, slide element manipulation, and slider interaction. Each example pairs a screenshot with an instruction and bounding box coordinates for a drag action.
Use Cases
Training visual grounding models for GUI agents based on screenshot-instruction pairs.
Benchmarking agent performance on drag-based tasks across the four specified domains.
Developing models for cross-domain GUI interaction, such as transferring skills from text highlighting to slide resizing.
Strengths
Contains over 100,000 training images and 1,000,000 training tasks, indicating substantial scale.
Covers four distinct GUI interaction domains: text_highlight, sheet, slide_resize, and slider.
Includes a public evaluation set of 250 tasks for benchmarking.
Limitations
Column-level documentation is absent; field semantics must be inferred after download.
Row count for the full dataset is unknown, which may limit suitability assessment.
Description metadata is limited; actual data quality requires manual inspection after download.
Provenance
Source
Hcompany via Hugging Face.
Collection Method
Likely contains synthetic or programmatically generated GUI interactions.
Freshness
Last updated 2026-06-04 09:49:41; freshness should be verified.
License is unknown; terms of use must be verified before application.