Synthux Economy R6 W063: Simulated Desktop Computer-Use Trajectories

Grounded visual computer-use trajectories generated by SynthUX. Each record captures one worker's device session inside a simulated company, where a goal expands into a node tree driving real desktop-simulator apps through low-level mouse/keyboard input. The observed trajectory is recorded as a screen video plus per-node frames.

Use Cases

• Train or evaluate AI agents for desktop task automation based on low-level mouse and keyboard input sequences.
• Study human-computer interaction patterns within a simulated multi-application office environment.
• Develop models for goal decomposition and task planning based on the described node-tree structure.
• Benchmark computer vision models for screen understanding using the recorded screen videos and per-node frames.

Strengths

• Trajectories are grounded in a simulated multi-application desktop environment, likely providing realistic context.
• Data captures low-level user interactions (mouse/keyboard) driving applications like Terminal, VS Code, and Slack.
• Records include both screen video and per-node frames, offering multiple modalities for analysis.

Limitations

• Description metadata is limited; actual data quality, scale, and column definitions require manual inspection after download.
• Row count and file formats are unknown, which may limit suitability assessment for large-scale projects.

Provenance

Source

huggingface user jacob-valdez

Collection Method

Generated by the SynthUX system simulating user interactions within a desktop environment.

Freshness

Last updated 2026-05-31 03:59:58; freshness should be verified.