Synthux Economy R6 W074: Simulated Desktop Interaction Trajectories

SynthUX generated grounded visual computer-use trajectories within a simulated company environment. Each record captures one worker's device session, where a goal expands into a node tree and terminal nodes drive real desktop-simulator applications through low-level mouse and keyboard input. The observed trajectory is recorded as a screen video plus per-node frames, authored by jacob-valdez and last updated on 2026-05-31.

Use Cases

• Train or evaluate AI agents for desktop task automation based on low-level mouse and keyboard input sequences.
• Analyze user workflow patterns and goal decomposition based on the node tree structure of tasks.
• Develop computer vision models for screen understanding using the recorded screen video and per-node frames.
• Study human-computer interaction in office productivity software suites (e.g., VS Code, Browser, Slack, Sheets).

Strengths

• Data includes screen video recordings, providing a rich visual record of interactions.
• Trajectories are grounded in a simulated company environment with specific goals, suggesting contextual realism.
• Interactions span multiple common desktop applications (Terminal, Notes, VS Code, Browser, Slack/Teams, Mail, Sheets, Slides).

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Collection Method

Generated by the SynthUX system simulating worker sessions in a company environment.

Freshness

Last updated 2026-05-31 05:33:39; freshness should be verified.