Synthux Economy R6 W072: Simulated Desktop Computer-Use Trajectories

Grounded visual computer-use trajectories generated by SynthUX simulate worker device sessions within a simulated company. Each record captures a worker's session, where a goal expands into a node tree and terminal nodes drive real desktop-simulator applications through low-level mouse and keyboard input. The observed trajectory is recorded as a screen video plus per-node frames.

Use Cases

• Train or evaluate AI agents for desktop task automation based on simulated mouse and keyboard input sequences.
• Study human-computer interaction patterns based on recorded screen videos and per-node frames.
• Develop models for workflow prediction based on the described goal-to-node-tree expansion structure.
• Benchmark multimodal models that integrate visual screen data with low-level input events.

Strengths

• Data includes screen video recordings, providing a rich visual trace of user activity.
• Trajectories are grounded in a simulated company environment with specific goals, suggesting structured task contexts.
• Interactions drive real desktop-simulator applications like Terminal, VS Code, and Slack, indicating realistic software use.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Row count is unknown, which may limit suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.

Provenance

Source

jacob-valdez on Hugging Face.

Collection Method

Generated by the SynthUX system simulating worker device sessions.

Freshness

Last updated 2026-05-31 05:17:22; freshness should be verified.