Computer Use Data PSAI: 3,167 Multimodal HCI Task Recordings

3,167 completed human-computer interaction tasks featuring video, screenshots, and DOM snapshots, released by Paradigm Shift AI in 2025. This multimodal collection captures granular interaction events to support the development of AI agents capable of navigating graphical user interfaces.

Use Cases

• Training GUI agents using DOM snapshots and interaction event sequences
• Video classification of human-computer interaction patterns
• Visual Question Answering (VQA) based on desktop screenshots and UI elements

Strengths

• 3,167 completed interaction tasks
• Multimodal coverage including video, screenshots, and DOM snapshots
• Granular interaction event logging
• MIT licensed for open research

Limitations

• Small sample size of 3,167 tasks compared to web-scale interaction datasets
• Specific software environments and task diversity are not explicitly enumerated in the metadata

Provenance

Source

Paradigm Shift AI

Collection Method

Captured human-computer interactions

Freshness

Last updated October 2025.

Geography

United States