Guidebench: 67.5 Hours of GUI Screen Recordings for User Intent Detection

GUIDE (GUI User Intent Detection Evaluation) is a benchmark for evaluating multimodal models on perceiving user behavior and inferring intent in open-ended GUI tasks. It consists of 67.5 hours of screen recordings from 120 novice user demonstrations with think-aloud narrations, across 10 software applications. The dataset was created by Saelyne Yang, Jaesang Yu, Yi-Hao Peng, Kevin, and others, and was last updated on Hugging Face in June 2026.

Use Cases

• Benchmarking multimodal AI models on their ability to infer user intent based on screen recordings and narration.
• Training models to provide real-time assistance in software applications based on observed user behavior.
• Studying novice user interaction patterns across a variety of common software tools.

Strengths

• Contains 67.5 hours of annotated screen recordings, providing substantial multimodal data.
• Includes think-aloud narrations from 120 novice users, offering direct insight into user intent.
• Covers demonstrations across 10 different software applications, suggesting task diversity.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count and file formats are unknown, which may limit suitability assessment.
• Data may reflect bias inherent to the specific user pool and software applications used.

Provenance

Source

Hugging Face (user: saelyne).

Collection Method

Likely collected from screen recordings and think-aloud narrations of 120 novice users performing tasks.

Freshness

Last updated 2026-06-01 21:05:25; freshness should be verified.