Web Screenshots with Annotated Instructions and Click Targets

1,639 English-language web screenshots from over 100 websites are paired with natural-language instructions and pixel-level click targets. The dataset provides a high-quality benchmark for evaluating multimodal navigation models, created by Hcompany and released in June 2025.

Use Cases

• Benchmarking model localization accuracy by predicting pixel-level click targets from natural-language instructions.
• Training multimodal agents to navigate web interfaces by learning from screenshot-instruction pairs.
• Evaluating instruction-following capabilities by measuring success rates on precise click actions across diverse websites.

Strengths

• 1,639 annotated web screenshots provide a substantial evaluation corpus.
• Data is precisely annotated with pixel-level click targets, enabling fine-grained evaluation.
• Screenshots sourced from over 100 distinct websites ensure diversity in web interfaces.

Limitations

• Sample size of 1,639 screenshots is modest compared to larger web-scale datasets.
• Limited to English-language websites and instructions, introducing a language bias.
• Dataset scope is restricted to static screenshots, lacking interactive or dynamic page elements.

Provenance

Source

Hcompany

Collection Method

Web screenshots paired with annotated natural-language instructions and click targets.

Freshness

Last updated June 2025.