Rico Screen2Words: Mobile UI Screen Summaries

Screen2Words provides image captions for mobile application screens. It is built upon the RICO mobile app image database. The dataset was uploaded by rootsautomation to Hugging Face in April 2024.

Use Cases

• Train automatic captioning models for mobile UI screens based on the image-summary pairs.
• Benchmark multimodal learning algorithms for understanding graphical user interfaces.
• Develop accessibility tools that generate screen descriptions for visually impaired users based on the provided summaries.

Strengths

• Built on the established RICO mobile app dataset, suggesting a foundation of real-world UI images.
• Focuses on a specific task of UI summarization, which is a defined research area.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

RICO image database; repository google-research-datasets/screen2words.

Collection Method

Likely contains images paired with textual summaries, gathered for research purposes.

Freshness

Last updated 2024-04-16 18:54:01; freshness should be verified.