Web-Harvested Image Captions with Alt-Text Descriptions

Conceptual Captions contains approximately 3.3 million images paired with captions. The captions are raw descriptions harvested from the Alt-text HTML attribute of web images, representing a wider variety of styles than curated annotations. The dataset was created by google-research-datasets.

Use Cases

• Train an image captioning model on 3.3M image-caption pairs to generate descriptive text from visual content.
• Analyze the stylistic variety of Alt-text HTML attribute descriptions compared to curated caption datasets.
• Fine-tune a multimodal model using web-harvested images and their associated text for vision-language tasks.

Strengths

• Approximately 3.3 million image-caption pairs provide a substantial scale for training.
• Captions are harvested from web Alt-text, offering a wider variety of descriptive styles than curated datasets.
• The dataset is designed for the image captioning task, providing direct supervision for vision-language models.

Limitations

• The raw, web-harvested Alt-text descriptions may contain noise, errors, or irrelevant captions.
• Specific image and caption quality metrics, column details, and data size are not provided in the input.
• Potential geographic or content bias exists as images and text are harvested from the web without specified curation.

Provenance

Source

google-research-datasets

Collection Method

Images and raw descriptions were harvested from the web, specifically from the Alt-text HTML attribute associated with web images.

Freshness

Last updated on the platform was 2024-06-17.

Geography

Region tag indicates 'us', but the web-harvesting method suggests potentially global coverage.