Multimodal Image-Text Pairs with Source Metadata

A collection of multimodal image-text pairs, with each sample including an image or image URL, associated text strings, a source identifier, and JSON-formatted metadata. The dataset was created by HuggingFaceM4 and was last updated in June 2022.

Use Cases

• Train a multimodal model using the `image` and `texts` fields for tasks like image captioning or text-to-image retrieval.
• Analyze the distribution of data `source` identifiers to understand the composition of aggregated datasets.
• Parse the JSON-formatted `meta` field to extract and utilize additional structured information for each sample.
• Filter samples based on the presence of `image_url` or `image` to create subsets for different processing pipelines.

Strengths

• Dataset structure is explicitly defined with a `datasets.Features` object, specifying data types for each field.
• Supports multiple associated `texts` per image, accommodating datasets like COCO.
• Includes a `meta` field for flexible storage of additional JSON-formatted information per sample.

Limitations

• The total number of rows, columns, and overall dataset size are unknown.
• The specific license, geographic coverage, and temporal range of the data are not provided.
• Sample data is unavailable, preventing assessment of data quality and content.

Provenance

Source

HuggingFaceM4

Collection Method

Aggregated from multiple sources, as indicated by the `source` field and reference to versions like PMD.

Freshness

Last updated on 2022-06-13.