InpaintCOCO: Fine-Grained Multimodal Concept Understanding Dataset

A dataset containing paired images and captions designed for fine-grained multimodal concept understanding. Each data sample contains two images and two corresponding captions that differ only in one object, the color of an object, or the size of an object. The dataset was created by author 'phiyodr' and was last updated on October 2, 2024.

Use Cases

• Benchmarking fine-grained vision-language retrieval models based on object, color, and size differences.
• Training models for fine-grained visual question answering based on nuanced concept distinctions.
• Evaluating multimodal model sensitivity to specific attribute changes mentioned in the description.

Strengths

• Dataset structure is explicitly defined, with each sample containing two images and two captions differing in specific attributes.
• Focuses on fine-grained distinctions in object, color, and size, which are described as overlooked in overall performance metrics.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

huggingface

Freshness

Last updated 2024-10-02 17:46:36; freshness should be verified.