DreamLIP Long Captions with 30 Million Synthetic Annotations

DreamLIP-Long-Captions contains approximately 30 million image annotations consisting of detailed long captions. The captions were generated using pre-trained Multi-modality Large Language Models, with an average length of 247 characters.

Use Cases

• Train vision-language models on 30 million detailed long captions for image understanding tasks.
• Fine-tune text generation models using the long-form synthetic captions with an average length of 247 characters.
• Analyze the style and structure of machine-generated detailed descriptions for synthetic data research.
• Use the large-scale synthetic annotations for pre-training multimodal architectures like CLIP or BLIP.

Strengths

• Approximately 30 million image annotations provide a large-scale resource.
• Captions have an average length of 247 characters, offering detailed descriptions.
• Generated using multiple pre-trained Multi-modality Large Language Models (ShareGPT4V/InstructBLIP/LLava1.5).

Limitations

• All annotations are synthetic, which may introduce biases or artifacts not present in human-written captions.
• The underlying source images and their distribution are not described, limiting understanding of visual content coverage.
• The dataset's quality and coherence are dependent on the performance of the generative models used.

Provenance

Source

Author qidouxiong619 on Hugging Face.

Collection Method

Annotations generated synthetically using pre-trained Multi-modality Large Language Models (ShareGPT4V/InstructBLIP/LLava1.5).

Freshness

Last updated on 2024-09-23.