DenseFusion-1M: 1 Million Granular Image Descriptions for MLLMs

DenseFusion-1M provides 1 million image-text pairs for multi-modal perception, released by the Beijing Academy of Artificial Intelligence (BAAI) in 2024. The dataset uses a Perceptual Fusion approach to combine outputs from specialized vision experts and GPT-4V into detailed descriptions.

Use Cases

• Training MLLMs for detailed image captioning using the 1 million image-text pairs
• Benchmarking Visual Question Answering (VQA) tasks using the question-answer pairs
• Fine-tuning vision-language models on granular scene descriptions generated via Perceptual Fusion

Strengths

• 1,000,000 image-text pairs
• CC BY 4.0 open license
• Expert-integrated annotations using GPT-4V

Limitations

• Potential synthetic bias from GPT-4V generation
• Unknown specific column schema in initial metadata

Provenance

Source

Beijing Academy of Artificial Intelligence (BAAI)

Collection Method

Synthetic/Annotated (MLLM + Vision Experts)

Freshness

Last updated October 2024