Infinity-MM: 10-100 Million Multimodal Bilingual Instruction Samples

Infinity-MM provides between 10 million and 100 million multimodal instruction samples, developed by the Beijing Academy of Artificial Intelligence (BAAI) in 2024. It utilizes a synthetic generation pipeline to create detailed image annotations and diverse question-answer pairs for bilingual model training in English and Chinese.

Use Cases

• Training multimodal large language models (MLLMs) using the instruction-following pairs
• Fine-tuning image-to-text systems for detailed scene description using synthetic annotations
• Developing bilingual visual question answering (VQA) models for English and Chinese

Strengths

• Scale of 10 million to 100 million samples
• Bilingual support for English and Chinese
• Quality filtering and deduplication applied to the large-scale collection

Limitations

• Heavy reliance on synthetic data generation from open-source models which may introduce model-specific biases
• Potential for model-generated hallucinations in the detailed image annotations

Provenance

Source

Beijing Academy of Artificial Intelligence (BAAI)

Collection Method

Synthetic data generation using open-source models and a labeling system combined with quality filtering and deduplication

Freshness

Last updated December 2024

Geography

Global (Bilingual English and Chinese)