LLaVA-NeXT: 100K-1M Multimodal Instruction Tuning Pairs

LLaVA-NeXT Data contains between 100,000 and 1,000,000 instruction-tuning pairs for multimodal large language models, released by lmms-lab in August 2024. It provides the specific data mixtures used to train the LLaVA-NeXT and LLaVA-NeXT (stronger) models, featuring synchronized image and text instruction sets.

Use Cases

• Fine-tuning multimodal models using the provided image-text instruction pairs
• Benchmarking vision-language models against the LLaVA-NeXT training distribution
• Researching instruction-following capabilities in large vision-language models

Strengths

• Contains between 100,000 and 1,000,000 records
• Includes raw JSON and image formats for direct compatibility with LLaVA training pipelines
• Used to train state-of-the-art open-source multimodal models

Limitations

• Specific column names and schema are not detailed in the provided metadata
• Large storage requirements due to high-volume image data

Provenance

Source

lmms-lab

Collection Method

Curated instruction tuning data mixture

Freshness

Last updated August 30, 2024, reflecting the most recent LLaVA-NeXT training iterations.