Visual-Centric Instruction Following Dataset For MLLM Training

VCIF-10K provides data for training Multimodal Large Language Models on visual instruction following tasks. The dataset is structured in a messages format with user instructions and assistant responses, referencing images from sources like LLaVA-Instruct and Visual Genome. It was created by WoofWoof and supports both Supervised Fine-Tuning and Direct Preference Optimization training paradigms.

Use Cases

• Fine-tune MLLMs using the 'messages' format with 'user' and 'assistant' roles for visual instruction following.
• Implement Direct Preference Optimization (DPO) training using the dataset's structured prompt-response pairs.
• Train models to generate 'assistant' role 'content' based on multimodal 'user' inputs containing image references and text instructions.

Strengths

• Dataset is associated with a published research paper titled 'Empowering Reliable Visual-Centric Instruction Following in MLLMs'.
• Images are sourced from established multimodal datasets including LLaVA-Instruct, Visual Genome, and ALLaVA-4V.
• Data format is explicitly defined for SFT and DPO training methodologies.

Limitations

• The exact number of rows, columns, and total dataset size are unknown.
• Sample data and specific file formats are not provided for inspection.
• Potential label noise or biases may exist as the image sources are aggregated from other datasets.

Provenance

Source

Author WoofWoof on Hugging Face; images sourced from LLaVA-Instruct, Visual Genome, and ALLaVA-4V.

Collection Method

Created for the VC-IFEngine project, structured for SFT and DPO training of MLLMs.

Freshness

Last updated on 2026-03-24.