LLaVA-OneVision Multimodal Instruction Data

A 2024-09-01 upload of filtered VisualWebInstruct data for the OneVision training stage. The dataset, created by lmms-lab, contains subsets like ureader_kg and ureader_qa, provided as processed JSON files and compressed image folders.

Use Cases

• Training models for visual question answering using the question-answer pairs in the ureader_qa subset.
• Enhancing model knowledge grounding by leveraging the structured knowledge graphs in the ureader_kg subset.
• Fine-tuning vision-language models on instruction-following tasks using the VisualWebInstruct filtered data.
• Benchmarking model performance on multimodal reasoning with provided image-text instruction pairs.

Strengths

• Contains data specifically filtered and prepared for the OneVision training stage, indicating curated relevance.
• Includes multiple structured subsets (e.g., ureader_kg, ureader_qa) for different training objectives.

Limitations

• Unknown total size, row count, and specific column schema for the datasets.
• Potential class or topic imbalance inherent in filtered web-sourced instruction data.

Provenance

Source

lmms-lab, hosted on Hugging Face.

Collection Method

Processed from the VisualWebInstruct dataset and other sources, formatted for model training.

Time Range

Data uploaded for use in 2024.

Freshness

Last updated on the platform 2025-05 24; core data uploaded 2024-09-01.

Geography

null