LLaVA-Plus: GPT-Generated Multimodal Tool-Augmented Instructions

LLaVA-Plus-v1-117K is a set of 117,000 GPT-generated multimodal tool-augmented instruction-following data points. It was collected in September 2023 by prompting the ChatGPT/GPT-4-0314 API to build large multimodal agents with vision and language capabilities. The dataset was created by the LLaVA-VL organization.

Use Cases

• Train multimodal agents for tool use based on GPT-generated instruction-following data.
• Benchmark vision-language models on tool-augmented tasks based on the described multimodal instructions.
• Develop and evaluate large language models with integrated vision capabilities based on the multimodal agent construction goal.

Strengths

• Contains 117,000 instruction-following data points.
• Data was generated using the GPT-4-0314 API, suggesting a high-quality source model.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is known but other scale metrics like file size and formats are unknown, which may limit suitability assessment.
• Last updated 2023-11-09 16:04:57; freshness should be verified.

Provenance

Source

LLaVA-VL

Collection Method

Generated by prompting ChatGPT/GPT-4-0314 API.

Time Range

Collected in September 2023.

Freshness

Last updated 2023-11-09 16:04:57.