LLaVA-Video Training Data With 178K Caption Entries

178,510 caption entries and 960,792 open-ended question-answer pairs were compiled by lmms-lab for training the LLaVA-Video model. This multimodal dataset aggregates video-language data from five primary sources. The dataset card was last updated in October 2024.

Use Cases

• Train video-language models on 178,510 caption entries for video captioning tasks.
• Fine-tune visual question answering models using the 960,792 open-ended question-answer pairs.
• Benchmark model performance on video-to-text generation tasks with the provided captions and QA data.

Strengths

• Contains 178,510 distinct caption entries.
• Includes 960,792 open-ended question-answer pairs for diverse training.

Limitations

• Specific row counts, column names, and data formats are not provided.
• Potential geographic or content bias from the five aggregated sources is unknown.
• Contains OpenAI GPT-4 generated data, requiring users to check OpenAI's usage policy.

Provenance

Source

lmms-lab, aggregating data from five primary video-language sources.

Collection Method

Compiled for training the LLaVA-Video model; includes GPT-4 generated data.

Time Range

null

Freshness

null

Geography

Region tag indicates 'us', but full spatial coverage is unspecified.