ShareGPT4Video: 4.8 Million GPT-4V Generated Video Captions

ShareGPT4Video provides 4.8 million multi-modal video captions generated via GPT-4-Vision to improve modality alignment in Large Video-Language Models. Developed by the ShareGPT4Video team in 2024, the collection includes a specific 40,000-record subset for fine-grained visual perception tasks.

Use Cases

• Training Large Video-Language Models (LVLMs) using the 4.8M multi-modal captions
• Enhancing Text-to-Video Models (T2VMs) using fine-grained visual concept perception data
• Visual Question Answering (VQA) tasks using the sharegpt4video_40k.jsonl subset

Strengths

• 4.8 million video-caption pairs
• GPT-4-Vision generated annotations
• Includes a curated 40,000-row subset for high-quality alignment
• CC BY-NC 4.0 licensed

Limitations

• Synthetic label noise from GPT-4V generation
• Non-commercial license restriction
• Potential geographic or cultural bias in source video selection

Provenance

Source

ShareGPT4Video (Arxiv 2406.04325)

Collection Method

Synthetic annotation using GPT-4-Vision

Freshness

Last updated March 2025