LongVideoBench: 3,763 Hour-Long Videos for Multimodal QA

3,763 web-collected videos with subtitles and multiple-choice questions comprise this long-context multimodal benchmark. Created for NeurIPS 2024, it evaluates large multimodal models on video-language interleaved inputs with durations reaching up to one hour.

Use Cases

• Evaluating long-context visual question answering using the multiple-choice question labels
• Testing temporal reasoning in LMMs by analyzing interleaved video frames and subtitle text
• Benchmarking multimodal retrieval across hour-long video durations using the provided ground-truth answers

Strengths

• 3,763 video samples
• Video durations up to 60 minutes
• NeurIPS 2024 peer-reviewed benchmark
• Interleaved video and subtitle modalities

Limitations

• Restricted to multiple-choice question formats
• Potential web-source bias in video content
• High computational requirements for processing hour-long video inputs

Provenance

Source

NeurIPS 2024 (Arxiv 2407.15754)

Collection Method

web-collected

Freshness

Last updated October 2024.

Geography

global