LVBench: Long Video Understanding Benchmark with Two-Hour Durations

LVBench is a benchmark for long video understanding featuring videos up to two hours in duration, released by zai-org in June 2024. It contains approximately 1,000 records designed to evaluate multimodal models on visual question answering and multiple-choice tasks. The dataset addresses the challenge of extracting information from extended temporal windows that exceed standard video benchmarks.

Use Cases

• Evaluating multimodal model performance on long-form video comprehension using multiple-choice questions
• Testing information extraction capabilities across two-hour video durations
• Benchmarking visual question answering (VQA) systems on temporal reasoning

Strengths

• Includes videos with durations up to two hours
• CC BY-NC-SA 4.0 licensed for research use
• Features approximately 1,000 annotated records for benchmarking

Limitations

• Small sample size of approximately 1,000 records
• Non-commercial license restriction (CC BY-NC-SA 4.0)
• High computational requirements for processing two-hour video files

Provenance

Source

zai-org (ArXiv: 2406.08035)

Freshness

Released June 2024; last updated June 13, 2024.