OmniPro: A Benchmark for Proactive Video Understanding in Multimodal LLMs

OmniPro is a benchmark designed to evaluate proactive video understanding capabilities in multimodal large language models. Unlike traditional reactive question-answering benchmarks, it tests a model's ability to monitor video streams and respond at the right moment when specific conditions are met. The dataset was created by RuixiangZhao and was last updated on May 19, 2026.

Use Cases

• Benchmarking proactive video monitoring capabilities based on the described evaluation paradigm.
• Training models to identify and respond to specific conditions in video streams as mentioned in the description.
• Evaluating the three core capabilities of omni-proactive models outlined in the dataset's design.

Strengths

• Designed around three core capabilities that define a good omni-proactive model.
• Focuses on proactive video understanding, a distinct task from traditional reactive QA.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality requires manual inspection after download.

Provenance

Source

RuixiangZhao on Hugging Face

Freshness

Last updated 2026-05-19 08:38:33; freshness should be verified.