Video Summarization Dataset with Most Replayed Viewer Engagement Scores

52,678 in-the-wild videos feature synchronized visual, audio, and text data. Ground-truth importance scores are derived from YouTube's 'Most Replayed' statistics, reflecting collective viewer engagement. The dataset was created by author hminjeong and was last updated in March 2026.

Use Cases

• Train models to predict per-frame importance scores from synchronized visual, audio, and text features.
• Develop adaptive triple-modality fusion algorithms for video summarization.
• Benchmark video summarization models using reliable ground-truth based on viewer engagement patterns.
• Analyze correlations between specific audio features, visual scenes, and viewer replayed segments.

Strengths

• 52,678 videos provide a substantial scale for training.
• Ground-truth annotations from YouTube's 'Most Replayed' offer high reliability.
• Synchronized visual, audio, and text features enable multimodal analysis.

Limitations

• Dataset size and specific column details are unknown.
• Geographic and content bias may exist as videos are sourced from YouTube.
• Temporal coverage of the source videos is unspecified.

Provenance

Source

YouTube videos.

Collection Method

Annotations derived from YouTube's 'Most Replayed' statistics.

Freshness

Last updated March 2026.

Geography

Region 'us' tag suggests a potential US focus, but coverage is unspecified.