Multimodal Video Highlight Dataset for Summarization

TripleSumm-Mr.HiSum reconstructs the original MR.HiSum dataset by crawling source videos to provide aligned visual, audio, and text features. The dataset supports multimodal research for video highlight detection and summarization. It was created by hminjeong and updated in March 2026.

Use Cases

• Train models to detect video highlights using aligned visual, audio, and text features.
• Develop multimodal summarization systems that integrate features from the three distinct modalities.
• Benchmark the performance of video understanding models on the Most-replayed Highlight Detection (MR.HiSum) task.
• Research cross-modal alignment and fusion techniques using the independently extracted feature sets.

Strengths

• Provides features across three distinct modalities: Visual, Audio, and Text.
• Reconstructed from original videos to support multimodal research, addressing a limitation of the unimodal original.

Limitations

• The specific number of videos, rows, or feature dimensions is unknown.
• Potential geographic or content bias is unknown as the original metadata and source videos are not fully described.

Provenance

Source

Reconstructed from the original MR.HiSum dataset by independently crawling source videos.

Collection Method

Crawling of original videos using provided metadata and subsequent feature extraction.

Freshness

Last updated March 2026.

Geography

Region tag suggests 'us', but specific coverage is unknown.