Instructional Video Narrations Across 23,000 Activities

HowTo100M contains 136 million narrated video clips sourced from 1.2 million YouTube instructional videos spanning 15 years. The dataset focuses on videos where creators teach complex tasks, covering 23,000 activities in domains like cooking, crafting, and fitness.

Use Cases

• Train a video-language model on the alignment between video clips and their associated narration subtitles.
• Develop activity recognition models across the 23,000 activity categories, such as cooking or gardening steps.
• Analyze temporal grounding by linking specific narration phrases to visual events within the instructional video clips.
• Build a retrieval system for instructional content using the narrated subtitles as a dense text corpus.

Strengths

• Large scale of 136 million video clips provides substantial data for training.
• Broad coverage of 23,000 distinct instructional activities across multiple domains.
• Longitudinal collection representing 15 years of video content from YouTube.

Limitations

• Narration subtitles are automatically downloaded from YouTube, which may introduce transcription errors or noise.
• Dataset scale may necessitate significant computational resources for processing and model training.
• Potential bias towards activities and languages most prevalent on the YouTube platform.

Provenance

Source

YouTube

Collection Method

Automated collection of narrated instructional videos and their subtitles.

Time Range

15 years of video content

Freshness

2022-05-18