PE Video Dataset: 1 Million Diverse Videos with 120,000 Annotations

Meta released the PE Video Dataset (PVD) in April 2025, featuring 1 million high-quality videos for perception encoding research. The collection includes 120,000 clips with human-verified annotations, while the full set is accompanied by descriptions and keywords.

Use Cases

• Training video captioning models using the video description field
• Developing video retrieval systems based on the keywords metadata
• Fine-tuning perception encoders using the 120,000 human-verified annotations

Strengths

• 1 million high-quality video records
• 120,000 human-verified and expert-annotated clips
• Dual metadata including both descriptions and keywords

Limitations

• CC BY-NC 4.0 license restricts commercial application
• Potential noise in automated annotations for the 880,000 non-verified clips

Provenance

Source

Meta (Facebook), introduced in 'Perception Encoder' (Arxiv 2504.13181)

Collection Method

Automated collection with human-verification for a 120k subset

Freshness

Last updated April 2025.

Geography

Global