TRECVID Video Captions from 2016 to 2024

73,893 short videos from the TRECVID VTT task, each ranging from 3 to 10 seconds in duration. The dataset includes between 2 and 5 human-written captions per video, created by dedicated annotators hired by NIST.

Use Cases

• Train video-to-text models on 73,893 short videos with multiple captions per video for caption generation.
• Benchmark video retrieval systems using the human-written captions as queries or ground truth.
• Analyze caption diversity and consistency across the 2 to 5 captions provided for each video.

Strengths

• 73,893 videos provide a substantial collection for training and evaluation.
• Multiple captions (2-5) per video offer varied textual descriptions for each visual sequence.
• Data spans a long temporal range from 2016 to 2024, capturing diverse content.

Limitations

• Video duration is limited to short clips (3-10 seconds), not suitable for modeling long-term temporal dynamics.
• The specific source and content diversity of the videos are not described, which may introduce bias.
• Lack of column or metadata details limits structured analysis beyond video and caption files.

Provenance

Source

National Institute of Standards and Technology (NIST)

Collection Method

Videos and captions were created for the TRECVID Video-to-Text (VTT) task, with captions written by dedicated hired annotators.

Time Range

2016 to 2024

Freshness

Data includes content up to 2024 and was last updated in March 2025.